Creating a recommender system on the web using user profiles and machine learning methods

Number of pages: 85 File Format: word File Code: 31033
Year: 2014 University Degree: Master's degree Category: Computer Engineering
  • Part of the Content
  • Contents & Resources
  • Summary of Creating a recommender system on the web using user profiles and machine learning methods

    Computer Engineering Master's Thesis

    Abstract

    Web development that lacks a unified structure creates many problems for users. Not finding the information needed by users in this huge warehouse is one of the problems of web users. In order to deal with these problems, web personalization systems have been provided, which are able to provide suggestions according to the users' interests by finding user behavior patterns without their explicit request. Therefore, nowadays, it is necessary to have a recommender system that can automatically provide recommendations to the current user based on the patterns discovered from user navigation. Recently, web mining methods are used to personalize the web. In the meantime, application-based web mining techniques have been presented in order to discover user behavior patterns, and these techniques can implicitly extract user behavior patterns by using web server records. In this research, a method for creating a user profile is presented, which uses web mining based on the application to create a movement pattern for users using neural networks in order to predict the user's future requests and then produce a list of the user's favorite pages. The research results show that the proposed system is more accurate than previous systems.

    Key words: user profile, neural network, clustering, application-based web mining. The structures used in the web are non-integrated and complex structures, and users often do not achieve the purpose of their query when interacting with this huge amount of information. On the other hand, e-commerce is also expanding rapidly and the fact that users do not meet their needs has been noticed more than before. In the meantime, with the help of web personalization techniques, users' needs can be provided automatically without their explicit request. In fact, the goal of web personalization systems is to discover the behavior and movement patterns of users interacting with the web, so that they can predict the user's next move based on these patterns and help him reach the goal of his inquiry. In order to achieve this goal, web mining methods have been taken into consideration and these methods are used in the production of web personalization systems. These techniques are divided into three categories: content-based web mining, structure-based web mining, and application-based web mining. Among these methods, application-based web mining techniques have been widely used to discover user movement and behavioral patterns. These techniques take advantage of the information in the web server's log files [1] to generate user patterns. And in order to build recommender systems, these behavioral patterns of users can be used.

    1-1-          Statement of the problem

    Recently, the web has become a great source of information, which has caused problems for users with the increase in the use of the Internet as well as the increase in websites. One of the problems faced by users is to find the information they like or need in this massive amount of information. Meanwhile, the competition created in the e-commerce sector also requires the creation of websites that users like. For this reason, web personalization methods are proposed in order to solve these problems so that with the help of these methods, websites of interest and in accordance with users' needs can be created. The goal of web personalization is to produce dynamic suggestions based on user behavior patterns.  Recently, web mining techniques are used to discover user behavior patterns. Among the web mining techniques, the application-based web mining technique is widely used to discover user behavior patterns and model these behaviors. This technique uses the data in the server log files, in fact, without the explicit request of the users, application-based web mining techniques are able to extract the user interests from the information in the log files and model the user's behavior based on them and generate their behavior patterns.

    Many research works have been done in this field.In this research, it is suggested to use the application of web mining and neural networks for this purpose in order to be able to predict the future requests of the user and then generate a list of the user's favorite pages. In other words, it is possible to obtain a detailed profile[2] of users' behavior and predict the page that the user will choose in the next move. These dynamic suggestions can be the links [3] of the pages.

    In this thesis, by using the application-based web mining method and considering the features that accurately determine the user's behavior, user profiles are created, then the user's movement patterns are obtained with the clustering technique. In other words, by combining clustering and features such as page view history, the best profile that describes user behavior can be produced. After finding the movement patterns of the users, the suitable pattern for the user will be found using the neural network and suitable suggestions will be generated for the user's future requests. 1-2- Necessity of research The increasing growth of the web has led to the growth of information, which has created problems for web users. Web users cannot easily find their purpose and the information they need on the web. And in fact, in a way, the excess of data and the mass volume of data on the web causes such problems.

    According to the widespread use of the Internet all over the world, the need for tools that can manage the web is felt more than ever. In the meantime, web personalization systems have been provided in order to manage web pages and provide users' needs. Due to the competitive environment created and the increase in research in the field of recommender systems, today website managers need systems that can improve the interactions and interactions of users with the web until the lifetime of customers on websites is increased, and this issue will help both website managers and users because they will achieve their search goals easily.  

    In this research, a method is proposed in order to work in targeted advertising and e-commerce. In this method, by using application-based web mining and combining clustering with history feature and using neural network, a system will be presented that has the ability to predict the future requests of users and finally be able to produce a list of the user's favorite pages.

    1-3- Using application-based web mining techniques, the knowledge needed to predict the next page can be extracted. And by means of this knowledge, he created a profile for the users and by means of this profile he obtained the movement patterns of the users. And by having users' movement patterns, the task of predicting the next page can be realized. The objectives of the research are:

    Creating a profile of users in order to predict their behavior on the web

    Improving user interactions with the web

    Providing the information needed by users without their explicit request

    1-4-         Research structure

    The structure of the research is as follows:

    Chapter Two: In this chapter, the preliminary concepts and work background are mentioned. Concepts such as: web mining and its types, web personalization, clustering and related algorithms, and finally neural networks are mentioned.

    Chapter three: In this chapter, the works done in the field of web personalization and creating profiles for users will be discussed, and while mentioning the different methods, their strengths and weaknesses will also be examined for each of the methods.

    Chapter four: In this chapter, the proposed method will be described and The steps of the method will be explained. The fifth chapter: In this chapter, the simulation results of the proposed method will be presented, and also the proposed method will be presented in the form of a case study [4]. The sixth chapter: In this chapter, the research results and suggestions for future work will be presented. are called Concepts such as: web mining and its types, web personalization, clustering and related algorithms, and finally neural networks are mentioned.

  • Contents & References of Creating a recommender system on the web using user profiles and machine learning methods

    List:

    List of Figures D

    List of Tables E

    Chapter One: General Research. 2

    1-1- Statement of the problem. 3

    1-2- Necessity of research. 4

    1-3- Research objectives. 5

    1-4- Research structure. 5

    The second chapter: Literature of the subject. 7

    2-1- Definition of words and terms. 8

    2-2- Web challenges. 9

    2-3- Types of web mining methods. 10

    2-3-1- Application-based web mining. 11

    2-3-2- content-based web mining 14

    2-3-3- structure-based web mining. 15

    2-4- Web personalization. 16

    2-4-1- The benefits of the web personalization system. 20

    2-4-2- Law-based filtering systems. 20

    2-4-3- Content-based filtering systems 20

    2-4-4- Collaborative filtering systems. 21

    2-5- Recommender systems. 21

    2-6- Web personalization based on application-based web mining. 22

    2-7- Data sources 24

    2-7-1- Application data. 24

    2-7-2- Content data 25

    2-7-3- Structure data. 25

    2-8- Clustering. 26

    2-8-1- K-Means algorithm. 27

    2-8-2- Similarity criteria. 28

    2-9- Neural networks. 30

    Chapter Three: Previous works. 32

    3-1- Approaches based on exploring association rules and clustering. 33

    3-2- Combined methods in web personalization. 38

    3-3- Approaches based on indexing and keywords. 42

    3-4- An overview of web recommender systems. 43

    3-4-1- Web recommender system. 43

    3-4-2- Proposal production methods. 44

    3-4-2-1- methods based on collaborative filter. 44

    3-4-2-2- Content-based methods 44

    3-4-2-3- Knowledge-based methods. 45

    3-4-2-4- Combined recommender systems. 46

    3-4-3- Comparison of proposal generation methods. 46

    Chapter Four: Suggested method. 49

    4-1- Data pre-processing 51

    4-1-1- Data cleaning 51

    4-1-2- Identification and reconstruction of user visit sessions. 52

    4-2- Creating a profile for personalization on the web. 52

    4-2-1- Creation of the sitting vector. 52

    4-2-2- Clearing sessions 54

    4-2-3- Creating user profiles. 55

    4-3- Clustering profiles based on user behavior. 55

    4-4- Building recommender system using neural networks. 56

    Chapter five: Implementation and evaluation of the proposed method. 58

    5-1- Pre-processing stage and construction of meeting vectors. 59

    5-2- Clustering stage. 61

    3-5- The stage of generating suggestions using neural network. 62

    5-4- Evaluation of the proposed system. 63

    Sixth chapter: conclusion and future suggestions. 66

    6-1- Results of research and analysis 67

    6-2- Future works. 68

    List of references. 70

     

    Source:

    Persian sources:

    [1] Arzanian B., Moradi Dolatabadi P., Akhlikian F., 1388, "Personalization of search engines using fuzzy conceptual networks and data mining tools", 3rd Iran Data Mining Conference, Amirkabir University of Technology.

    [2] Jackson, T., Bill R. ., 1380, "Acquaintance with Artificial Neural Networks", translated by Mahmoud Albarzi, Tehran Institute of Scientific Publications, number of pages 146.

    [3] Khoger, A., Asdalahi Yazdi, H., Babaei, A., 1390 "Fault detection using RBF neural network", 5th Iran Data Mining Conference, Amir Kabir University of Technology, article code 136.

    [4] Rahmani S., Maibdi M., 2018, "Web personalization using developed associative rules", the third data mining conference of Iran, Amirkabir University of Technology.

    [5] Rezaei, Z., Harunabadi, A., Mir Abedini, J., 2019, "Presenting a method derived from user interactions, based on Petri nets in order to increase the matching of search engine responses with user queries", 5th data conference Iran Mining, Amir Kabir University of Technology, article code 291.

    [6] Zargani, A., Ahmadi, A., 1390, "Price prediction in the global currency market using artificial neural network based on harmonic pattern modeling", 5th Iran Data Mining Conference, Amir Kabir University of Technology, article code 123.

    [7] Sagheri A., Bagheri A., 1388, "Personalization Data recovery in peer-to-peer systems using to, 1388, "Personalization of information retrieval in peer-to-peer systems using user profile sharing", 3rd Iran Data Mining Conference, Amir Kabir University of Technology.

    [8] Qazizi, R., Haj Akbari, M., Ahsan, R., Minaei, B., 1390, "Using data mining methods in predicting work-related accidents", 5th Iran Data Mining Conference, Amir Kabir University of Technology, article code 323.

    [9] Forsehi R., Meybdi M., 2017, "An algorithm based on the link structure of pages and user usage information for recommending web pages", the second data mining conference of Iran, Amirkabir University of Technology.

    [10] Mohammadi Dostdar, R., Forsehi, R., Meybdi, M., 2013, "A hybrid web recommender system based on a two-layer graph and graph partitioning", 5th Iran data mining conference.

    [11] Malek Nia, A., 2013, "Presenting an algorithm for classifying legal clients of Parsian Bank and providing appropriate services to different groups of clients using data mining and neural networks", summer, thesis for obtaining a master's degree in the field of information technology management, information resource management. University of Tehran

    [12] Motamedi Mehr, S., Taran M., Baradaran Hashemi A., Meybdi M., 2018, "Web recommender system using distributed learning automata and graph partitioning", 4th Iran Data Mining Conference, Sharif University of Technology.

    [13] Nouri, R., Ashrafi, K., Ezhderpour, A., 2018, "Comparison The application of artificial neural network and multivariate linear regression methods based on principal component analysis to predict the daily average concentration of carbon monoxide: a case study of the city of Tehran", Journal of Earth and Space Physics, Volume 34, Number 1, Pages 135-152

    [14] Yousefi, F., Nemat Bakhsh, M., 2013, "Using Web Mining to Evaluate Digital Library Journals", 5th Data Mining Conference Iran, Amir Kabir University of Technology.

    English sources:

    [15] Azimpour, M., Azmi, R., 2011, "A Webpage Similarity Measure for Web Sessions Clustering Using Sequence Alignment", IEEE, International Symposium on Artificial Intelligence and Signal Processing (AISP), pp. 20-24.

    [16] Baglioni M., Ferrara U., Romei A., Ruggieri S. and Turini F., 2003, "Preprocessing and Mining Web Log Data for Web Personalization", Springer, advances in artificial intelligence, online ISBN: 978-3-540-39853-0, Vol. 2829, pp 237-249.

    [17] Breese, J., Heckerman, D. & Kadie, C., 1998, “Empirical Analysis of Predictive Algorithms for Collaborative Filtering”, in: Proc. of the 14th Conference on Uncertainty in Artificial Intelligence, pp. 43-52.

    [18] Billsus, D., Pazzani, M., 2000, "User Modeling For Adaptive News Access", in: User-Modeling and User-Adapted Interaction", vol. 10, no. 2-3, pp. 714-720.

    [19] Burke, R., 2002, "Hybrid Recommender Systems: Survey and Experiments", user modeling and user-adapted interaction, volume 12, issue 4, pp. 331-370.

    [20] Castellano G., Fanelli A. M., Torsello M. A., 2011, "NEWER: a System for NEuro-fuzzy WEb Recommendation", in: Applied Soft Computing, Elsevier Science Publishers B. V. Amsterdam, The Netherlands, vol. 11, no. 1, pp. 793-806.

    [21] Cooley, R., Mobasher B., Srivastava, J., 1999, "Data Preparation for Mining World Wide Web Browsing Patterns", Springer, Knowledge and information sys. Vol 1, Issue 1, pp 5-32.

    [22] Eirinaki, M., Vazirgiannis, M., 2003, “Web Mining for Web Personalization”, ACM Transactions on Internet Technologies (ACM TOIT), NY, USA, vol.3 no. 1, pp. 1-27.

    [23] Feldman, F., Sanger, J., 2007, “The Text Mining Hand Book”. Cambridge university press, 424 pages.

    [24] Faloutsos C. and Chan R., 1988, “Fast Text Access Methods for Optical and Large Magnetic Disks: design and performance comparison”, In Proceedings of the 14th VLDB Conference, pages 280-293.

    [25] Guo J., Keselj J. & Gao Q., 2005, “Integrating Web Content Clustering Into Web Log Association Rule Mining”, Proc. of 18th Conference of the Canadian Society for Computational Studies of Intelligence, Victoria, BC, Canada, pp 182-193.

    [26] Gong, S., Cheng, G.

Creating a recommender system on the web using user profiles and machine learning methods