Personalized Product Recommendation using HybridFiltering in Recommendation SystemsKetan Navanath Karandex17100062School of ComputingNational College of IrelandDublin,IrelandEmail.id [email protected]—Due to latest technological advances, buying yourthe desired product has become very easy.Online buying isthe latest trend and E-commerce companies are booming bythe day.To increase user experience, product recommendationsystems are used by many E-commerce companies, but dueto increase in traffic leading to a huge collection of data,recommending a the correct set of products to correct groupof customers have become an ever-evolving challenge.Along withtackling this problem, companies are also spending a majorityof their research budgets on designing a personalized recommendationengine which can understand the user’s choices andtolerance for diversity in products recommended to them.Thispaper focuses on the methods that are currently used in designingsuch recommendation engines and highlights the grounds forimprovement in the subjectKeywords—Recommendation Systems; Collaborative Filtering;Content Filtering; Diversity; Sparseness; Bipartite Graphs; NetworkBased Inference Algorithm; Typicality Based AlgorithmI. INTRODUCTIONClustering and Segmentation are used since many years inthe field of Statistics to deal with large numerical problemsand scenarios.However, In last two decades, the world haswitnessed a paradigm shift where the use of this techniqueshave found their roots in the online markets and are provingvery beneficial.Due to excessive demand for online shoppingand surfing, the amount of data collected on a daily basisis huge. Use of this data is very crucial for improving Ecommercebusiness.When a user wants to search something,he uses search Engines like Google, Yahoo, Bing etc.But manytimes the user is not sure about the content he is looking for. Insuch cases, search Engines can come to rescue through variousRecommendations which can navigate the user in the rightdirection.This boosts the user experience and ultimately helpsthe business to grow.Same is the case for E-commerce businesseslike Amazon, Netflix, Youtube etc. So to enhance userexperience and improve business, Recommendation Enginesare the best tools.Recommendation Engines work on the principle of Clusteringhuge database and create smaller manageable sectionstermed as Segments.Recommendations are made across suchsegments. The technique of Collaborative Filtering is theadvanced way of creating segments for Users1 and ContentFiltering is the way to create segments for Items2.However, product Recommendations can have differenttypes of issues like the ones were customers are reluctant forexperimenting while buying the products or when the productfeature vectors are too sparse to create segments or when thecustomer is new to the system and Recommendation Enginehave no information about his choices and preferences3.Thispaper highlights such shortcomings of the recommendationengines and stresses on various ways to solve them usingmodern techniques like Hybrid Filtering with Typicality BasedAlgorithms and Network Based Inference Algorithms.II. COMPONENTS OF RECOMMENDATION SYSTEMSRecommendation Engines mainly work on the Segmentscreated through clustering of the database. A group of certainproducts is recommended to a certain group of users based onthe previous products searched or bought by the users.Let’sdiscuss the components in detailsA. ClusteringVarious Clustering techniques like K-means and Kmedoidsare used to group the users according to various featurevectors4. Some of the feature vectors are Ratings givenby the users for different items5, weights of customer RFM(Recency, Frequency, and Monetary)values acquired from thedatabase6, and scores of reviews given on various socialmedia platforms like Twitter and Facebook7.K-means technique uses a method where the clusters aremade by taking the arithmetic mean as a real data in the datapoints to calculate distances with other points8 while Kmedoidsuses a method where one of the actual data pointsis used as a median i.e point with smallest average distancefrom other points in the cluster5B. SegmentationSegmentation is a technique that takes results from clusteringand uses it to create groups in the data.This paperemphasizes on various segmentation techniques used in recommendationengines.The process of segmentation can be broadlydivided into following three parts1) Collaborative Filtering: Collaborative filtering is basedon the psychological fact that people with similar tastes inthings tend to choose similar items as well.This type of filteringcan be termed as User-based filtering. Companies like Netflix9rely heavily on the movie ratings given by the users sincethis not only displays the user experience but it also helps inclubbing users with similar ratings .Here, ratings is used as afeature vector to group like minded users and create genre wisesegments like Action movie Lovers, Adventure movie lovers,Sci fi movie lovers and so on.2) Content Filtering: Content Filtering is termed as Itembasedfiltering.In this type of filtering an individual user cart isanalysed. Based on this, a segment of products is made whosecharacteristics are very similar to the items in the cart. Theitems from this new product segment are shown to the userin the form of recommendations2 whenever he is online andthe chances of the user, buying this newly recommended itemsis high since the user has already shown his interest in similaritems .3) Hybrid Filtering: The above-mentioned types of filteringalthough being famous and effective in their own wayshave a certain set of limitations as well.When the enginegives recommendations based on collaborative filtering, it justtakes into account the preference of the user but it fails toconsider the tolerance of the user to the diversity in therecommendations9 given for e.g. A user with a preferencefor Action movies may not like the movies that are consideredunder the Action-Adventure genre.Similarly, when the engine gives recommendations basedon content filtering, it just takes into account the informationof a single user based on his cart and recommends himaccordingly but it fails to consider that the user may likeproducts that he has never added to his cart before.To solve this problem a Hybrid approach of filtering shouldbe used that can take into account the users tolerance todiversity in the recommendations along with finding differentitems that the user may like by taking reference from the cartof other users with similar tastes.Thus, Hybrid based filteringcan be termed as a mixture of User-based Filtering and Itembased Filtering.Other related problems are discussed in furtherpaperC. AccuracyFor every Recommendation Engine, accuracy is the mostimportant element. The Engine should be well versed with theknowledge that guides it when recommendations are given tothe users. This ranges from what to recommend to how long tokeep recommending9.Following components helps in guidingthe Engines1) Average Ranking Score: The list of Items that areviewed by the user is taken and the items actually purchased bythe user is removed.Then a list of uncollected items is createdin which the items are ranked9.Based on these, an averageranking score is calculated for all the user-item pairs.2) Precision: From the above-mentioned list, only a fewranked uncollected items are shown to the user.The exactnumber of items to be shown is calculated using Precision9.3) Recall: After calculating precision, the short list ofranked uncollected items is made and it is repeated forindividual users for a specific amount of numbers and time9.4) F1 measure: F1 measure comprises of Precision andRecall9 and it guides the Engine accordingly.Any Enginethat runs on such guide lines have a better chance of reachingaccuracy.III. CHALLENGES ASSOCIATED WITH TRADITIONALFILTERING APPROACHAll of the following mentioned challenges arise for Contentand Collaborative Filtering because of the recommendationsthat cannot be personalized i.e this two methods hold goodwhen a certain population of users is considered and not to anindividual user.A. Cold StartThe problem of cold start can arise in item side as well ason the user side. It is relatively easy to recommend productsto users based on their historical data as compared to anew user3. Also recommending items with less rating/userreviews is always difficult since they do not fit in any cluster.B. SparsenessData Sparseness occurs due to lack of information. Thereis a vast variety of products and different kinds of users withunique behavioural shopping and surfing patterns3. Due tothis lack of knowledge, recommending items from a groupwith incomplete information on feature vectors like ratings andreviews lead to inaccuracies in recommendations.C. DiversityThis problem arises when the engine tries to give recommendationsto users about the items they have never usedbefore.Tolerance of diversity can be very different for everyuser since not everyone is open to experimentation9. Thiscan lead to inaccurate recommendations with unsatisfied userexperiences. Diversity can be divided into following categories1) Inter Diversity: User segments formed through collaborativeFiltering can be very different from each other.Thatis why, the recommendation lists offered to every segment isunique in its own way.This difference in lists comes under themetrics of Inter Diversity92) Intra Similarity: Item segments formed through ContentFiltering is different for every individual user.However theitems in this segments can also show a huge variety in theirnature.Such varied degree of items in an item segment comesunder Intra Similarity9.D. Time factorThe factor of time can change user requirements to agreat extent.If this factor is not taken into consideration thenthere can be inaccuracies in the recommendations2. Timenot only changes the user requirement but it also changes hispreferences.Example of Requirement Shift: For a user who just hada baby, Business can be a good recommendation but it won’tbe the case in 2 years when the requirement of the user shiftsfrom baby products to a Tri-cycleExample on Preference Shift: for a user with kids and toddlersin-house, movie recommendations with animation genre canbe appropriate but within years the choice might shift toeducational documentariesIV. PERSONALIZED SOLUTIONS OFFERED BY HYBRIDFILTERINGTraditional Filtering, when combined with various techniques,can help in improving the performance of the RecommendationEngines.Such a Hybrid approach helps in designingfollowing solutions to the above-mentioned problems and takesa leap forward towards designing personalized recommendations.A. Collaborative Filtering with Demographics based recommendationFor problems such as cold start, it is generally verydifficult to judge what a user might like since the engine dontknow anything about the user.In such cases understanding thepersonality of the user becomes crucial for recommendations.Social Media can be an important resource to understanddemographics of the user3.Collaborative Filtering based onsuch demographical information can help the engine learnmore about the user and thus give appropriate suggestions.Sucha Hybrid approach can help to tackle issues related to new userwho is just introduced in the system and this will be uniquefor every user.B. Collaborative Filtering based on Network Partition andGroupingFor problems such as Data Sparsity, the main problem isto understand the incomplete data and make segments.Thedegree of sparseness is given by E/(M*N) where E is thelink number, M is a number of Items and N is number ofusers9. These metrics are used to draw a Bipartite Graphwhich helps in joining the data points and create segmentsaccordingly.Many such Bipartite graphs are created in caseof the huge datasets and a Network of user-item segmentsis made.This Network helps in visualizing the groups moreproperly and thus help Engines in partitioning and groupingpairs into segments, across which various recommendationlists are established and fed to the systems.This helps inpersonalizing the engine according to each and every userbased on his links(E) to various items.C. Network Based Recommendations and Maximum MarginalRelevanceFor problems such as Diversity, understanding the tolerancelevel of the user to the diversity in products is very important.Not every recommendation is going to please the user. Tosolve this problem collaborative filtering can be used withMaximum Marginal Relevance5. In this method, the user’ssearch queries are ranked according to the items chosen byhim after execution of every query. Hence, a Bipartite Graphis drawn which pairs user search queries with items checked byhim in that query.Based on this a Network is made for that userwhich can help in understanding the extent of users preferenceswhen items that are not relevant to him are displayed.SuchNetwork Based Inference algorithms helps in solving the issuesof diversity and help engines to decide a variety of range ofitems that should be displayed to the user.This variety willdiffer amongst users based on their MMRs and consequentlya personalized engine is created.V. FUTURE SCOPEThis Paper highlights the challenge of incorporating TimeFactor in Recommendation Engines to create a heavily accurateSelf-Evolving Engine which can change the face of RecommendationSystems that we know today.VI. CONCLUSIONDespite high performing Recommendation Engines, thereis still a lot of ground to cover in terms of personalizing theengine to individual users.This paper highlights the Hybridapproach where traditional filtering methods are used with newtechniques like Network-Based Inference Algorithm, MaximumMarginal Relevance, Demographics Based recommendingetc. to take a step closer to personalizing the engines.Italso focuses on the future challenges that the engines are yetto adhere.REFERENCES1 B. Deshpande, “Collaborative filtering is thenew customer segmentation,” 2016. Online. Available:http://www.simafore.com/blog/collaborative-filtering-is-the-newcustomer-segmentation2 B. Smith and G. Linden, “Two decades of recommender systems atamazon.com,” in IEEE Internet Computing 21(3),7927889, 2017, pp. 12–18.3 D. S. Jain, A. Grover, P. S. Thakur, and S. K. Choudhary, “Trends, problemsand solutions of recommender system,” in International Conferenceon Computing, Communication and Automation (ICCCA2015), 2015.4 X. Liu, Z. Wang, and F. Ye, “A recommendation system algorithmbased on large scale internet environment,” in 13th Web InformationSystems and Applications Conference, WISA 2016 – In conjunction with1st Symposium on Big Data Processing and Analysis, BDPA 2016 and1st Workshop on Information System Security, ISS 2016, vol. 7878241,2016, pp. 108–112.5 F. Eskandanian, B. Mobasher, and R. Burke, “A clustering approach forpersonalizing diversity in collaborative recommender systems,” in UMAP2017 – Proceedings of the 25th Conference on User Modeling, Adaptationand Personalization, 2017, pp. 280–284.6 A. Umair and D. Z. Alamgir, “Product recommendation using typicalitybased collaborative filtering and churn analysis,” in 2016 6th InternationalConference on Innovative Computing Technology, INTECH 2016,vol. 7845012, 2017, pp. 220–225.7 P. Adamopoulos and V. Todri, “Personality-based recommendations:Evidence from amazon.com,” in CEUR Workshop Proceedings 1441,2015.8 L. H. Ungar and D. P. Foster, “Clustering methods for collaborativefiltering,” in AAAI Technical Report WS-98-08, 1998.9 L. Hu, L. Ren, and W. Lin, “A reconsideration of negative ratings fornetwork-based recommendation,” in Physica A: Statistical Mechanicsand its Applications, vol. 490, 2017, pp. 690–701.