Nov 1 - 14 Journal


On our visit last Thursday to Caltech, the group and I meet with graduate student Ethan and discussed recommender systems. Recommender systems are used all around us from Youtube to Spotify but we focused on the Netflix recommender system as Netflix actually has datasets published. Ethan told us that this dataset is out there because Netflix is challenging people to develop an even better recommender system than their current one using that data. He added that there was a prize of $1 million dollars so that also added some extra incentive.


The dataset that is used to make the recommender system is similar to that pictured above with a matrix created by users and their ratings of movies that they have watched. The empty spaces are the movies that users have not watched yet and we are trying to solve which blank spaced movies should be recommended to the user. After discussing, we explored three ways to break down the matrix and make recommendations. The first is similarity comparison where a user is compared with other users. The movies watched by users with similar tastes will then be recommended to the specific user. The more complicated second process is feature extraction where an algorithm breaks the matrix up into two matrices. The first matches users with feature preference and the second matches movies with features present. The final method we learned was simple clustering which is supplemental to the other two methods. Because this topic has very applicable, everyday life uses, I am very excited to further explore recommender systems these coming weeks.

The days after the Caltech visit, the group and I tried to test out matrix multiplication processes to try to further understand how the feature extraction process worked. Edmond and I tried to figure out how the matrices would multiply to get the original matrix as matrix multiplication involves summing the products of the numbers. We deduced that instead of actually creating the original matrix, the user vs feature and movie vs feature matrices are multiplied and a threshold value was applied to convert the values into favorable (1) or unfavorable (0). This seemed to make sense as if a movie's features only matched with a select few user preferences, then that movie should be disregarded or prioritized less as a recommendation. 


I also enjoyed finishing up my partner concept map with Connie on the Friendship Paradox. While it was kind of sad to figure out why your friends on average have more friends than you, it was fun to explore the proof and its applications. Connie did the video on the proof (above) and I did the video on the paradox's applications and comparison to everyday events (below). In it, I compared the friendship group to a gym and how people are right when they think they are less fit than the average person at the gym. Because the social network (gym membership) appeals to the health conscious, that group is going to be more fit like how friendly people who make up your group will be more social and outgoing.


Comments

Post a Comment

Popular posts from this blog

April 10 Technical Journal

Image
Code: Working right now as sort of a middle man between the Q learning and code sides, I was helping out a bit with the code as well since the last technical journal on March 13. While I am still mainly focused on the Q learning side, I worked with Edmond on how to improve user interaction with our game. This would come in the form of changing how the lines are inputted. Our current game requires us to input coordinate points to create lines and our new idea is to designate certain coordinates' pairings as a number (depicted below). For example, the line connecting (0,0) and (1,0) is 1. The pattern would be like a zig zag. Q Learning Algorithm: Continuing on the reinforcement learning look into the Q learning algorithm, Connie and I were working with code we got off of Github which stored Q values in a Q table file. Running it over the period, a statement of "Kolo #,#" was printed. The number started off very unbalanced with the first at 0 and the second at 99. Ove...
Read more

March 13 Technical Journal

Image
Tic Tac Toe Data: We first planned on spending this two weeks looking into datasets for tic tac toe and processing it as a pre-trial for our dots and boxes. What we found and did though, I though was far more insightful. Initially, I was looking around for datasets and found from UVA a dataset for tic tac toe intermediate game states. We thought this could be something as the UCI dataset that we had first come across only gave endgame states. An image of the UVA dataset is posted below. Discussing with Mr. Lee, though, we could not decipher what the non-zero/one numbers represented and decided to move on looking, a sort of dead end. We inferred that -1 and 1 were X and O while 0 was blank. We would eventually later find another way to gather datasets (see Tic Tac Toe Q Learning below). Part of Tic Tac Toe intermediate game states data Tic Tac Toe Monte Carlo: While we had been looking into datasets, it turns out that Will was experimenting with the Monte Carlo search tre...
Read more

Start of School - Sep 12 Journal

Image
Through these several weeks, we have begun our journey to exploring research in machine learning by starting our paper airplane research project, hearing undergraduate research presentations, and learning the foundations of machine learning and the basic principles of research.  Foundations: Our first several weeks covered the foundations of research and more specifically machine learning. In a class activity, all the groups came up with several truths about research and the class was able to collectively develop a set of rules and expectations for the research journey. From "always keep on trying" to "look to nature for inspiration," many of these truths resounded with me as I hope to learn more about this field via research in this course.  The foundations of machine learning were also covered through a series of videos. From explaining neural networks to linear regression, these videos established a baseline from which my group and I could work o...
Read more