Learning Machine Learning with Aston

Triumphs: One area of advancement in our project has been our working with Mr. Lee and one of his students to develop a UI (user interface) for our dots and boxes game. Before this, we were playing through the PyCharm terminal so this is a step forward. The first version of the UI is depicted below. Other progress we made was in the integration of our Q-learning work with the Monte Carlo search tree. The Q table would be used instead of random rollout in the Monte Carlo. We also completed the Problem Statement and Background sections for our official report. Preliminary user interface for the dots and boxes game Struggles: Because some people were out due to illness and college visits, some progress was slowed down. Our plan had been to transfer our Q-table into the Monte Carlo Search Tree but formatting differences resulted in incompatibility. As a result, a new Q-table had to be made for the Monte Carlo. The generation of the Q-table (part depicted below) was also an i...

Code: Working right now as sort of a middle man between the Q learning and code sides, I was helping out a bit with the code as well since the last technical journal on March 13. While I am still mainly focused on the Q learning side, I worked with Edmond on how to improve user interaction with our game. This would come in the form of changing how the lines are inputted. Our current game requires us to input coordinate points to create lines and our new idea is to designate certain coordinates' pairings as a number (depicted below). For example, the line connecting (0,0) and (1,0) is 1. The pattern would be like a zig zag. Q Learning Algorithm: Continuing on the reinforcement learning look into the Q learning algorithm, Connie and I were working with code we got off of Github which stored Q values in a Q table file. Running it over the period, a statement of "Kolo #,#" was printed. The number started off very unbalanced with the first at 0 and the second at 99. Ove...

Because we just did a journal last week, this journal may be a bit shorter. For the Caltech visit last Thursday, issues with the bus resulted in us leaving at 12:50 instead of 12:30. This probably stemed from Club Rush happening that day and the closed-campus policy. The bus, however, left a tad earlier than 12:50 so we left Connie at school for this visit. Because we arrived around 20 minutes late, Dr. Hassibi had left. This visit also conflicted with Robotics' Idaho competition so only Robert, Puja, and I were at Caltech.  At Caltech, we spent our time brainstorming how our end of year presentation would be done. We concluded on an idea of a funnel, going from the larger concepts into our specific project. We would have a section on the different concept maps we made as concepts like decision trees and clustering are not super related to dots and boxes. We would then go into our possible project ideas we had. These would include the vape detector, food recommender, etc. ...

STEM: I was absent Monday and Tuesday of this week as I was out sick so this journal is mainly going to be about my spring break and what I've done since the technical journal. For the technical stuff, I have begun to move towards a middle job area of helping around both with the q learning with Robert and Connie and our game code with Puja, Edmond, and Will. I helped with the debugging of the game with an example of us fixing the game crash when wrong inputs are given. Other work includes trying to develop the reward aspect of the Q learning algorithm. Because Will and Edmond are out for a robotics competition, I see myself working with Puja on the code more this week. Spring Break: Over spring break, I went to Italy with my family. We stopped in Rome, Florence, and Milan over the week! Rome was our first stop and we went to see the Colosseum and the Roman Forum which were very historical and impressive. We also visited the Pantheon and I was amazed by the large d...

Tic Tac Toe Data: We first planned on spending this two weeks looking into datasets for tic tac toe and processing it as a pre-trial for our dots and boxes. What we found and did though, I though was far more insightful. Initially, I was looking around for datasets and found from UVA a dataset for tic tac toe intermediate game states. We thought this could be something as the UCI dataset that we had first come across only gave endgame states. An image of the UVA dataset is posted below. Discussing with Mr. Lee, though, we could not decipher what the non-zero/one numbers represented and decided to move on looking, a sort of dead end. We inferred that -1 and 1 were X and O while 0 was blank. We would eventually later find another way to gather datasets (see Tic Tac Toe Q Learning below). Part of Tic Tac Toe intermediate game states data Tic Tac Toe Monte Carlo: While we had been looking into datasets, it turns out that Will was experimenting with the Monte Carlo search tre...

Genetic Algorithm: After briefly looking into SARSA learning, I decided to switch and explore genetic algorithms after receiving a research paper Connie sent me. In this paper, researchers who were teaching an AI to play dots and boxes explained that they had used a genetic algorithm to create their neural network. As a result, I looked into genetic algorithms and was quite excited. This was because I could really connect with the algorithm as it was based on biology that I have studied. The close comparison made this research more tangible to me and I felt that I could relate more to it. The algorithm is explained in my Concept Map video below but it is basically an algorithm that simulates natural selection to choose the most "fit" neural network.  Caltech Visit: We visited Caltech on Thursday though we were short on people. This was because Edmond and Will were at a Robotics competition and Puja was stuck in Chemistry. Nevertheless, Robert, Connie, and I talked w...

This past week, Robert and I were in Yosemite for the annual trip up to the national park. We were originally supposed to leave on Sunday but snow delayed us to Monday. Because of the heavy snow, the usual Crane Flats campsite was inaccessible so we stayed in Yosemite Valley at the Yosemite Lodge. While this was not our original plane, I actually enjoyed staying in the valley more than at Crane Flats because we were at the center of the many attractions. For example, out hotel was right in front of Yosemite Falls (pictured below). On our first day of activity, the Winter Survival group was split into three different groups and my group and I got together to pass out lunch and discuss our day plans. While waiting for our group counselors from Naturebridge, we played in the snow and had a snow ball fight. After our counselors arrived, we hiked over to Yosemite Falls. At the base of the trail, we went through Spider Cave first. This was a cave made from large rocks that fell from the...

Before going to Caltech, we had narrowed down our ideas to one ambitious and one simpler project. The ambitious one was the e-cigarette image recognition system and the simpler one was going to be the game like tic-tac-toe. Upon getting to Caltech, though, our selection was pretty much dismissed as we went through all of our ideas with the graduate students. This was because Dr. Hassibi had to be in a meeting. Because we received further commentary on the other ideas, the visit really had us back to square one.  Some things I learned at Caltech exposed possible areas of weakness in our project ideas. The grad students pointed out that for the e-cigarette idea, we would need a lot of data, meaning that we would need a lot of examples of someone vaping in front of the thermal sensor. Robert explained that he saw a person train the image recognition algorithm with just 25 images. While the certainty wasn't great, the answer was typically correct. Because we are focusing on th...

After our visit to CalTech last Thursday, we narrowed down our semester 2 project ideas to the four that will be elaborated upon below. Before I do that, here's what I have been doing since last Thursday. First, I have been looking into current e-cigarette vapor sensors and how thermal imaging recognition works. I have also discussed the different project ideas with group mates and am working with Robert to focus on this more ambitious project. I am looking forward to exploring how to gather a dataset of thermal images for algorithm training and how effective our sensor would be. Idea 1: E-Cigarette Thermal Sensor Alert System One major issue that afflicts the youth of today is the use of e-cigarettes, wiping away remarkable progress in fighting smoking. This rise not only creates another issue for health professionals but issues in a new wave of modernization in our response to high-tech nicotine. Due to privacy laws, most students use use e-cigarettes in school restrooms and...

My first semester of Caltech STEM Research really exposed me to a lot of more in-depth concepts that I have come to really appreciate. Such concepts include the wide range of clustering algorithms, decision trees, recommender systems, and more. It felt very fulfilling to be exploring machine learning in a high school environment. For example, the iris decision tree project (depicted below) was quite different from my past interactions with computer science. We were actually making real-world deductions from the iris dataset and not just coding a game of battleship. The explorations of these concepts also enabled me to get to know my machine learning group mates better and working with them has also made the class a lot more fun and collaborative. A very useful skill I learned last semester was the screen capture video recording. This key aspect of my concept maps proved useful in creating a video in my Economics class and sending a video to Naviance regarding an issue. I also le...

Dimensionality Reduction: Before going to Caltech, I finished up looking into dimensionality reduction for the recommender system concept map. My concept map video is linked below! I learned more about the different techniques for dimensionality reduction like PCA and low variance filters. I also explored the applications and found how it can be used to reduce data noise in images. It consequently makes the picture clearer. I thought it was quite satisfying to look into an aspect of the concept that enables everything else to work! CalTech: We visited Caltech on Thursday where we were introduced to linear predictors. They are sort of a variation of linear regression. Dr. Hassibi explained how we could use linear predictors to find an equation that uses our survey data. The equation would sum up the product of coefficients and input survey answers to get a value. A threshold value would then be set to determine where the final question is a yes or no. I also learned that whe...

Recommender Systems: After my last journal, I looked into prior to Thanksgiving break and am looking into now the topic of dimensionality reduction in the larger concept map of recommender systems. I found several videos with one linked below that were really helpful. I learned how dimensionality reduction is used to simplify and streamline the processing of the data. Using principle component analysis, dimensionality reduction is used to create a plane of lesser dimensions that contains all the data points at a higher dimension. I look forward to sharing what I have learned with the group, and I feel that dimensionality reduction will play an important part in ensuring the matrix processes run smoothly. Thanksgiving Break: time to relax With the start of break, I spent time finishing all schoolwork during the first weekend. My family and I went to pick up my sister at the airport and got Japanese soba in Torrance on the way back. I tried out a brunch cafe near my home that I...

On our visit last Thursday to Caltech, the group and I meet with graduate student Ethan and discussed recommender systems. Recommender systems are used all around us from Youtube to Spotify but we focused on the Netflix recommender system as Netflix actually has datasets published. Ethan told us that this dataset is out there because Netflix is challenging people to develop an even better recommender system than their current one using that data. He added that there was a prize of $1 million dollars so that also added some extra incentive. The dataset that is used to make the recommender system is similar to that pictured above with a matrix created by users and their ratings of movies that they have watched. The empty spaces are the movies that users have not watched yet and we are trying to solve which blank spaced movies should be recommended to the user. After discussing, we explored three ways to break down the matrix and make recommendations. The first is similarity c...

Concept Maps:  Finishing off my Concept Map on k-means  algorithm, I explained more about k-means to the rest of my group while we all shared information about what we learned. I created my Concept Map video on the k-means algorithm (pasted below) while the group and I discussed how centrality worked in data graphing (pictured below). While we were originally planning on focussing in on centrality for our next concept map, our visit to Caltech changed the course of our plans as centrality was not as focussed upon. Will had figured out parts of the centrality concepts though, so he taught us how betweenness centrality, closeness centrality, and degree centrality would be used to analyze nodes in the data graphs. The centrality established that the higher the centrality value, the more "central" that point is. If you imagine the data graph to be a social network, the node with the highest centrality is the most popular person. After visiting Caltech on Thursday...

Clustering : These two weeks, I researched the k-means clustering algorithm after the group split amongst ourselves different clustering algorithms. I already knew the basics of k-means, that data points would be clustered depending on the centroids that would be shifted until all the data stayed in the same clusters. One of the key issues with the k-means algorithm, however, is finding exactly how many clusters the algorithm should create, because the user must input the "k" value, the number of cluster. I found that the "Elbow Method" (pictured below) was the most common way to determine the "k" value. The Elbow Method uses the Least Squares Method to calculate how many clusters minimizes the distance between centroids and the cluster data points. I also learned of how the k-means algorithm can be applied. This clustering can be used to cluster customer purchases, personality test respondents, or even typical Youtube recommendations. During t...

Decision Trees: This week, I continued looking into decision trees with my concept map being conditionals (if/elif/else statements) and also working a bit on the Titanic survivability tutorial. Before the new deadline for the concept map was set, I decided to complete my concept map entry due to other tests later in the week and compared conditionals to ordering at In-n-Out. I explained how conditionals are used by the decision tree to create the branches and categorize the data. I also learned about the entropy (randomness) of the decision tree from Will today with the group. I thought the discussion with Will and the group was very informative but had reservations about the extent to which each person should research individually. My concern was how much individual research is too much so that our skillsets do not become so different that it impedes discussion. A possible area of further research is the random forest trees area within decision trees. Learning Decision Trees Ent...