Feb 21 - March 7 Journal

Genetic Algorithm:
After briefly looking into SARSA learning, I decided to switch and explore genetic algorithms after receiving a research paper Connie sent me. In this paper, researchers who were teaching an AI to play dots and boxes explained that they had used a genetic algorithm to create their neural network. As a result, I looked into genetic algorithms and was quite excited. This was because I could really connect with the algorithm as it was based on biology that I have studied. The close comparison made this research more tangible to me and I felt that I could relate more to it. The algorithm is explained in my Concept Map video below but it is basically an algorithm that simulates natural selection to choose the most "fit" neural network. 


Caltech Visit:
We visited Caltech on Thursday though we were short on people. This was because Edmond and Will were at a Robotics competition and Puja was stuck in Chemistry. Nevertheless, Robert, Connie, and I talked with the grad students about which algorithm would be best for the project as the game had already been coded. Robert and Connie focused on Q learning while I talked about the genetic algorithm and showed them the paper. In the end, they decided that Q learning would be better and that it would be better if we shrunk the game space to a 4x4 dots and boxes game. Mr. Lee brought up Will's Monte Carl search tree idea but they thought it would be too complicated.


Tic Tac Toe:
As a tutorial for our applying of Q learning and supervised learning (backup) to dots and boxes, we were tasked with looking into tic tac toe first. This was because tic tac toe was simpler and likely had more data available. I looked around and found for the group three datasets from UVA with tic tac toe game states. However, we were unsure about some values within those datasets and also how to convert this dataset into input. Today, Will showed us a tic tac toe game that ran on the Monte Carlo search tree (depicted below) after finding code for it. It seemed to not be overly complicated and was also quite interested in it as it is similar to the genetic algorithm.



Next:
On Monday, we were still not very sure as to what we would be doing next. After filling in Puja, Edmond, and Will though, I am looking forward to exploring Q learning with Robert and Connie. We found a tutorial for Q learning that involved the Fifa soccer game and I am currently working on part of that tutorial. The tutorial was split amongst us three, but I'm a bit unsure as to converting the Fifa aspects of the code to dots and boxes. I am also looking into a bit on how to save game states though, I do feel that my coding skills need some more work for editing this code. 


Comments

Post a Comment

Popular posts from this blog

April 10 Technical Journal

Image
Code: Working right now as sort of a middle man between the Q learning and code sides, I was helping out a bit with the code as well since the last technical journal on March 13. While I am still mainly focused on the Q learning side, I worked with Edmond on how to improve user interaction with our game. This would come in the form of changing how the lines are inputted. Our current game requires us to input coordinate points to create lines and our new idea is to designate certain coordinates' pairings as a number (depicted below). For example, the line connecting (0,0) and (1,0) is 1. The pattern would be like a zig zag. Q Learning Algorithm: Continuing on the reinforcement learning look into the Q learning algorithm, Connie and I were working with code we got off of Github which stored Q values in a Q table file. Running it over the period, a statement of "Kolo #,#" was printed. The number started off very unbalanced with the first at 0 and the second at 99. Ove...
Read more

March 13 Technical Journal

Image
Tic Tac Toe Data: We first planned on spending this two weeks looking into datasets for tic tac toe and processing it as a pre-trial for our dots and boxes. What we found and did though, I though was far more insightful. Initially, I was looking around for datasets and found from UVA a dataset for tic tac toe intermediate game states. We thought this could be something as the UCI dataset that we had first come across only gave endgame states. An image of the UVA dataset is posted below. Discussing with Mr. Lee, though, we could not decipher what the non-zero/one numbers represented and decided to move on looking, a sort of dead end. We inferred that -1 and 1 were X and O while 0 was blank. We would eventually later find another way to gather datasets (see Tic Tac Toe Q Learning below). Part of Tic Tac Toe intermediate game states data Tic Tac Toe Monte Carlo: While we had been looking into datasets, it turns out that Will was experimenting with the Monte Carlo search tre...
Read more

Start of School - Sep 12 Journal

Image
Through these several weeks, we have begun our journey to exploring research in machine learning by starting our paper airplane research project, hearing undergraduate research presentations, and learning the foundations of machine learning and the basic principles of research.  Foundations: Our first several weeks covered the foundations of research and more specifically machine learning. In a class activity, all the groups came up with several truths about research and the class was able to collectively develop a set of rules and expectations for the research journey. From "always keep on trying" to "look to nature for inspiration," many of these truths resounded with me as I hope to learn more about this field via research in this course.  The foundations of machine learning were also covered through a series of videos. From explaining neural networks to linear regression, these videos established a baseline from which my group and I could work o...
Read more