Self-Driving Cars: Learn Log
A log of everything I do to learn about Self-Driving Cars, and trying to build something interesting.
Current Level: Noob
28th November 2021
Today I started to try out working out a map for learning “How to build Self-Driving Cars?” and the first thing that I did was what we do when we have a Question and Internet.
And I came across Max Deutsch’s Blog: How to build a self-driving car in one month? I gave it a read till a few days and started on looking for a Lane-Detection project on Github, where I came across: Conditional Lane Detection.
5th December 2021
I started going through the Github README.md of the project to get a gist of what was being done in the project. And the first thing that came to my mind was “This might just be a tad bit too complex for my dumbass mind.”
I recollected myself and thought that if I have started with a difficult one I might be able to learn something in the process of figuring out how this project works.
16th December 2021
Got a mail from AIcrowd regarding their new challenge with Carnegie Mellon Robotics Academy and Team Arrival called Learn to Race Challenge. The challenge is to train a Reinforcement Learning model that can drive a car around a race track with Highest Average Speed and lowest Safety Infractions(i.e. time off-track). I thought that this should be the best way for me to start learning in a way where I do not have to find software and other different things and can get my hands dirty by directly jumping into it. Started by going through the docs and the starter kit to get a little context on how exactly will I need to go through the challenge.
19th December 2021
Finally started going through the problem statement. To run the code we need the simulator. While going through the installation process I found out that the program needs an Ubuntu 18.04 machine with Nvidia GPU which I (who is using my office Mac 😬) do not have. So, I looked for the Docker file to use but that wasn’t released because the Dockerfile had some sensitive data related to the competition.
Discord community to the rescue!!
I posted this issue on Discord and they told me to use an AWS VM for which they’ll give us free credits. Yay!!
So, I started setting up the AWS EC2 instance and at the last step, I realised that I need to get the limit for GPU instances, for which I created a request and seems like I’ll have to wait for a couple of days for the approval.
20th December 2021
I had a conversation with AWS support today and shared some information that they required. They said that it would take a day or two more to get this resolved.
To utilise my time I tried setting up a free-tier machine just to brush up on my AWS skills which presented a new issue which was my lack of familiarity with MacOS. I tried looking up tutorials online and for some reason, there is a huge lack of tutorials for Ubuntu/Linux EC2 instance with MacOS local machine. Though, after a lot of looking up on Youtube and Stack-Overflow, I was able to connect to my machine.
This brought me to the next step which was to set up the GUI, which was another challenge in itself. Again, I went through the cycle of Search-Implement-Error-Repeat, I was able to finally able to set up the GUI on a t2.micro machine and saying that it looked crappy would be a gross understatement.
I thought that this might be due to the very minimal specs of the t2.micro machine (i.e. 1 vCPU and 1GB RAM). To confirm this theory I set up a c6g.large machine with 2 vCPUs and 4GB RAM but to no avail. Although, it did seem that the system was a bit more responsive but even that wasn’t enough for my use case.
My next best guess is GPU should make it better hopefully or else I’ll have to check some other light-weight GUI to run on my Ubuntu machine.
8th January 2022
I have been trying to get the AWS limit increased but haven’t been able to. Due to the lack of enough machines, my request is getting rejected again and again. (If someone has any idea on how to get it approved please comment down and help me out.)
So, I moved on and started working on learning DL some other way. I went through Open AI’s Gym and found a Box2D env called CarRacing-v0. After going through the documentation, I started installing the dependencies and starting up the env.
9th January 2022
Today, I started running a random model to run the car in the environment and go through the observations sent by the environment. The observations were a huge amount of data and I didn’t know the meaning of that. So, I tried going through the source code of the environment but wasn’t able to figure out what the observations meant.
12th January 2022
I finally got to know what the input meant (which was the pixel values for the current state of the scene) and started working on a strategy to build my model to run through the course.
The first approach I came up with was as the road starts to turn the number of green pixels in the image towards the direction it is turning towards should reduce because those pixels would be taken by grey colour (i.e. road).
So, what I did was use OpenCV and split the screen into 2 halves and just let it count the number of pixels and steer towards the direction it wanted to go towards. There were 2 problems with this:
- Acceleration/Braking
- Hairpins
First of all, the car was accelerating constantly which meant I had to specify a value and it would use that acceleration throughout which isn’t the most efficient way of driving. And secondly, the car would drive aggressively towards the direction of the hairpin which was practically impossible with a considerable amount of constant acceleration.
To solve this I used the ratio of the difference in the green pixels on each side to the total number of pixels and used it to control the throttle of the car. The lower the ratio the higher the acceleration and the higher the ratio the lower the acceleration. This worked perfectly with most simple turns but it still broke down on the hairpin because it made the car basically stop or spin out.
To solve the hairpin problem, I started to use the smallest possible window that wouldn’t let the car detect the exit of the hairpin. This meant that the car would limit the operating space to just outside of the road it is currently on.
This system worked brilliantly and in a very reliable way. In the next step, I need to make a Deep Learning Network that can do the optimizations I did to get the best parameters for our system.
23rd March 2022
So, the next step was to learn Deep Learning. Luckily, I had tried a couple of years ago and had a good idea of where to start.
I looked up Alexander Amini’s Youtube Channel, who is a tutor for MIT’s Introduction to Deep Learning 6.S191. He uploads all the lectures they take every year to his Youtube channel and I knew that was the perfect starting point.
This is an in-progress story. Stay tuned for the updates. I will update on my LinkedIn and Twitter.
For more content, check out my Youtube Channel.
