Self Balancing Bot
This project aims at stabilising a two-wheeled robot (naturally in non-equilibrium)
using Reinforcement Learning.
It is completed under the guidance and mentorship of Artificial Intelligence and Electronic Society(ArIES) of IIT Roorkee.
Description
This project is completed under two phases.
- Software Phase
- Hardware Phase
Sofware Phase
In this phase, a virtual bot is created using ROS URDF by defining its body structure, joints and their degree of freedom. PyBullet is used to simulate physics (gravity, friction, etc.) The checkboard environment is selected as default env. in pyBullet. The simulation is made accordingly to match the real-world scenario (like g=9.8m/s^2) Q-learning is used as a Learning algorithm for the agent implemented using deep Q-Network, a deep neural network.
Hardware Phase
After the software phase, a trained weight is obtained to be deployed and fine-tuned in the real world.
Two 12V DC motors are used of 18000 RPM is used in wheels.L298N motor driver is used for DC motor control.
MPU-9250 is used for sensing purposes.
It has two chips:
- The MPU-6500, which contains a 3-axis gyroscope, a 3-axis accelerometer, and a Digital Motion Processor
- The AK8963, a 3-axis digital compass.
Raspberry pi3 B+ model is used as motherboard with rasbian as operating system. All codes are written in python3
Installation and Set-Up
For running and testing the software simulation, go through the readme of balancebot-project.
Testing
The real-world testing guide will be updated soon.
Learning Experience (Take Aways)
This project’s duration was around 4 months. In the beginning, I had no knowledge of reinforcement learning. So go through theoretical part and learning the mathematics behind, was quite satisfactory.
Key Take-Aways:
- Hands-on experience and fair insight in Reinforcement Learning (particularly Q learning).
- Explored OpenAI for modelling Agents in its different envs.
- Experience with pyBullet (a physics simulation package) and ROS URDF (for creating virtual agents).
- Experience with managing hardware and deploying models.
Learning Resources
- Reinforcement Learning: An Introduction, book by Richard S. Sutton and Andrew G. Barto
- RL course by David Silver, Professor of Computer Science. University College London.
Other resources will be updated soon.
Feedback
All kinds of feedback (code style, bugs, comments etc.) are welcome. Please open an Issue on this Repository.
Contribution Guidelines
If you are familiar with the basics of contributing to GitHub repositories, feel free to skip this section. For total beginners who landed up here, before contributing, take a look at the blog-post to get started. Peace out!