Some Fun With Data: Robot Orientation Rest API on Heroku

So this post combines a lot of topics. We recently read a blog post about deploying apps on heroku. And we learned about deploying scikit-learn machine learning models with a restful api using flask. All that got us excited to deploy our own model using flask and heroku, but first we needed a problem to apply machine learning on. So let's start by introducing robi.

Robi is our house cleaning robot. He's very stupid as in he only drives around by rotating. Since he doesn't have any sensors we usually can't really do much with him (other than cleaning) but then we had the idea to add our own sensors! First we wanted to simply put a nintendo 3ds on it, since it has a stereo camera and can therefore produce images and depth maps. However the low resolution of the 3ds camera does not go well with the rotating speed of robi so instead we used our cellphone camera. Now robi can drive around and observe his surroundings.

Next we took pictures of the four sides of our room and trained a machine learning model to recognize them. As features we used primary colors of the pictures which we got through kmeans clustering and finally we trained a linear regression model on those colors. The final model pipeline is built in such a way that given an image of our room it will produce four estimates for which direction it thinks the image was taken. We implemented that by using binary labeling, e.g. 'north' -> (1, 0, 0, 0), 'east' -> (0, 1, 0, 0) and so on. Basically the robot can drive around and have the model estimate which direction it's currently facing. Here's a sketch of how the pipeline looks like:

In our next step we put the model in a flask app offering a restful api. We uploaded that app to our github and created a heroku app which we connected to the git repository. In order to deploy the app on heroku we also needed a package called gunicorn (Not entirely sure why we need that). Our model is currently available on this address: https://robotorientationrest.herokuapp.com/api

Here's a sketch of what the total process looks like. Also we understand that sending image data as json is very inefficient, but for now that's what we do.