From Zero to Hero

Apparently a common question that is asked in job interviews for jobs involving programming is about fizzbuzz. The question is how would you write code that creates numbers from 1 to 100 but replaces all numbers divisible by two with fizz, by three with buzz and by both with fizzbuzz. Solving that can be done using simple for loops and if statements, so instead we came up with a more complex question which we would find interesting in a job interview.

Imagine you had to create a process that analysis data in some meaningful way. You decide what kind of data and what kind of analysis you want to do. Next that process should be put behind an API, there must be a way to send data to that API and receive the analysis result from it. Again the API and data format is up to you. How would you implement that?

We think there are many interesting aspects about this task, like what tools do you use for data processing? Do you implement the API yourself or use some ready-to-use tools? And where do you put the service? Last year we saw an exciting workshop at a PyData event where someone live coded a REST API for a scikit-learn model on an Amazon serverless instance, which in part inspired this idea.

Here's what we came up with as a solution to this task. A 15 line code that implements an http API using flask that expects time series data in a json format. There's a cool package called tsfresh for time series analysis, it basically computes all kinds of features from given time series which can then be used for further processing. Our function runs tsfresh on the data it was sent and returns the results again in a json format.

We put that code on github and connected a heroku app with it. And that was all we needed to make our processing idea a reality reachable from the internet. Since we were able to set this up so quick from nothing and put it on heroku we call our service from zero to hero.

We tested our service by creating some time series data and sending it to our server. As data we used clearsky irradiance time series created with pvlib, which can be sent as json simply by using curl. Here is an example of how we can use our API.

!curl -X POST https://from02hero.herokuapp.com/api -d

'{"level_0":{"0":1467356400000,"1":1467356400000,"2":1467356400000,"3":1467357300000,

"4":1467357300000,"5":1467357300000,"6":1467358200000,"7":1467358200000,"8":1467358200000,

"9":1467359100000}, "level_1":{"0":"ghi","1":"dni","2":"dhi","3":"ghi","4":"dni", "5":"dhi",

"6":"ghi","7":"dni","8":"dhi","9":"ghi"}, "0":{"0":0.0,"1":0.0,"2":0.0,"3":0.0,"4":0.0,

"5":0.0,"6":0.0,"7":0.0,"8":0.0,"9":0.0}}'