Evolutionary Regression

Genetic or evolutionary algorithms are algorithms that are the result of an optimization process inspiried by evolution. It is very similar to genetic optimization in that you start with an initial population, rank them using a fitting function and create new populations using mutation and cross breeding operations. Even though genetic algorithms can be used for a variety of problems we tried to apply them on regression tasks.

You can think of a function as a tree where every node holds an operation, like multiplikation of its children or applying exponential or trigonometrical functions. We built some function trees using multiplication, subtraction, addition, exponential, sin and cosin functions. Our trees consisted therefore of three kind of nodes, functions, constants and input parameters. Sometimes the results are a bit funny in that a tree that holds 0.98 * sin(x) can be written as sin(x) * exp(cos(4.7)) wich approximates sin(x).

We applied our approach on a few regression tasks and compared the results with linear regression:

The tree defining code was inspired by Toby Segarans Collective Intelligence book and is on our github. We did the computations in a notebook.