Evolution of Adaptation Rules

We have developed a method to evolve the ability to "learn" on-line how to solve a task, instead of evolving a solution for a task as is common practice in evolutionary computation. On this page you find an overview of the method, of several experimental results, and links to the most important articles, video clips, and other resources.

Motivation

When one encodes in the genetic string the weights of a neural network (or any other fixed parameter of a system, such as routing and cell functionalities of FPGA), evolution often comes up with solutions that are tuned to the properties of the evolutionary environment. This has three important consequences:

• If the environment changes, the evolved solution won't necessarily work.
• The direct and fine mapping between genotype and phenotype allows evolution to genetically encode mechanisms that rely upon constant relationships --invariants-- in the environment and/or in the interaction between agent and environment. This promotes minimalism, that is simple and often surprising solutions that rely upon invariants not immediately visible to an external observer, but it is not guaranteed that it scales up to more complex solutions.
• Furthermore, since these invariant relationships, are often invisible to an external observer, it is hard to predict what type of change will cause system failure.

Goals

• Evolve systems that can adapt online to unpredictable sources of change.
• Go beyond minimalist solutions by making it hard for evolution to exploit trivial environmental invariants (such as ratios between wall lengths, patterns of lights, etc.) encoded in genetically specified connection strengths.
• Explore the evolution of new adaptive modalities for artificial neural networks.
• Evolve the ability to incrementally learn, i.e. to acquire new knowledge and abilities and store them.

Method

The method consists of encoding in the genetic string a set of local Hebbian learning rules to be applied to the connection weights, but we do not encode the connections strengths. The initial weights are always set to random values and can change after each sensory-motor loop (100 ms) according to the type of learning rule specified in the corresponding genes. That means that at the beginning of "life" each individual must develop from scratch the abilities necessary to solve the task. It also means that different learning rules can co-exist within the same neural network.

The figure above compares conventional genetic encoding of synaptic weights, whereby an evolved individual is entirely genetically-determined, with genetic encoding of Hebbian learning rules whereby an evolved individual is adaptive during its entire life. Notice that in the latter case the genotype encodes also the sign and the learning rate of the synapses. Both types of genetic encoding occupy the same genetic space.

Reactive Navigation

We first proposed this approach in 1996 and applied it to a simple reactive navigation task. The goal was to evolve a neural controller capable of going as straight as possible while avoiding obstacles. The robot was put in a looping maze shown below that had been used in previous experiments with the genetically-determined approach.

This experiment showed four main results:

• Evolutionary time was much shorter, about half generations than those required by genetically-determined encoding.
• Evolved adaptive individuals developed an obstacle avoidance behavior with straight navigation in few seconds and continuously refined it while moving around the maze.
• The synaptic strengths continuously changed during the motion of the robot, but the overall pattern of change became stable. In other words, the network reached an attractor whereby whenever a synapse went down, it was compensated by another synapse going up.
• When the evolved adaptive robot was put in a completely different environment, such as that shown in the videoclip below, the robot developed a navigation strategy from scratch suitable for that particular environment (genetically-determined individuals instead were not capable of performing well in new environments).

People

• Dario Floreano
• Francesco Mondada

Related documents

• Video clip of an adaptive individual evolved in the looping maze above and put into a new environment with different layout, lighting conditions, and wall reflection properties:
  Download quicktime version (2MB).
• The original conference paper where we describe these results: