Evolution of Adaptation Rules
We have developed a method to evolve the ability to "learn" on-line
how to solve a task, instead of evolving a solution for a task as is common
practice in evolutionary computation. On this page you find an overview of the
method, of several experimental results, and links to the most important articles,
video clips, and other resources.
When one encodes in the genetic string the weights of a neural
network (or any other fixed parameter of a system, such as routing and cell
functionalities of FPGA), evolution often comes up with solutions that are tuned
to the properties of the evolutionary environment. This has three important
- If the environment changes, the evolved solution won't necessarily work.
- The direct and fine mapping between genotype and phenotype allows evolution
to genetically encode mechanisms that rely upon constant relationships --invariants--
in the environment and/or in the interaction between agent and environment.
This promotes minimalism, that is simple and often surprising solutions
that rely upon invariants not immediately visible to an external observer,
but it is not guaranteed that it scales up to more complex solutions.
- Furthermore, since these invariant relationships, are often invisible
to an external observer, it is hard to predict what type of change will cause
- Evolve systems that can adapt online to unpredictable sources of change.
- Go beyond minimalist solutions by making it hard for evolution to exploit
trivial environmental invariants (such as ratios between wall lengths, patterns
of lights, etc.) encoded in genetically specified connection strengths.
- Explore the evolution of new adaptive modalities for artificial neural networks.
- Evolve the ability to incrementally learn, i.e. to acquire new knowledge
and abilities and store them.
The method consists of encoding in the genetic string a set of local
Hebbian learning rules to be applied to the connection weights, but we do not
encode the connections strengths. The initial weights are always set to random
values and can change after each sensory-motor loop (100 ms) according to the
type of learning rule specified in the corresponding genes. That means that
at the beginning of "life" each individual must develop from scratch
the abilities necessary to solve the task. It also means that different learning
rules can co-exist within the same neural network.
The figure above compares conventional genetic encoding of synaptic weights,
whereby an evolved individual is entirely genetically-determined, with
genetic encoding of Hebbian learning rules whereby an evolved individual is
adaptive during its entire life. Notice that in the latter case the genotype
encodes also the sign and the learning rate of the synapses. Both types of genetic
encoding occupy the same genetic space.
We first proposed this approach in 1996 and applied it to a simple reactive
navigation task. The goal was to evolve a neural controller capable of going
as straight as possible while avoiding obstacles. The robot was put in a looping
maze shown below that had been used in previous experiments with the genetically-determined
This experiment showed four main results:
- Evolutionary time was much shorter, about half generations than those required
by genetically-determined encoding.
- Evolved adaptive individuals developed an obstacle avoidance behavior with
straight navigation in few seconds and continuously refined it while moving
around the maze.
- The synaptic strengths continuously changed during the motion of the robot,
but the overall pattern of change became stable. In other words, the network
reached an attractor whereby whenever a synapse went down, it was compensated
by another synapse going up.
- When the evolved adaptive robot was put in a completely different environment,
such as that shown in the videoclip below, the robot developed a navigation
strategy from scratch suitable for that particular environment (genetically-determined
individuals instead were not capable of performing well in new environments).
- Video clip of an adaptive individual evolved in the looping maze above and
put into a new environment with different layout, lighting conditions, and
wall reflection properties:
Download quicktime version (2MB).
- The original conference paper where we describe these results: