Active Vision Project > Overview > Coevolution

Coevolution of Active Vision & Feature Selection

Dario Floreano, Toshifumi Kato, Davide Marocco and Eric Sauser

Abstract

Introduction

Computational complexity of visual processing can be greatly simplified by the co-development of active vision and feature selection. Active vision is the sequential and interactive process of selecting and analyzing parts of a visual scene (Bajcsy 1985, Bajcsy 1988, Ballard 1991). This process can simplify the computation involved in vision processing by reducing the information load on the system and by selecting only characteristics of the visual scene that are relevant for the task to be solved (Aloimonos 1993). Active vision is largely inspired by ways in which both mammals (Yarbus 1967) and insects (Srinivasan et al. 1997) gather information from their environments. Feature selection instead consists of filtering the image in order to enhance features that are relevant for the task to be solved and discard all the rest. In computer vision, filtering can be accomplished by convolving images with a set of operators, such as the Difference of Gaussians (Marr 1982) to detect edges and discard overall illumination. In biological vision, filtering is implemented by means of matched receptive fields and lateral connections that respond maximally only to some properties of the image. An example is the pattern of center-surround, antagonist synapses found in the early stages of both verterbrate and invertebrate vision systems. However, the combination of active vision and feature selection has been investigated only to a limited extent. The dominant approach in computer vision consists of defining the set of features that an active vision system exploits to explore a visual scene. Interestingly, most models do not take into account that the number and type of visual features that an organism is sensitive to depend also on the sensory-motor and behavioral characteristics of the organism in its environment (Gibson 1979). The co-development and interaction of feature sensitivity and active vision behaviors are still largely unexplored. From a designer perspective, an interesting complication is that behavior is both determined by visual information and at the same time affects what type of visual information is gathered. This is probably the reason why either one aspect or the other are predefined and fixed in most engineered systems. We describe a series of experiments on co-evolution of active vision and feature selection for behavioral systems equipped with primitive retinal systems and deliberately simple neural architectures. In a first set of experiments, we show that sensitivity to very simple features is co-evolved with, and exploited by, active vision to perform complex shape discrimination. We also show that such discrimination problem is very difficult for a similar vision system without active behavior. In a second set of experiments, we apply the same co-evolutionary method and architecture for driving a simulated car over roads in the Swiss alps and show that active vision is exploited to locate and fixate simple features while driving the car. In a third set of experiments, we apply once again the same co-evolutionary method and architecture to an autonomous robot equipped with a pan/tilt camera that is asked to navigate in an arena located in an office environment. Evolved robots exploit active vision and simple features to direct their gaze at invariant features of the environment and perform collision-free navigation. Despite different experimental conditions, evolved systems display similar strategies for active vision and feature selection across the three experiments.

Top

1. Shape Discrimination

Toshifumi Kato and Dario Floreano

System Overview

Filtering Methods

Variable Resolution

Active Motion

Neural Architecture

Figure: Architecture of Neural Network

• 6 OUTPUTS
  
  • Two outputs corresponds to the answer of the Retina Window. One node for a triangle and the other for a square. The Retina Window takes higher one.
  • The other two outputs corresponds to the image processing of the Retina Window. One node selects filtering method, sampling or averaging, and the other node selects the size of the Retina Window.
    The Retina Window size variations: 15 by 15(side by side, in pixels), 30 by 30, 60 by 60, 240 by 240(Only once per image, at the beginning)
  • The last two nodes control the action of the Retina Window. One node determines how far it moves to the next position(from 0 to 50 pixels) and the other node computes which way to go(from 0 to 359 degree).
• 17 INPUTS
  
  • 9 input nodes take its values from the 9 cells of the Retina Window.
  • 6 inputs are recurrent connections of the outputs of the net. They are previous output values.
  • A Step consists of following executions.
    1: Select the size and filtering method.
    2: Get input values.
    3: Compute output values.
    4: Compute next position of the Retina Window
    
  • One node is a boundary bit. When the Retina Window tries to go out of the image, this node is set to 1. Otherwise it is 0. The Retina Window will be fixed back at the boundary. It is not allowed to go out of the image.
  • It has a bias.

Genetic Encoding

Experimental Details and Videos

• Simulation 1: Triangle-Square (including videos)
• Simulation 2: Triangle-Square (including videos)
• Simulation 3: Concave-Square (including vidoes)

An Evolved Retina

Conclusion

Publications