2.8.9. Topology Optimization by Genetic Algorithms

The application
of genetic algorithms for evolving neural networks is not only limited to the
selection of input variables but can also be used for optimizing the complete
topology of the neural networks [118]-[121].
The different approaches found in literature can be classified according to
their encoding mechanism as direct and indirect methods. In the direct methods,
all the information about the structure is directly represented in the chromosome.
The most common way is the representation of the connections in a matrix (connectivity
matrix) and then linking this matrix row by row into the chromosome. The indirect
encoding methods are also called grammatically encoding methods as the chromosome
contains development rules, which have to be interpreted to build the corresponding
net. This allows a compression of the topology resulting in a smaller length
of the chromosome and thus a better scalability.

A promising
approach of genetic algorithms for the optimization of neural network topologies
was proposed by Boozarjohehry et al. [145] using a grammatically encoding procedure.
This algorithm is also applied to real world data of a neutralization process
besides of simple benchmark problems. Yet, a problem of this approach is that
the solutions found by the algorithm are randomly depending on the initial weight
initialization and on the parameters. Another very complex approach for evolving
neural networks by genetic algorithms using a direct encoding was proposed by
Braun et al. [146],[147].
The corresponding software ENZO is available for free [148]
and has been applied to several real world problems [149]-[151].
The problem of this approach is its complexity with more than 100 parameters,
which can be adjusted by the user. Although the default settings work well in
many cases, the excellent results demonstrated in the references mentioned before
need adjustments of these parameters rendering a general application of this
approach with only little input by the analyst virtually impossible.

In general
all approaches of optimizing the topology of neural networks by genetic algorithms
are faced by a poor scalability [152] and by
complex genetic operations [147]. An example is the structural
mapping causing problems to the crossover operator. For two networks with an
identical topology, the contributions of the hidden neurons to the overall solution
may be internally permuted (only visible by a permutation of the weights). If
a crossover operator is applied to these networks, one offspring is created
with partly doubled internal contributions and one offspring is created with
partly missing internal contributions. In most cases, the optimization of the
neural network topology has been used only for simple benchmarks like the XOR
problem [152]. Another general problem is that similar
to the pruning algorithms the networks cannot be bigger and more complex than
a largest possible reference network predefined by the user (see also section
2.8.8). Due to these quite complex problems, the genetic algorithms are
used only for a variable selection and not for the optimization of the topology
in this work.