A neural net architecture based on principles of neural plasticity and development evolves to effectively catch prey in a simulated environment

One of the most profound challenges of research into artificial life (A-Life) is to construct agents whose behavior is generated in ways deeply analogous to the way an animal’s brain drives its motions. Many researchers argue for using neural networks to control behavior [1, 2, 3, 3] and some favor an approach that builds an ersatz nervous system, whose operations mimic aspects of an animal’s nervous system [4, 5, 6].

However, there is no consensus about how closely construction must hew to biology Furthermore, for large neural networks, the number of possible network configurations is so large that it is difficult to search for network constructions that drive adaptive or effective behavior. Evolutionary algorithms have a solid track record for solving problems with a large search space [7]. We introduce here an evolutionary approach to constructing a neural network controller for a small animat in a 2-dimensional world.

Our approach is distinctive in several ways. First, behavior will emerge through dynamic equilibria of the network and continuous feedback from the surroundings carried by a rich array of simple sensors, rather than having the network compute a representation of behavior, which is then implemented by mechanical systems. The network architecture will be recurrent feedback loops, analogous to loops between cortex and subcortical structures, such as thalamus and basal ganglia. Second, neurons will compete and oppose the actions of other neurons, so that the dynamic equilibrium may be rapidly altered as circumstances change. Third, each instance of a controller will be constructed by methods modeled on developmental processes in animal brains, as detailed below. To our knowledge, this is the most life-like approach yet taken to constructing A-life simulations.

In our simulations, agents started by capturing an average of 2 or 3 food items in generation 1 and usually attained an average of  25 food items per 5 minutes simulated lifetime after 50 or 100 generations at which point their performance stabilized until several thousands of generations, at which point there typically was a rapid increase in fitness, which then plateaued at several hundred food items captured, while showing a strong preference for the high-calorie prey. After the second larger jump in most runs, in the stable population regime, agents ate an average of 282 prey items, with a standard deviation of 151. After performance of the top 10One issue for assessing performance, is how much the measure reflects the genome and how much it reflects (random) starting conditions. To test this we extracted 10 successful agents within the stable population regime, and then placed them in 100 random worlds, in which the starting location of the prey is randomly generated, we see that the distribution of food eaten for each agent throughout the random worlds (4) begins to look similar to the histogram of food eaten for all agents (successful or not) in the stable population regime. Looking at the histograms visualizing these count numbers, it’s easy to spot the pattern: the lackluster agents consume next to nothing, there is a relatively large gap in-between, then there is a larger group of well-performing agents arranged in a (left-skewed) normal distribution. The most successful agents were exceptional about identifying and chasing prey. These agents seemed to set and follow a trajectory for the nearest cluster of prey items, then after momentum carried the agent out of the group, they could loop back around or find the new nearest cluster of prey, which is an impressive behavior, as the agent moves quite quickly. The agent can traverse the entire range encompassing all 50 prey in 1 second of simulated time. This is in contrast to the behavior of agents before the jump, whose movement was characterized by a large, consistent looping pattern - a relatively successful strategy, but largely inefficient, which we termed “filter feeding.” Evolutionary theorists often ask how inevitable a specific evolutionary outcome is. Although so far the large majority of our simulations have found a stable high-performing population, the distributions of construction parameters (‘genes’) are very different from each other, any pair overlapping less than 1%. This indicates that even in this simple 22-parameter system a great many distinct possible optima, with comparable performance, may be found.