On the Dynamics of Small Continuous-Time Recurrent Neural Networks

Dynamical neural networks are being increasingly employed in a variety of contexts, including as simple model nervous systems for autonomous agents. For this reason, there is a growing need for a comprehensive understanding of their dynamical properties. Using a combination of elementary analysis and numerical studies, this article begins a systematic examination of the dynamics of continuous-time recurrent neural networks. Specifically, a fairly complete description of the possible dynamical behavior and bifurcations of one- and two-neuron circuits is given, along with a few specific results for larger networks. This analysis provides both qualitative insight and, in many cases, quantitative formulas for predicting the dynamical behavior of particular circuits and how that behavior changes as network parameters are varied. These results demonstrate that even small circuits are capable of a rich variety of dynamical behavior (including chaotic dynamics). An approach to understanding the dynamics of circuits with time-varying inputs is also presented. Finally, based on this analysis, several strategies for focusing evolutionary searches into fruitful regions of network parameter space are suggested.