Effects of synaptic connectivity on liquid state machine performance

Abstract

The Liquid State Machine (LSM) is a biologically plausible computational neural network model for real-time computing on time-varying inputs, whose structure and function were inspired by the properties of neocortical columns in the central nervous system of mammals. The LSM uses spiking neurons connected by dynamic synapses to project inputs into a high dimensional feature space, allowing classification of inputs by linear separation, similar to the approach used in support vector machines (SVMs). The performance of a LSM neural network model on pattern recognition tasks mainly depends on its parameter settings. Two parameters are of particular interest: the distribution of synaptic strengths and synaptic connectivity. To design an efficient liquid filter that performs desired kernel functions, these parameters need to be optimized. We have studied performance as a function of these parameters for several models of synaptic connectivity. The results show that in order to achieve good performance, large synaptic weights are required to compensate for a small number of synapses in the liquid filter, and vice versa. In addition, a larger variance of the synaptic weights results in better performance for LSM benchmark problems. We also propose a genetic algorithm-based approach to evolve the liquid filter from a minimum structure with no connections, to an optimized kernel with a minimal number of synapses and high classification accuracy. This approach facilitates the design of an optimal LSM with reduced computational complexity. Results obtained using this genetic programming approach show that the synaptic weight distribution after evolution is similar in shape to that found in cortical circuitry. © 2012 Elsevier Ltd.