High Capacity Neural Network Associative Memories




One of the most attractive features of associative memories (AM) is their abilities of associative recall, especially recall by incomplete or noisy inputs. However, most existing neural network AM models suffer from their limited storage capacities. Due to the increased nonlinearity brought in by the hidden nodes, backpropagation (BP) networks (multi-layer feedforward networks constructed by BP learning) are able to associate more pattern pairs than the network size if these pairs are used as learning samples. However, conventional use of BP networks with single passes of forward computing for associative recalls gives these networks only very limited noise resistance capability. This research project is aimed at developing a new class of AM which combines the relaxation dynamics of the recall mechanism of traditional Hopfield type AM models and the representational power of BP networks. The resulting AM may have significantly increased storage capacity (up to 2^n n dimensional binary patterns can be stored in a network of size O(n), according to our recent experiments) yet at the same time maintain a high level of noise resistance capability. This project may deepen our understanding of the mechanism underlying BP networks.



Contact Yun Peng, ypeng@umbc.edu .