Learning associations for directly and indirectly represented features
A.R.Gardner-Medwin, Dept. Physiology, UCL, London WC1E 6BT &
H.B.Barlow, Physiological Laboratory, Cambridge CB2 3EG

Some features of the visual environment have a 1:1 correspondence with the activity in particular cells: we call these directly represented features. The presence of other features may be inferred from the pattern of activity in many cells, but not from any one cell alone. The features for which animals can establish that associations are significant (i.e. not due to chance co-occurrence) are not necessarily only directly represented features. This presents a problem, since the information necessary for learning associations is not available all at one site and cannot be combined according to optimal statistical algorithms. This leads to statistical inefficiency, i.e. the need to accumulate larger amounts of data from the environment than would be needed with a direct representation, before conclusions achieve a given degree of statistical reliability.

We have considered classes of features that have a strict representation on a subset of the cells within the nervous system, but not a direct representation: a precise pattern occurs on the subset when the feature occurs, but no single cell is active exclusively with the feature. The features represented in this way are mutually exclusive events for the subsystem. Efficiency for learning associations for features or events with such representations is less than for direct representations. Efficiencies >50% are readily achieved however (with certain assumptions), provided a parameter ?=Z?p/<?> [see definitions below] is greater than 1. High efficiency is favoured by a low mean activity ratio <?>. For events of differing probability, uniform efficiency is obtained by ensuring that common events have activity ratios below average and that rare events are represented with more active cells.

Distributed representations of features for which associations are to be learned have the advantage over direct representations that the number of representable events on a particular number of cells is hugely greater: e.g. of order 1024 instead of 103 for Z=1000, ?=0.01. The loss of learning efficiency is only modest (<50%). Since ?(?p)=<?>, summing over all the events actually experienced (i.e. for which p>0), it follows that the number of actual events that can be handled with ?>=1 (and therefore with reasonable efficiency) is limited to Z, the number of cells. This number is no greater than could be handled with direct representations on the same number of cells. However, the use of distributed representations is much more flexible since, with fixed rules of representation, the actual events can be almost any selection out of so much larger a population.

Preliminary results have been published (J.Physiol.(1992) 452,282P).