SDM Read Operation

For the connection to Attention we focus on the SDM read operation and briefly summarize the write operation: all patterns write their pointers pp in a distributed fashion to all neuron addresses located within Hamming Distance d. This means that each neuron will store a Superposition of pattern pointers from those pattern addresses within d: x⌧v = P {p:d(paμ,x⌧a)d,8μ} pp. Having stored patterns in a distributed fashion across nearby neurons, SDM’s read operation retrieves stored pattern pointers from all neurons within distance d of the query and averages them. This average is effectively weighted because the same patterns have distributed storage across multiple neurons being read from. The pattern weighting will be higher for those patterns with addresses nearer the query because they have written their pointers to more neurons the query reads from. Geometrically, this weighting of each pattern can be interpreted as the intersection of radius d circles2 that are centered on the query and each pattern address paμ for all μ. A high level overview of the SDM read and write operations is shown in Fig. 1.

[…]