[nupic-theory] Proposed temporal pooling mechanism
jakebruce at live.com
Sun Aug 24 01:29:07 EDT 2014
The new temporal pooling (name-for-a-sequence) mechanisms have been mentioned a lot on this list, but the new developments haven't been discussed much in detail. I suppose these developments happen primarily at Numenta's physical location. As a result of the sparse discussions I've been doing some independent work, and I've come up with a pooling mechanism that may be identical, similar, or unrelated to the official new temporal pooling. I'm hoping for some feedback on biological and information-theoretic plausibility, and maybe some insight into the official concepts in development.
Some of the discussion by Jeff and others mentioned that the new mechanisms operate on the proximal segments in a feedforward manner, so here's my feedforward approach to temporal pooling.
- All the cells of the child region send feedforward input to cells of the parent region. - Assuming we are at time step t, with an SDR formed as per usual, from activation of child cells.- For time step t+1, active parent cell C1 only changes state if enough of its input bits were from bursting (unpredicted) columns in the child region.
A closely related but different approach: parent cell C1 could keep its active state until a cell C2 comes along with an activation stronger than the one that activated C1, and replaces it. The usurping activation threshold would slowly decay, to avoid deadlock.
The way I see it, with a design like this, parent SDRs will represent the attractor states derived from a child sequence, so while they may not uniquely name the sequences, they might be statistically related enough to enable the desired sequence-naming behavior. And this would have the desired property of parent patterns changing slowly while child sequences are predicted, but quickly ("bubbling up" the hierarchy) when not. And since each cell decides whether to change state independently, the moment-to-moment difference between changing patterns is on a continuum depending on the novelty of the sequence, rather than all-or-nothing. That seems like a good thing.
Checking the number of cell activations that come from bursting columns strikes me as biologically unlikely, but having a refractory period where a cell must be beaten by an even stronger cell before it gives up its active state seems more plausible.
What do people think of this: does it seem reasonable?
And to the folks at Numenta, does this resemble the new temporal pooling mechanisms at all?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the nupic-theory