Hippocampal Consolidation Hypothesis

I sketched a theory in Section 4.13 of my 1996 thesis of what I think the hippocampus might be doing. In that sketch, shown animated below [note: four trials of the same input (to EC) are shown], a cortical code at time T begets a DG/CA3 code, which begets a CA1/Subic. code. This last code sends a wave of synaptic signals that would arrive back at cortex contemporaneously with intra-cortical signals propagating via the local intrinsic (horizontal) L2/3 matrix, which were also begotten by the original time T cortical code. That original code might not have yet changed [due to assumed longer persistence of EC codes (e.g., persistence = 4 time steps)] or if it has changed, then the updated EC code will be determininstically related, through perhaps several intermediate EC codes, to the original time T cortical code.

If we assume that the synapses impinging or internal to the hippocampus are updated in all-or-none (binary) fashion (heavy arrows), then the rapidly (e.g., single-trial) learned hippocampal trace, can function to assist the more gradual strengthening of intra-cortical weights (i.e., consolidation) (visually suggested here by the gradually thickening intra-cortical arrows) on successive trials. Note: although this animation suggests multiple trials orginating in the environment, the basic concept can accommodate a scheme in which the "repeated trials" actually arise from within the hippocampus itself, consistent with studies showing recapitulation of traces in sleep periods following training experience, e.g., Ji & Wilson (2007) and many others.

So, this model predicts that the spatiotemporal trace (i.e., phase sequence of cell assemblies) laid down in cortex during the initial experience is stable from that initial occurrence. However, initially it is stable only with the assist from the spatiotemporal hippocampal trace that will also have been laid down during that initial occurrence. As the intra-cortical associations strengthen, the purely cortical trace of the experience becomes independent of the hippocampus. Once the intra-cortical pathways are firmly entrenched, the intra-hippocampal pathways that aided consolidation are no longer needed and are free to be overwritten by newly experienced associations. Thus, the weights in the hippocampal path can have much faster forgetting rates than the strictly cortical weights. Thus, while the hippocampus is small compared to the cortex, such a faster forgetting rate protects it from becoming oversaturated, which would lead to information loss due to crosstalk interference. Further protection from interference is likely also conferred by the neurogenesis that has been found to occur in the hippocampus throughout the lifespan.

The animation below provides additional intuition as to the proposed scheme. It suggests that the hippocampus, being at the apex of the cortical hierarchy, essentially provides: a) a complex infrastucture for organizing top-down (recurrent) feedback at the scale of the whole hierarchy, which augments and perhaps provides meta control for the local top-down recurrence between adjacent cortical regions; and b) a subcircuit that has evolved to specifically assist with learning/handling temporal/sequential relations.

Beyond the hippocampus's role in consolidation, the scheme described here also effectively increases the (horizontal) connectivity rate of the entorhinal cortex. It does this by allowing associations to be formed via multi-step mappings. That is, reliable connections between cells separated by more than one synapse can be embedded. This should generally allow richer, more multimodal concepts to be learned.

N.b.: As noted at the top, this overall scenario was conceived during work leading up to my 1996 PhD Thesis. At that time, I had not yet discovered a general functional role for a second synaptic parameter, which I call permanence, which is a measure of resistance to passive decay of a synapse's strength, vis-a-vis the overall problem solved by the brain, which can be summed up in the term "lifelong learning". [Aside: Many are now working on lifelong learning (sometimes called "continual learning" or "continual on-line learning"), having been guided to this problem due to the fundamental (systemic) problem of optimization-based learning of distributed representations (as practiced by mainstream machine learning (ML) models), called "catastrophic forgetting" (McCloskey and Cohen, 1989)]. I realized later that the overall solution to lifelong learning is most efficiently addressed if we introduce this second synaptic parameter, permanence, to the computations. Permanence and its use is described in my 2014 Frontiers paper. The gist is that it is NOT synaptic strength that is modulated (increased) across trials (e.g., across replays instigated by hippocampus), but permanence. Specifically, Sparsey's current learning rule is that whenever a synapse experiences a pre-post, it's strength is set to the max wt. Once increased, that weight will passively decay until it experiences another pre-post. The schedule for how the weight decays is determined by model parameters, but the function essentially resembles a reverse log curve (i.e., slow decay at first, but progressively faster decay). When a subsequent pre-post occurs, the weight is set again to the max. Should that subsequent pre-post occur prior to passively decaying all the way back to zero (i.e., a naive synapse), or perhaps to within some range of zero, in addition to the weight being reset to the max, the synapse's permanence will be increased. That means that it's schedule of passive decay will be substantially increased, probably at least an order of magnitude. And this same principle might be generalized to have several levels permanence (with progressively longer timecourses of passive decay). In practice, we currently have only a few such levels, such that after just a few pre-posts (having fallen within the corresponding timeframes), the synapse's permanence is set to 1, i.e., that synapse is made permanent, it will have that max wt for the remainder of the system's life, which prevents catastrophic forgetting. So, it is not synaptic weights that are the primary thing modulated across hippocampal replays, but rather synaptic permanences. This permanence principle/mechanim that I conceived (Rinkus, 2014) and have implemented (and now am only beginning to test), can be thought of as a kind of "tagging" concept (cf. Buzsaki, Morris, Fusi, others).