Transitions among metastable states underlie context-dependent working memories in a multiple timescale network

Our model is based on a previous study[9] that showed that an RNN with multiple timescales enables the learning of sequential neural patterns including non-Markov sequences. The model has two populations with different timescales connected to each other (Fig. 1A). One population comprises $N$ fast neurons $x$, and the other comprises $N$ slow neurons $y$. The fast population receives an external stimulus and generates a target response corresponding to the given input. These neurons evolve according to

where $J_{ij}^{X}$ is a recurrent connection from the $i$ to $j$-th neuron in the population of $x$ and $\boldsymbol{J}^{XY}$ is a connection from the $i$-th neuron in the population of $y$ to the $j$-th neuron in the population of $x$. The mean values of $J^{X}$ and $J^{XY}$ are set to zero with a variance equal to 1/$N$. $N,\beta_{x}$, and $\beta_{y}$ are set to $100,2.0$, and $2.0$, respectively, while the time scales of $x$ and $y$, denoted as $\tau_{x}$ and $\tau_{y}$, are set to 1 and 33, respectively. $I_{i}$ is the $i$ th element of an input pattern.

In our model, only $J^{X}$ is plastic and changes according to

where $\epsilon$ is the learning speed, and it is set to $0.03$. $\xi_{i}$ is the $i$-th element of a target $\boldsymbol{\xi}$, which is an $N$-dimensional pattern. In previous studies[7, 8, 5], we demonstrated that a single population with a single timescale learns mappings between constant input and target patterns using this learning rule. In the current two-population model, there are two inputs for the fast sub-network—one is from an external input and the other is from the slow sub-network that stores previous information. Thus, the synaptic dynamics can modify the connection to generate a target pattern depending not only on the currently applied input but also on the preceding input, as shown in [9]. We, in the present study, demonstrate this model performs a context-dependent task.

First, we consider learning a simple context-dependent task that is composed of two context signals, a delayed signal and a cue signal (Fig. 1B). In this task, one of the context signals is applied to fast neurons, followed by the delayed signal. Finally, the cue signal is applied. On applying the cue signal, the network is required to generate “Go” pattern or “No-Go” pattern depending on the context signal. When the applied context signal is $A$, the network should generate the Go pattern, while it should generate the No-Go pattern when the context signal is $B$. Here, we denote Go and No-Go patterns as $R_{A}$ and $R_{B}$, respectively. Thus, the network must maintain the context signal to generate an adequate response pattern.

The signal and response patterns are random $N$-bit binary patterns, each element of which corresponds to a neuron of $x$, with probabilities $P(X_{i}=\pm 1)=1/2$, where $X_{i}$ is the activity state of the $i$-th element of patterns $X=A,B,C,R_{A},$ and $R_{B}$. All elements of the delayed signal $D$ are $-1$. We apply the context, the delayed, and the cue signal sequentially with duration time $T_{stim}=60$ as an external input $\eta$ in Eq. 2, as illustrated in Fig. 1B. If the fast dynamics reach $R_{A}$ (or $R_{B}$), namely $\sum{R_{A}}_{i}x_{i}/N>0.95$, a trial is completed, and the next trial starts. Now, the target patterns are defined only when the cue signal is applied. Therefore, the learning process runs only during the cue signal application; otherwise, only the neural dynamics run.

We show successful trials in the context-dependent task in Figure 2A. In the presence of the context signal $A$, fast neural dynamics already show a high overlap with $R_{A}$, while they show quite different patterns in the presence of $B$. The slow dynamics follow the fast dynamics in both cases. The fast dynamics change slightly upon the delayed signal after the context signal $A$, while they change drastically after $B$. Although the network receives the same delayed signals, quite different neural dynamics emerge owing to the slow dynamics that reflect the preceding context signals. Finally, the fast dynamics converge to the correct target $R_{A}$ or $R_{B}$ depending on the preceding context signals $A$ and $B$, respectively. We measured the success rate across 20 trials for each context signal, resulting in a 95% success rate ( Fig. 2B).

To examine the role of the slow dynamics in this task, we analyzed the neural dynamics with $\tau_{y}=1$, i.e., the dynamics of the two populations change with the same timescale. In this case, the neural dynamics $y$ do not store the history of the fast dynamics $x$ over a long time. We plot two trajectories for the context A and B in Fig. 2C. In the present of the context signals, two trajectories show different behaviors depending on the identity of the context signals. After the presentation of the context signal, the difference between the two trajectories rapidly decreased during the delayed epoch and the cue one. Thus, the network generates the same neural patterns (in this case, $R_{A}$) in both contexts $A$ and $B$ in the presence of the cue signal. Altogether, these results show that the the population with the slow timescale enables successful performing the context-dependent task.

In the task analyzed above, the network is required to store only the context information irrespective of the cue signals; this is because the correct response is uniquely determined by the context signal. However, the required responses are commonly dependent on cue signals in addition to context signals, as analyzed in [11, 19]. Subsequently, we examined whether our model performs such a complex context-dependent task (denoted as a delayed match to the sample task), as shown in Fig. 3A. This task is based on procedure similar to the task analyzed above. There are two context signals, $A$ and $B$, and two cues, $a$ and $b$, and the same-name patterns (i.e., $A$ and $a$, or $B$ and $b$) belong to the same category. A network is required to “Go” (denoted as matched pattern $M$) when the context and cue signal in the same category are given, while it is required to “No-Go” (denoted as non-matched pattern $NM$) when these cues are in a different category. Unlike the previous task, the network should generate two different patterns depending on the cue signals, in addition to the context signals.

The context and cue signals ($A,B,a$, and $b$) and response patterns ($M$ and $NM$) are the random $N$-bit patterns generated in the same manner as in the previous task. There are two $M$ patterns for $A-a$ and $B-b$ as well as two $NM$ patterns for $A-b$ and $B-b$. When the two $M$ ($NM$) patterns are the same, the learning performance is poor. Thus, we add random perturbation parts to $M$ and $NM$, as shown in Fig. 3A. The delay signal and learning procedure are the same as that in the previous task.

First, we present four trajectories corresponding to the four conditions (two contexts by two cues) in Figs. 3B-D. Here, the fast neural dynamics are projected onto the 2D principal component (PC) space after averaging over 20 trajectories for each condition. The projected trajectories for four different conditions are shown by solid and dotted lines in different colors.

In the presence of context signals, neural trajectories are separated into two groups according to the context signals and converge into neural states $S_{A}$ and $S_{B}$. Following the context signals, the delay signals are applied. Two groups of trajectories evolve from $S_{A}$ and $S_{B}$ and reach $S_{A}^{\prime}$ and $S_{B}^{\prime}$ with a decrease in the distance between the two groups. Although the difference decreases, that between neural states in the slow dynamics remains high at the end of the delay epoch, and the information of the preceding context signal is retained in the slow dynamics. Finally, applying the cue to the network separates the four trajectories into the match ($S_{M}$) and non-match ($S_{NM}$) states depending on the contexts and cues. The trajectories diverge into different states in the presence of the same cue owing to the slow dynamics that reflect the context signals. In this manner, the network performs the delayed match to sample task. To precisely evaluate the performance of this model, we measured the success rate for 20 trials for each condition, resulting in more than $90$ % trials being successful for all conditions.

In a noisy system such as a neural system, information processing should be robust against noise and perturbations. We explore the robustness of our model against perturbations in the initial states and the change in stimulus duration $T_{stim}$. First, for the modification of $T_{stim}$, we changed $T_{stim}$ for each epoch (the context signal, the delay signal, and the cue signal) from $T_{stim}=60$ to $T_{stim}=66$ in the test process after training the network with $T_{stim}=60$. We measured the success rate by 20 trials for modified $T_{stim}$ and observe that such perturbations do not reduce the success rate compared to the normal situation, as shown in Fig. 3E.

Next, we analyze the robustness against the perturbations of the initial states, wherein the initial states of the fast and slow dynamics, $\boldsymbol{x}$ and $\boldsymbol{y}$, in the test process are selected randomly from a larger distribution around the origin in the neural state space than those in the learning process. We set the initial states of $\boldsymbol{x}$ and $\boldsymbol{y}$ in the test process to random $N$-dimensional states uniformly sampled from a closed interval $[-0.2,0.2]^{N}$, whereas in the learning process, the initial states are limited to a smaller closed interval $[-0.01,0.01]^{N}$. Therefore, almost all initial states in the test process are novel for the network. By measuring the success rate against 20 trials for the perturbation case, we observed that $90$ % of all trials were successful for three conditions, while more than $50$ % of all trials were successful for one condition (context A – cue b). The network successfully performs the delayed match to sample task with highly volatile initial states, except for A – b condition. In total, these results indicate that our learning process generates a network model robust against perturbations of the initial states and signal duration.

Why is such robust information processing possible? To answer this question, we analyzed the stability of neural trajectories in each epoch (i.e., in the presence of context, delayed, and cue signals) in the test process. First, to explore the robustness during the signal epoch, the initial states are randomly generated around $S_{O}$ in the same manner as the perturbation case, and the fast and slow dynamics run according to Eqs. 2 and 3. Figure 4A demonstrates these trajectories in the presence of the context $A$. Neural trajectories from the perturbed initial states converge to and remain around $S_{A}$ for some time. $S_{A}$ is stable and attracts trajectories starting from a broad area of the neural state space in the context $A$ epoch. The same analysis in the delayed and cue epochs revealed that $S_{B},S_{A}^{\prime},S_{B}^{\prime},S_{M}$, and $S_{NM}$ are stable states in the corresponding epochs, as shown in Figs. 4B and C. After changing the signal, each state is destabilized. $S_{A}^{\prime}$ and $S_{B}^{\prime}$ are stable for the same delay signals depending on the preceding contexts. $S_{M}$ and $S_{NM}$ are stable dependent on the context and the cue. The slow dynamics store the context information and stabilize the different states through interaction with the cue signal. These results demonstrate that successive stabilization and destabilization of the epoch-specific states underlie the stable performance of the delayed match to the sample task.

In contrast, $S_{MN}$ in the presence of $b$ (i.e., the correct target state for A – b condition) is less stable than the other task-related states, resulting in some trajectories not reaching $S_{NM}$, but $S_{M}$, as shown in Fig. 4D. Thus, the task often fails. These results indicate that our model robustly performs a delayed match to sample task by sequential stabilization of the task-related states.