How the Stroop Effect Arises from Optimal Response Times in Laterally Connected Self-Organizing Maps

The Stroop effect is believed to arise from processing in the cortico-striatal network. This network is involved in efficient language processing through the regulation and allocation of cognitive resources such as verbal attention and working memory (Jacquemot \BBA Bachoud-Lévi, \APACyear2021). The cortex-basal ganglia-thalamus loop is a crucial part of information routing and selection for cognitive control. The basal ganglia (BG) acts as a cognitive selection mechanism through by optimizing information gating, as it receives input from the striatum containing processed signals from the cortex, and projects output through the thalamus back to the cortex (Stafford \BBA Gurney, \APACyear2005). The left caudate nucleus of the BG plays an important role in supressing competing alternatives in the indirect pathway and facilitating selected cortical representations, as well as language selection and control (Abutalebi \BBA Green, \APACyear2007; Friederici \BOthers., \APACyear2003).

Neuroimaging studies that focused on goal-directed behavior in the prefrontal cortex (PFC) component of the cortico-striatal network have led to a "cascade-of-control" model examining multiple levels of Stroop interference and cognitive control (Banich, \APACyear2019). The posterior regions of the lateral prefrontal cortex regions bias processing towards task-relevant and away from task-irrelevant information, the caudal mid-cingulate regions are involved in response selection, and the anterior cingulate cortex (ACC) sends feedback to the lateral prefrontal cortex based on response accuracy.

The Stroop effect has been replicated computationally through various neural network (NN) models of automaticity and control processes. In one such approach, parallel distributed processing, word reading and color naming pathways run simultaneously with different connection strengths — word reading has stronger connections due to habitual reading while color naming is weaker (Cohen \BOthers., \APACyear1990). Task demand units modulate attention in these pathways by enhancing processing in one pathway while reducing activation of the other. Normally , the stronger word reading connections result in correct naming with slower reaction times in the incongruent condition, but increased attention in the color naming units can be used to induce errors. Sensory input factors such as color types have been examined using this NN architecture. Using trichromatic processing as an input layer, opponent color pairs (blue-yellow as compared to non-opponent color pairs such as blue-red) resulted in reduced interference in incongruent naming conditions. The NN simulations thus revealed interference at the perceptual encoding stage of color processing in the Stroop task (Laeng \BOthers., \APACyear2005).

Another approach is to build a neuroanatomical model of cognitive interference in Stroop tasks (Nishikawa \BOthers., \APACyear2019). A Deep Neural Network (DNN) was trained to predict task-relevant brain patterns in prefrontal cortex regions, thus capturing neural dynamics of cognitive control. The results suggest that during incongruent conditions, there is increased activation in the medial prefrontal cortex (involved in cognitive control) and left inferior frontal gyrus (involved in language processing). The temporal patterns indicated low neural activation at the start, with progressive increases in activation levels during the Stroop task. This result suggests that cognitive load accumulates over time due to the interference between color naming and word reading.

Further computational studies compared Stroop task performance in normal subjects and in patients with prefrontal cortical lesions. NN modules of sensory-motor networks for color-word processing were integrated with PFC circuit-based inhibition and error detection modules, as well as a habitual response module inspired by the BG (Kaplan \BOthers., \APACyear2007). The BG uses the fronto-cortical-striatal loop to train the cortex to produce automated stimulus-response associations, and PFC allocates attention on task-relevant behavior against competing stimuli by inhibiting automatic responses from the BG. Different combinations of inhibition efficiency and likelihood of generating error signals base values were tested across simulations. In comparisons to normative Stroop task data, the model with higher likelihood of error and low inhibition values performed similarly to patients with prefrontal cortical lesions, with significantly longer interference times and more errors. This paper builds on existing computational models by implementing self-organizing maps (SOMs) to study the emergence of the Stroop interference from the model structure itself.

BiLex consists of self-organizing maps (SOMs; Kohonen (\APACyear1982); Kohonen \BOthers. (\APACyear2001)) that learn to organize word forms and their meanings according to semantic and lexical similarity. Maps are connected by learned associative connections that are used to translate between alternative semantic or lexical representations of a given word. As an example of a lexical task, picture naming is simulated in BiLex by presenting a semantic word representation to the semantic map and propagating the resulting map activation to a lexical map, which then uses the resulting map activation to generate lexical output.

The original BiLex was designed two model Spanish/English bilingual subjects, and therefore had one semantic map and two lexical maps, both modeling spoken words (i.e. phonetic maps). For the Stroop effect, only one lexical map is needed; Spanish was used for the current study. For simplicity, it was used to represent both the concurrent linguistic input and the task output. Figure 1 illustrates the architecture of the model.

Previous versions of BiLex used traditional SOMs, where map activation and learning is governed by a predefined neighborhood function, centered around the maximally responding map unit. In contrast, the SOMs in the present model are laterally connected, i.e. map activation settles over a number of time steps through lateral interactions between map neurons. This settling process is an important extension of the original BiLex: it allows modeling response times which is crucial for the Stroop effect. Inhibitory lateral connections are uniform, i.e. connection strengths between each pair of units are the same. Excitatory connections focus on a small Gaussian map neighborhood with standard deviation $\sigma$. Both the outgoing excitatory and inhibitory connection weights for each map unit sum to one.

When an input $c\in C$ (a feature vector representing its semantic meaning or its lexical word form) is presented to a map, the initial activation for each map neuron $i$ is calculated as

where $u_{i}$ is map unit $i$’s feature vector, $c$ is the map input, $C_{m}$ is the median Euclidean distance between pairs of vectors in the entire input set $C$, and $T$ is a temperature parameter that controls initial activation focus.

The initial activation is then updated for all units $i$ over a number of time steps $t$:

where $\alpha$ controls how fast activation fades, $\beta$ controls the strength of lateral interactions, and $w_{ij}^{+-}$ are excitatory and inhibitory connections strengths between units $i$ and $j$. Over time, long-range inhibition and short-range excitation focus the map’s activation on a group of neighboring similar map neurons.

At each time $t$, the likelihood of a specific map input $c$ is calculated as

where $M_{c}$ is the set of map units $i$ that are closer to $c$ than to any other input in $C$ (shown as background colors in Figure 2). Calculating $L(c,t)$ for all inputs in $C$ gives a probability distribution whose (negative) entropy $E_{t}$ reflects the map’s uncertainty at time $t$ about which input $c$ is observed. In the reported experiments, $E_{t}<1$ was used as a threshold to stop lateral interactions and produce the maximum-likelihood response $c$ as the map’s output.

A set of 16 words was used for model training, each representing a standard color. RGB values of each color were used as semantic representations (e.g. red was (1, 0, 0)), and lexical representations were generated as in previous BiLex models (Peñaloza \BOthers., \APACyear2019), starting with IPA transcriptions of each color name, and transforming them into vectors of phonetic features (e.g. openness for vowels, place and manner for consonants).

Maps were trained similarly to standard SOMs, i.e. by repeatedly presenting the map with a randomly chosen word $c$, and adjusting a map neighborhood to become more similar to the input. However, instead of using a neighborhood surrounding the map unit most similar to the input, the map was presented with the input pattern (Eqn. 1), updated for 30 time steps (Eqn. 2). Each unit’s feature vector $u_{i}$ was then adjusted to become more similar to the input $c$, with more highly activated units changing more:

Where $\eta$ is the map’s learning rate. Over time, as with regular SOMs, this process results in neighboring units becoming more similar to each other, and the map as a whole representing the input vectors more and more closely (as shown on the left side of Figure 2).

Both maps in the experiments consisted of 20x20 units. They were trained for 1000 epochs, with the learning rate decreasing exponentially from 0.2 initially to 0.05 at the end of training. The standard deviation $\sigma$ of excitatory lateral connection weights decreased from 2.0 to 0.25 at the same time.

Associative connections between the semantic and lexical map were trained with Hebbian learning. Both maps were repeatedly presented with the same randomly chosen input color, and the resulting map activations were updated until the map’s entropy fell below one bit. Connections between semantic unit $i$ and lexical unit $j$ were then increased if they responded to the same input:

where $\theta$ is the learning rate for associative connections. In this way, co-activated neurons were connected more strongly over time. Associative connections were trained for 10 epochs; the learning rate $\theta$ was set to 0.1. After each epoch, associative connections were normed such that each neuron’s output weights summed to one.

The Stroop task can be simulated in the model as follows:

1. 1.

   The semantic representation of a color (its RGB values) are presented to the semantic map (Eqn. 1), resulting in an initial map activation like the one for "Maroon" shown in the top right panel of Figure 2.

2. 2.

   A congruent or incongruent lexical input is presented to the lexical map, as shown in the bottom right panel of Figure 2. In the ’no input’ condition, no lexical input is presented to the lexical map. The strength of the interference (referred to below as $r_{\mathrm{lex}}$) can be varied by multiplying the initial activation by a number between 0 and 1, where 0 reflects no interference.

3. 3.

   Map activations for both maps are updated using lateral interactions between map units (Eqn. 2), resulting in incrementally more focused activations.

4. 4.

   Activation is transferred from the semantic to the lexical map through associative connections: $a_{t,j}=a_{t,j}+\gamma\sum_{i}a_{t,i}w_{i,j},$ where $w_{i,j}$ is the associative connection weight between semantic unit $i$ and lexical unit $j$, and $\gamma$ determines the strength of between-map interaction.

5. 5.

   If the entropy $E_{t}$ of the lexical map is below a given threshold, the maximum-likelihood lexical output $c$ (according to Eqn. 3) is produced as the color named by the model. If the maximum number of steps is exceeded, naming fails. Otherwise, return to Step 2.

The Stroop task simulation is counted as successful if the output color produced corresponds to the one presented to the semantic map, independent of the competing lexical input. The number of time steps $t$ is recorded as the time required to complete the Stroop task. A complete factorial design with 16 colors was used to evaluate the congruent (16 input cases), no input (16 input cases), and incongruent conditions (240 input cases).

An important part of setting up the model is to determine the right level of interference, i.e. the relative strength of routing the semantic and congruent lexical input. Optimal routing parameters were found by running the Stroop task simulation with a baseline $r_{\mathrm{sem}}$ value of 0.05 and varying the $r_{\mathrm{lex}}$ value from 0.0 to 1.0 in 0.05 increments. Error rates and response times across all three conditions were recorded. The quality $q$ of each $r_{\mathrm{lex}}$ value was measured as

where $e_{i}$ = mean incongruent error rate, $t_{i}$ = mean incongruent response time, $t_{c}$ = mean congruent response time (there were no errors in the congruent condition and therefore $e_{c}$ was not needed). The min and max indicate the smallest and largest observations of these values; min-max normalization converts $e_{i}$, $t_{i}$, and $t_{c}$ to values between 0 and 1. Note that a low $q$ score can be achieved by a very good performance in one metric even when the performance is poor in the others. The $q$ score thus balances the need to respond fast and to reduce errors at the same time. The $r_{\mathrm{sem}}$ and $r_{\mathrm{lex}}$ values with the best $q$ value were chosen for the rest of the experiments.

In the experiments, the threshold for $E_{t}$ was always one bit. The strength $\gamma$ of between-map interactions was 0.05. The temperature $T$ in Eqn. 1 was 0.7 for the semantic map and 0.5 for the lexical map. $\alpha$ and $\beta$ in Eqn. 2 were 0.8 and 0.1. The maximum number of time steps was set to 500, which was never reached in the reported experiments. All parameters were set empirically. Linear mixed model analysis with the semantic target color as a repeated measure was conducted to determine whether the response time differences were statistically significant between the Stroop conditions.