Incremental Comprehension of Garden-Path Sentences by Large Language Models: Semantic Interpretation, Syntactic Re-Analysis, and Attention

Garden-path phenomena have long been studied in psycholinguistics. Here, we focus on two studies of the phenomenon exemplified in Figure 1. \@BBOPcitet\@BAP\@BBNChristianson et al. (2001)\@BBCP had participants read 24 such garden-path sentences and tested their final understanding using two yes/no comprehension questions. For the sentence shown in Figure 1, the questions were:

(1) Did the man hunt the deer?

(2) Did the deer run through the woods?

Additionally, \@BBOPcitet\@BAP\@BBNChristianson et al. (2001)\@BBCP studied the effect of adding extra-syntactic information – a comma between chunks 1 and 2 (i.e., ”While the man hunted, the deer ran through the woods.”) – to signal the correct interpretation. This successfully minimized the ambiguity: a greater percentage of participants correctly rejected (1), the probe consistent with the misinterpretation, when the comma was present versus absent. \@BBOPcitet\@BAP\@BBNPatson et al. (2009)\@BBCP found a similar result in their paraphrasing experiment.

A number of studies have investigated the alignment of LLMs with human sentence processing \@BBOPcitep\@BAP\@BBN(Marvin & Linzen, 2018; Wilcox et al., 2019)\@BBCP. Some of these studies, like the current study, have considered the processing of garden-path sentences \@BBOPcitep\@BAP\@BBN(Jurayj et al., 2022; Wilcox et al., 2021, 2023)\@BBCP. However, most have focused on predicting word-by-word reading times, the coin of the realm in psycholinguistics, using surprisal values derived from LLMs. Thus, they have shed little light on the research questions which animate the current study, which concern direct measurement of the semantic and syntactic interpretations that models form when processing garden-path sentences, and whether these interpretations shift following the point of disambiguation.

NLP researchers are becoming increasingly aware of the importance of representing ambiguity in LLMs, which is currently a challenge \@BBOPcitep\@BAP\@BBN(A. Liu et al., 2023)\@BBCP. However, the human alignment of the technical solutions these researchers are developing is outside the scope of the current study.

The current study compared humans and LLMs on the incremental processing of garden-path sentences. We probed the online processing and final interpretations of a range of models to address the following research questions:

1. 1.

   Do LLMs represent the semantic misinterpretation of a garden-path sentence during the ambiguous region (i.e., chunk 3), and after they reach the point of disambiguation (i.e., chunk 4), do they switch to the correct interpretation?

2. 2.

   Is the switch from the misinterpretation to the correct interpretation at the point of disambiguation also reflected in the implicit syntactic parse trees that LLMs construct?

3. 3.

   Is the attention mechanism of transformer-based LLMs sensitive to the point of disambiguation?

We address these research questions using three novel methods that capitalize on the fact that LLMs can be directly interrogated in ways that human minds cannot. For the first research question, we present garden-path sentences to LLMs chunk by chunk. After each chunk, we used the comprehension questions – examples (1) and (2) above – to probe the strength of the misinterpretation and correct interpretation, respectively. This is a direct comparison between LLMs and humans that does not require indirect measures like surprisal \@BBOPcitep\@BAP\@BBN(Hale, 2001; Levy, 2008; Wilcox et al., 2020a)\@BBCP.

A parse tree is a hierarchical representation of the syntactic structure of a sentence. To address the second research question, we use the technique developed by \@BBOPcitet\@BAP\@BBNManning et al. (2020)\@BBCP to extract the parse tree at each chunk as an LLM incrementally processes a garden-path sentence. We evaluate whether this structure shifts at the point of disambiguation (i.e., chunk 4) from the misinterpretation to the correct interpretation.

The third research question is more exploratory in nature. We examine the attention weights of LLMs for evidence of sensitivity to the point of disambiguation (e.g., ran in the example in Figure 1). These reflect how LLMs weigh the different elements of the input. \@BBOPcitet\@BAP\@BBNVig (2019)\@BBCP and \@BBOPcitet\@BAP\@BBNClark et al. (2019)\@BBCP introduced tools to visualize these weights in transformer models. A substantial amount of linguistic information can be found in the attention weights of models \@BBOPcitep\@BAP\@BBN(Clark et al., 2019; Y. Liu et al., 2019)\@BBCP. We ask whether this includes information about the point of disambiguation.

Our study evaluated four LLMs ranging in performance, size, and architecture. We tested GPT-2 \@BBOPcitep\@BAP\@BBN(Radford et al., 2019)\@BBCP and LLaMA-2 \@BBOPcitep\@BAP\@BBN(Touvron et al., 2023)\@BBCP, both decoder-only models with the latter being instruction-tuned. The third model was Flan-T5 \@BBOPcitep\@BAP\@BBN(Chung et al., 2022)\@BBCP, an encoder-decoder LLM. Finally, we evaluated RoBERTa \@BBOPcitep\@BAP\@BBN(Y. Liu et al., 2019)\@BBCP as the most performant encoder-only-architecture model. RoBERTa is trained using a different paradigm (masked language modeling) than GPT-2 (next-token prediction) and LLaMA-2 and Flan-T5 (next-token prediction and instruction tuning).

We used two tasks originating in \@BBOPcitet\@BAP\@BBNChristianson et al. (2001)\@BBCP. The first used garden-path sentences and yes/no comprehension questions like examples (1) and (2) above. Preliminary testing revealed that all models correctly answered probe question (2) corresponding to the correct interpretation. We therefore focused on the models’ endorsement of probe question (1) corresponding to the misinterpretation.

To simulate the incremental processing, we presented sentences to models in chunks and interrogated their unfolding state. Specifically, we split sentences at the following points: (1) through the initial verb; (2) the misinterpreted “direct object”; (3) the descriptive clause; (4) the second verb, or point of disambiguation where we expect models to reanalyze the semantics and syntax of the sentence; and (5) the rest of the sentence, where reanalysis might spill over to; see Figure 1. We prompted the models as follows: prompt completion for GPT-2 and Flan-T5; in a chat format for LLaMA; and masked token prediction for RoBERTa. In all cases, we follow a question-answering template, prompting the language model with the garden-path sentence as context, the corresponding question, and “Answer: ”.

The second task spans the same 24 garden-path sentences and yes/no questions as the first task, but with a comma inserted after chunk 1 (i.e., the first verb; hunted in Figure 1). This extra-syntactic information rules out the incorrect transitive interpretation of the first verb, and thus the misinterpretation that the noun phrase in chunk 2 (e.g., the deer) is its direct object, potentially disambiguating the sentence.

With respect to the datasets, one is from (the final) Experiment 3B of \@BBOPcitep\@BAP\@BBN(Christianson et al., 2001)\@BBCP, where the performance measure was accuracy on the comprehension question (1) corresponding to the misinterpretation. The second is from \@BBOPcitep\@BAP\@BBN(Patson et al., 2009)\@BBCP who replicated this study but used a different accuracy measure: alignment of sentence paraphrases to the misinterpretation. Both datasets showed the same pattern of results: Inaccurate comprehension in the first task, but a positive effect of the extra-syntactic information and more veridical comprehension in the second task.

In our baseline experiment, we compute surprisal for the four models on the first and second tasks, plotted in Figure 2 and Figure 3. For the first task, all models see a modest increase in surprisal at the point of disambiguation (chunk 4), as expected. For the second task, where the comma should disambiguate the sentence, RoBERTa and Flan-T5 show modest evidence of taking advantage of this extra-syntactic information and avoiding the garden path. By contrast, LLaMA and GPT-2 continue to show an increase in surprisal at chunk 4.

The first research question is when incrementally comprehending garden-path sentences, whether LLMs favor the misinterpretation during the ambiguous region (i.e., chunks 2 and 3) and then switch to the correct interpretation at the point of disambiguation (i.e., chunk 4) or afterward (i.e., chunk 5).

Consider the first task, where commas are absent. Figure 4 shows the probability of the misinterpretation (solid lines) and correct interpretation (dashed lines) across the five chunks for each of the four models. The first prediction is that the misinterpretation will be favored during the ambiguous region spanning chunks 2 and 3. This was the case for Flan-T5 and LLaMA, and by a smaller margin for GPT-2: the models assign a higher probability to “yes” to the verification probe (1), which is consistent with the misinterpretation, than to “no”. The second prediction is that at the point of disambiguation, chunk 4, the pattern will reverse and the models will assign a higher probability to “no”. This was not the case; all models continued to endorse the misinterpretation. However, for LLaMA the probability of “no” decreased on chunks 4 and 5, which is a promising trend.

By contrast, the models were more successful for the second task, where a disambiguating comma appears between chunks 1 and 2, suggesting that the noun phrase in chunk 2 should not be misinterpreted as the direct object of the matrix verb in chunk 1; see Figure 5. Specifically, LLaMA favors the misinterpretation during the ambiguous region (i.e., chunks 2 and 3). However, beginning at the point of disambiguation, the probability assigned to the misinterpretation decreases (a trend that continues in the final chunk of the sentence). This drop was statistically significant for LLaMA ($p=0.003$) and also for GPT-2 (with $p=0.05$). However, only LLaMA successfully switched to the correct semantic interpretation.

Turning from incremental semantic interpretation to the final semantic judgment, Figure 6 shows the percentage of garden-path sentences for which the probe question (1) corresponding to the misinterpretation was correctly rejected (i.e., had a “yes” response probability less than 50%) at the end of the sentence for the two tasks and four models. Performance on the first task (i.e., sentences with comma absent) is shown in light blue, and on the second task (i.e., sentences with the comma present) in dark blue. Also shown is the human performance on the two tasks in the \@BBOPcitet\@BAP\@BBNChristianson et al. (2001)\@BBCP study. LLaMA, GPT-2, and Flan-T5 show human-like performance in being garden-pathed for sentences without the comma but capitalizing on extra-syntactic information when it is present and recovering from the garden path. For all three of these models, accuracy is significantly greater on the comma-present vs. absent sentences ($p<0.05$). LLaMA is notable in also showing human-like accuracies on the comma-present and comma-absent. By contrast, RoBERTa-large offers a poor fit for human performance.

The second research question concerns syntactic reanalysis: when LLMs reach the disambiguation point of a garden-path sentence, do they reanalyze the implicit parse tree they are constructing and shift to one consistent with the correct interpretation? Figure 7 shows the incremental parse trees across chunks 1-5 of the example sentence in Figure 1. These were extracted from the RoBERTa-large model. Note that the model initially incorrectly attaches the rocket in chunk 2 as the direct object of the main verb photographed in chunk 1. However, by the final chunk, it has shifted to the correct attachment, as the subject of the second verb sat. (More generally, many of the extracted parse trees identify the correct dependencies of the words and subordinate phrases.)

The performance of GPT-2 and RoBERTa-large across the 24 garden-path sentences is summarized in Table 1. The table also contains the human data from the \@BBOPcitet\@BAP\@BBNChristianson et al. (2001)\@BBCP and \@BBOPcitet\@BAP\@BBNPatson et al. (2009)\@BBCP studies. First, consider the model and human performance on the first task, where the comma is absent. Following chunk 4, the point of disambiguation, especially RoBERTa-large begins to shift to the correct parse tree. By the end of the sentence (i.e., chunk 5), it has computed the correct parse tree for 45.83% of the sentences. This is comparable to the performance observed in especially the original \@BBOPcitet\@BAP\@BBNChristianson et al. (2001)\@BBCP study. These findings coincide with those of \@BBOPcitet\@BAP\@BBNSlattery et al. (2013)\@BBCP, who found that reinterpretation of garden-path sentences spills over to the few words after the point of disambiguation.

Consider the second task, where sentences contain a disambiguating comma between chunks 1 and 2. As Table 1 shows, GPT-2 and RoBERTa-large shift to the correct parse tree after chunk 4, the point of disambiguation, for 45.83% and 50% of the sentences, respectively. The percentages increase to 50% and 62.5%, respectively, by the end of the sentence (i.e., chunk 5). Again, the accuracy of RoBERTa-large is comparable to the human participants in both studies. To summarize, RoBERTa-large performs most similarly to humans.

We confirmed that these descriptive patterns are also statistically significant by $t$-tests. For GPT-2, the presence of a comma had a significant impact on the proportion of sentences on which the model switched to the correct parse tree at the point of disambiguation, i.e., after chunks 1-4 ($p<0.01$), as well as at the end of chunk 5 ($p<0.01$). The results were similar for RoBERTa-large: the presence of a comma had a significant impact on switching to the correct parse tree after chunks 1-4 ($p<0.1$). However, the percentages at the end of chunk 5 are comparable ($p>0.1$). On the other hand, and collapsing across the two tasks, having seen chunk 5 had a significant impact on switching to the correct parse tree for RoBERTa-large ($p<0.01$). But it does not make much difference on the performance of GPT-2 ($p>0.1$).

The final research question is whether the attention mechanism of transformer models is sensitive to the point of disambiguation when processing garden-path sentences. Because LLaMA-2 and RoBERTa-large showed the strongest alignment with human performance in tracking semantic interpretations and extracting parse trees, respectively, we focus on these models.

We defined sensitivity as follows. LLaMA-2 is composed of 40 layers x 40 attention heads per layer; RoBERTa-large is composed of 24 layers x 16 attention heads. After a model processes a garden-path sentence, we can quantify the sensitivity of an attention head to the correct interpretation. Positive evidence is given by the attentional weight between the noun phrase in chunk 2 (e.g., deer) and the verb in chunk 4 (e.g., ran) representing the correct interpretation/attachment. Negative evidence is given by the attentional weight between the same noun phrase and the verb in chunk 1 (e.g., hunted) representing the misinterpretation/incorrect attachment. Then, positive evidence minus the negative evidence gives us the desired index: more positive values indicate good sensitivity, and more negative values have poor sensitivity.

Figure 8 shows the heatmap of the sensitivities across the attention heads of LLaMA-2 (left column) and RoBERTa-large (right column) for the comma-absent sentences in the top row. The heatmaps for the comma present sentences are in the middle row, and the thresholded difference of the middle row minus the top row is in the bottom row. The bright cells in the top and middle rows indicate the attention heads sensitive to the point of disambiguation, and thus the correct interpretation/parse, for the sentences of the first and second tasks. The bright cells in the bottom row indicate the attention heads that show particularly increased sensitivity in the context of the disambiguating comma of the second task.

These findings are exploratory in nature, and thus valuable in the new questions they raise. For example, one is whether the same attention heads that are sensitive to disambiguating information for the garden-path sentences studied here are also sensitive to disambiguating information in other temporarily ambiguous sentence structures.