Context-dependent communication under environmental constraints

In language sciences and philosophy, there is a marked tension between the perceived invariance of the meaning of natural language and its inevitable interaction with the actual context of usage. In this vein, it is common to distinguish between semantics, concerned with the context-independent meaning, and pragmatics, investigating the contribution of the context (for an overview, see Korta and Perry, (2020)). In much of linguistic work, the focus is on semantics, while pragmatics is peripheral (Steffensen and Harvey,, 2018).

Although such a relatively narrow framing of natural language may have been useful for a number of research goals, there is ample evidence that, in real-world communication, context-independent meaning is an exception rather than a rule. Human interlocutors seem to readily produce messages that are ambiguous (Wasow et al.,, 2005), relying on the context to complete their meaning (Barwise and Perry,, 1983). The mode and extent of this dependence vary widely: the context guides our understanding of homonyms such as bat (are we playing baseball or watching the night sky?); it allows us to interpret approximate quantities such as high price (how much do I usually spend on this?); and deictic expressions like this are perhaps the most vivid examples, as they cannot function without a situational or linguistic context. Yet humans understand language in context so effortlessly that potential misunderstandings are often only noticed by the linguist (Ferreira,, 2008).

In this work, we focus on two questions. First, what pressure is sufficient for the emergence of context-dependent communication (why it may be preferred over unambiguous communication)? Second, what environmental conditions and cognitive capabilities enable such communication? To address these issues, we turn to agent-based models as tools to examine various factors shaping language evolution in simplified and idealised simulations (Cangelosi,, 2001; Steels,, 2011; Cangelosi and Parisi,, 2012). We adopt a variant of the Lewis, (1969) signalling game – a cooperative scenario in which the sender’s task is to communicate the referent that the receiver must identify in a limited set. Within this model, we introduce the communication context as a set of possible referents, selected at random in each communication episode.

Even in the simple scenario of the signalling game, context may play a crucial role in at least two aspects: (a) it offers additional information that can be exploited by the sender and the receiver, and (b) context directly constrains the receiver’s action choice, which may impact the agents’ communicative strategy regardless of the receiver’s information-processing capabilities. We can illustrate this distinction by building upon an example given by Searle, (1980): when someone hears the utterance “we need to cut it” while standing on a lawn, they are likely to think about a mower. The same sentence uttered during a birthday party can be interpreted as a request to fetch a knife to cut a cake. However, when the sender knows that the right cutting tool, a mower or a knife, is already present at the receiver’s hand, they can be sure that the receiver will understand the utterance correctly even without knowing the context, as the possible course of action is already narrowed down.

While previous research on context-dependent communication has usually assumed agents’ nontrivial internal information-processing capabilities, we focus on the enabling constraints external to the agents and carry out experiments with the receiver’s minimal skills. We observe that the following conditions are sufficient for the development of context-dependent communication in this setting: (a) information on the receiver’s context is provided to the sender and (b) pressure is placed on the agents to minimise their vocabulary. Under these conditions, agents develop a language that is partly ambiguous and rely on the context for disambiguation rather than invent a fully precise language.

Linguists’ focus on context-independent language is reflected and arguably sustained by the commonly adopted metaphors of communication as a transmission problem and the resulting treatment of language as a code (Reddy,, 1979). The accompanying pervasive assumption of stable correspondence between language utterances and their meaning has roots in the long history of focus on language generativity (Katz and Fodor,, 1963). More pragmatically and ecologically-oriented works argue for a different picture, where natural language utterances are commonly ambiguous out of context – but they are successfully understood by humans based on the shared situational context (Sperber and Wilson,, 1986). As Barwise and Perry, (1983) indicated, while it is important that we can express an infinite number of propositions with a finite number of elements, due to the possibility of combination and recursion, it is equally amazing that we can express an infinite number of meanings with one expression due to its dependence on context (bringing to mind examples of entirely context-dependent expressions such as this).

Arguably, the conventional meaning of expressions can be extended to novel contexts due to their ambiguity (Brochhagen,, 2015). Recent works propose that expression meaning is negotiated in interaction (Chater and Christiansen,, 2022) and that language, rather than transmitting clear and bounded meanings, has the role of steering or controlling interactive events (Rączaszek-Leonardi,, 2012). These directions in the development of the theory of language, from the transmission model to viewing language as a prevalently ambiguous out of context, creative interaction control, situated in a particular environment, make understanding of the nature of context and its contribution to linguistic meaning and structure an urgent task ¹¹1Paradoxically, from such a standpoint it seems more challenging to account for the partial meaning context-independence than to acknowledge the effects of situational context; for example, how it is possible that we have a sense that utterances refer to something seemingly unrelated to the current circumstances, possibly distant in time and space (Rączaszek-Leonardi et al.,, 2018).. In this work, we investigate the role of the environment and, consequently, focus on the extra-linguistic context, understood as the current or past composition of the environment.

While the perspective adopted in this study foregrounds the positive and neutral aspects of linguistic ambiguity, we note that previous studies often associated ambiguity with miscommunication. As we see no compelling argument to make a clear distinction between these two cases, we consistently use the term ambiguity to refer to all cases of meaning uncertainty.

Piantadosi et al., (2012) propose that ambiguity arises under pressure for communicative efficiency. They consider the scenario of a sender and receiver in a shared context. If the context is at least slightly informative about the possible meanings of the sender’s message, the sender does not need to convey full information for the receiver to disambiguate the message, that is, infer its meaning from the context. Consequently, if symbol production is costly and the receiver’s inference is relatively cheap, it is suboptimal on the sender’s part to ensure that their utterance can only be interpreted in exactly one way. A shorter, partially ambiguous utterance will be preferred and will still be informative enough for the receiver. Additionally, if there are differences in the production cost of certain symbols, the sender may choose more economic, but more ambiguous ones.

While the bottleneck indicated by Piantadosi and colleagues concerned language production, Kirby et al., (2015) argue for an informational bottleneck in cultural transmission. They demonstrate that natural limitations in the cognitive abilities of language learners impose pressure for simplicity on a language system. Thus, despite the pressure for informativeness ²²2We follow the use of terms in Winters et al., (2018); the original terms were compressibility and expressivity (Kirby et al.,, 2015)., arising from the use of language, languages will never become perfectly informative about the intended meaning. Rather, they will gravitate towards a point of optimal trade-off between the competing pressures, where a degree of context dependence is to be expected. Under such a trade-off, language ambiguities can make communication less effective but still functional in general (Santana,, 2014; O’Connor,, 2015).

Intuitively, if language meaning relies on the context, then the composition of contexts, in which the interlocutors find themselves, should also play a role for ambiguity. Imagine a simple interaction between a shop assistant and a customer asking for a specific product visible on the display. Here, successful communication requires discrimination of the intended referent from other possible referents (e.g. Olson, (1970)). Winters et al., (2018) demonstrated that in this kind of situated setting, more predictable contexts allow for minimising information transmission through linguistic means. They manipulated the sender’s access to context and contextual object distribution in artificial language experiments, where pairs of participants were tasked to communicate about the target referent. When the possible referents in the context were random, the emergent signalling system was highly context-independent. However, when objects within each context were always of the same colour and the sender could see the receiver’s context, the emergent languages became ambiguous, not encoding colour information, and thereby relying on the stable object distribution in the context.

In summary, several factors seem to crucially contribute to the emergence of ambiguous language. One of them is the situational context shared between interlocutors, which is a source of additional information independent of what is conveyed by language. The second is an additional pressure that makes the language not only informative but also reasonably simple. The last is the regularity of the environment, which determines the relevant distinctions to be made by language.

The models relevant for this work follow the general schema of the influential Lewis, (1969) signalling game, where the sender and receiver learn to communicate about objects in their environment through rewards for communication success. In the original formulation of the game, as well as in its many extensions, there is no shared context between the agents. Under such circumstances, the only protocols that guarantee communication success are those that unequivocally map objects to signals, and they predictably emerge in simulations of evolutionary dynamics with realistic perturbations (Skyrms,, 2010; Santana,, 2014).

In some more recent models of visually grounded language emergence, neural agents faced an image discrimination task with shared context that directly constrained the referent choice (e.g. Lazaridou et al., (2017, 2016, 2018); Havrylov and Titov, (2017); Bouchacourt and Baroni, (2018)). The size of the context ranged from just two objects (e.g. Lazaridou et al., (2016)) to $128$ (Havrylov and Titov,, 2017). In these works, due to their engineering motivation, the only optimisation goal for agent policies was communication success. The results of Guo et al., (2021) suggest that the constrained choice of referents can itself lead to the emergence of context-dependent languages. Nevertheless, context dependence was not controlled for nor reported in these works.

Some works in evolutionary game theory investigated the origins of language ambiguity in signalling games where additional costs were included in the reward function. This included an additional cost imposed on the sender for using a larger vocabulary (Santana,, 2014; Mühlenbernd,, 2021) and / or the cost related to disambiguation imposed on the receiver (Mühlenbernd,, 2021). The communicating agents were provided with shared contextual information. The results show that partial ambiguity is evolutionarily the most likely strategy when the sender’s signalling cost is higher or comparable to the receiver’s disambiguation cost. In a similar vein, Brochhagen, (2020) showed that the context can be exploited by the sender to use its preferred but more ambiguous forms. In these game-theoretic studies, the receiver had the ability to infer the intended meaning of the sender, in congruence with Piantadosi et al., (2012). However, the direct impact of the environment on context-dependent communication was not accounted for because of the unlimited referent choice.

While the receiver’s inference is undoubtedly important for context-dependent communication (Frank and Goodman,, 2012), direct environmental constraint on the choice of possible referents substantially modifies the task (Winters et al.,, 2018; Guo et al.,, 2021). In a situated communicative setting with a constrained referent choice, the speaker’s signal needs to be only as precise as to discriminate the referent from other present objects, and the receiver’s task is arguably less challenging, as the agent’s inference is partly “offloaded” to the environment. Therefore, to investigate the role of the environment for context dependence, we use a model in which agents have minimal skills and in which the pressure for vocabulary size and sender’s access to context will be manipulated.

Importantly, we use an entirely random distribution for objects in the context. This is for two reasons. Firstly, it is to be expected that regularities in the encountered contexts can yield more ambiguous but still effective signals (Guo et al.,, 2021) but the possible outcomes for an entirely random object distribution under vocabulary size pressure are less clear. Secondly, we aimed for the most general conditions for the ambiguous communication to emerge and thus made as few assumptions as possible.

Let us define a contextual variant of the probabilistic Lewis signalling model. There are two agents: sender and receiver. The sender perceives one of the alternative states of the world $s\in S$, and emits one of the available signals $\sigma\in\Sigma$ with probability $f(\sigma|s)$. The receiver chooses a response $r\in R$, based on the received signal, with probability $g(r|\sigma)$. Upon receiving a correct response $r$ for a given state $s$ both agents are rewarded. Agent policies are represented through functions $f$ and $g$ and can be modelled, for example, using neural networks. Through adaptive learning, agents can reach an equilibrium in which signals are used for functional communication. Signals are conventional in the sense that $s$–$\sigma$ and $\sigma$–$r$ mappings are arbitrary but must be consistent. We can say that the signalling protocol established in a Lewis signalling game is unambiguous if there is a unique mapping from signals $\sigma$ to states $s$. Such a setting and its many varieties have been a kind of general testing ground for investigating various aspects of language emergence.

Let us call a non-contextual variant of this game where the receiver’s output belongs to the same space as the sender’s input (namely, the response belongs to the set of possible world states, $r\in S$), a reconstruction game, following recent usage of the term by Guo et al., (2021). In the reconstruction game, the receiver has to learn to reconstruct the target solely from the signal emitted by the sender, without any contextual cues (e.g. Chaabouni et al., (2020); Kharitonov and Baroni, (2020)).

In contrast, in another popular scenario, which we call discrimination game ³³3Guo et al., (2021) have called this scenario simply referential game. However, this name has been used across a wide variety of contexts and fields. To avoid confusion and distinguish this setting from reconstruction game more accurately, we choose the term discrimination. the receiver’s task is to identify the target from among the provided set of objects which constitutes the context. We will use the word context in this narrow technical sense. The world state is multi-object, that is, if $O$ is the set of all possible objects, the world state consists of the context $C\subset O$ and the target $t\in C$ – what is communicated is only a part of the world state.

Within a discrimination game, we can consider a receiver that is not “aware” of the context, but is still constrained by it, since only some objects can be selected. For such a case, we need to define an auxiliary selection function:

$$\iota_{C}(\mathbf{z})_{i}=\begin{cases}\dfrac{e^{z_{i}}}{\sum_{r_{j}\in C}e^{z_{j}}}&\text{if }r_{i}\in C\\ 0&\text{if }r_{i}\not\in C\end{cases}$$

The function $\iota_{C}$ operates on a vector of probabilities $\mathbf{z}$ corresponding to different responses $r\in O$. It selects only valid responses that belong to a given context and normalizes the probabilities.

On a spectrum of perceptual availability of the context, we can distinguish the following variants of the sender–receiver pairs playing a discrimination game, from the least to the most contextual:

1. 1.

   Neither sender nor receiver have access to the context. Sender’s signal is generated with probability $f(\sigma|t)$, receiver’s response is generated with probability $\iota_{C}(g(r|\sigma))$.

2. 2.

   Sender has access to the context, but the receiver does not. Sender’s signal is generated with probability $f(\sigma|t,C)$, receiver’s response is generated with probability $\iota_{C}(g(r|\sigma))$.

3. 3.

   Both sender and receiver have access to the context. Sender’s signal is generated with probability $f(\sigma|t,C)$, receiver’s response is generated with probability $g(r|\sigma,C)$.

Under variant 1) agents cannot use contextual information at all. The optimal solution is to use unambiguous signalling, where every world state is represented by a unique signal. Under variant 3) sender and receiver share context, and may use it as a common ground. This, in principle, opens a possibility for ambiguous, yet efficient, communication protocols, such that the sender emits the signal conditional on the context and the receiver disambiguates the signal in a process of inference within the particular context (Piantadosi et al.,, 2012).

Variant 2) is an interesting middle ground between these two possibilities. Since sender knows which objects are available for the receiver to choose from, it may safely use an ambiguous signal when other probable referents of the signal are not present in the current context. Signalling protocol that is highly ambiguous in the entire object space may still be optimal for the vast majority of contexts. Thus, accurate context-dependent communication may emerge even without inferential capabilities on the receiver’s side. This is the variant that we implement in our simulations.

Agents are modelled as multi-layer neural networks. We consider two agent pairs, corresponding to variant 1 and 2 from section 4, including the same type of receiver but differing with respect to the sender’s access to the context beyond the target object. Importantly, the receiver is designed to be unable to infer signal meaning in context.

Target-only (T) sender. A simple feed-forward network of fully-connected layers, which receives only the target as the input and maps it to probabilities of activation of each symbol (Fig. 1). Since this sender cannot access the context, it shares no common information with the receiver, and thus it is not expected to be able to communicate in a context-dependent manner in any setting.

Target-in-context (TC) sender . The input of this network is all objects in the context, including the target. It receives input sequentially (Fig. 2). As non-target objects are input to the network in subsequent steps, the target is passed along each of them. Such an input structure enables direct comparisons between the target and each of the objects. This network has an LSTM memory cell, which constitutes the first layer of the network. After the last input is provided, LSTM output is passed through fully-connected layers to produce activation probabilities for each symbol. Since this sender architecture is informed by the receiver’s context, it is expected to be able to communicate in a context-dependent manner.

Receiver The receiver is a simple network that generates the decision on the choice of the object within a single forward pass (Fig. 3). It is inspired by and very similar to the receiver architecture used by Lazaridou et al., (2017) in that each object may only be compared with the received signal independently from other objects. Each object is processed by the same sequence of network layers along with the signal emitted by the sender, which produces what may be interpreted as an alignment score between the input object and the received signal. All such scores are combined with a softmax operation into a probability distribution over objects, from which the final selection is sampled. In simple terms, this receiver most often selects the object that best matches the received signal.

We ran our model in the world where all objects have $2$ categorical properties, both with $3$ possible values. For illustration purposes, we can consider them as shape and colour. The context size was set to $3$. The context was entirely random, that is, each object appeared in the context with equal probability and, consequently, so did every possible object combination. The target, too, was selected from the context at random.

We tested our models in two main conditions: with unlimited vocabulary and with a vocabulary size penalty. The experiments were carried out with both T and TC senders (see section 5), paired with the same receiver type. We ran each simulation $10$ times for $50k$ iterations. The exact model hyperparameters used in both experiments and final metric values are specified in Appendix A.

We investigated a situated communicative scenario in which the environment directly constrains the receiver’s response to the objects in the current context and thus constrains its interpretation of the signal. In this setting, we examined the consequences of pressure to minimise vocabulary. Our results are consistent with previous modelling work in that the penalty on vocabulary size induced a degree of ambiguity (Santana,, 2014; O’Connor,, 2015; Mühlenbernd,, 2021). Furthermore, they show the ambiguity that arises as a trade-off between accuracy and minimisation of vocabulary size, in accordance with O’Connor, (2015) predictions that a degree of inaccuracy is optimal for a certain ratio between the cost of vocabulary size and the size of the meaning space to be expressed. In Experiment 2, with a large enough vocabulary size penalty, the accuracy dropped below the maximum value for both sender types.

In our model, unlike in the above works and others (Piantadosi et al.,, 2012), effective disambiguation of signal meaning depends on the environmental constraints on the referent choice, and not on the receiver’s inferential capabilities. The effectiveness of disambiguation can be seen in that the accuracy reached levels higher than our out-of-context baseline – target certainty – for both types of senders. While the target-in-context sender (TC) did not outperform the target-only sender (T) in terms of raw accuracy, it developed a more ambiguous language with a smaller vocabulary that was still reasonably effective. This was the case even though the receiver was always context-agnostic. The context-aware sender was able to capitalise on the situated context constraining receiver’s actions. This kind of context-dependent signalling constitutes a variant of audience design that does not rely on simulating inferential processes at the receiver’s end, as in some previous works (Batali,, 1998; Brochhagen,, 2020). The need for context awareness on the sender’s side is consistent with empirical experiments by Winters et al., (2018), where the most sophisticated languages that encoded only the contextually relevant semantic distinctions emerged only when the sender was able to perceive the context.

Disambiguation mechanisms similar to those present in our model arguably play a major role in shaping languages; natural languages evolve under pressures that favour ambiguity, such as minimising speaker effort (Piantadosi et al.,, 2012) or language learnability (Kirby et al.,, 2015), and these mechanisms may allow ambiguity to enter the language without sacrificing communication success. Although constrained referent choice in the absence of the sender’s context awareness does not offer a systematic disambiguation mechanism, it may still greatly reduce communication errors and thus pressure for meaning precision. As the two meanings of the word bat – the baseball bat and the flying mammal – are unlikely to be relevant in the same context, the word itself may safely remain ambiguous. When the sender becomes aware of the context, different or additional linguistic means can be used to specify the meaning. This was illustrated by the context-aware sender’s behaviour in our model and reflected in an experiment by Ferreira et al., (2005), where participants were tasked to describe a scene. Perceiving both the flying mammal and a baseball bat in one scene, the participants often used an additional specifier for one of the objects, producing descriptions such as “bat and baseball bat”. Clearly, complex human inferential capabilities that involve both sender and receiver, as considered in pragmatics (e.g. Korta and Perry, (2020)), will likely make the ambiguous language even more effective.

While context-dependent communication has usually been portrayed as relying on processes internal to the communicating agents, we demonstrate that a nontrivial part of the job of disambiguation is performed by the environment itself (Brooks,, 1990; Wilson and Golonka,, 2013). Consequently, it seems that the composition of contexts in which we find ourselves plays a key role. Guo et al., (2021) show in a computational model that languages developed by neural agents in signalling games with medium context size encode more information than those emerging within a smaller or larger context. We can speculate that another source of environmentally induced ambiguity may be simply the frequency of referents. Huttegger, (2007) shows that for an uneven distribution of referent types, the population is more likely to stay in a suboptimal ambiguous signalling strategy. As some types need to be communicated less frequently, the potential payoff of assigning them a unique symbol is reduced. In our simulations, we used a very simple environment with uniform distribution over the communicated targets and uncorrelated object occurrence within the context. More complex environments with predictable regularities open new possibilities. If two particular objects never co-occur (the baseball bat and the flying mammal), the same signal can be potentially used for both of them without causing communication errors. Winters et al., (2018) show that predictable composition of the context is, in fact, exploited by human participants who, consequently, arrive at simpler languages than those emerging for entirely random contexts.

There may be more mechanisms responsible for handling ambiguity in language than is often assumed. The phenomena discussed so far concerned one-directional communication. In contrast, human interlocutors engage in dialogues where they explicitly ask for disambiguation (Healey et al.,, 2018) or even co-construct meaning together (Kempson et al.,, 2019). Looking from this angle, we discover that not only sharing context, but also aggregating information from individually available contexts is important for dialogically co-constructed messages (Rączaszek-Leonardi et al.,, 2014). These multiple mechanisms can remedy meaning uncertainty when precision is vital; the very same uncertainty can allow for language flexibility, which is indispensable on other occasions (see Section 2).

There are a couple of directions where the current work can be developed. First, future research could examine the effects of various context properties, such as the distribution of objects in the environment. In our work, we used only a uniform distribution; moving away from it would affect relative benefits of distinguishing certain object properties and also introduce regularities that could be exploited by agents in their disambiguation strategies. Second, the analysis could be extended to answer questions concerning emerging language structure, an issue that we have left aside in our work. As in natural languages a neighbouring word often sufficiently disambiguates an otherwise ambiguous expression (Ferreira,, 2008), we could expect that complexifying signals would render their individual components even more ambiguous. One possible amendment to the model would be allowing agents to emit compound signals of variable length. Finally, the current model could be altered to allow bi-directional exchange between agents (in the vein of Kottur et al., (2017); Bancerek and Zubek, (2023)). This would allow us to investigate dialogical mechanisms of disambiguation and their interactions with situational context in constructing flexible language.

In this work, we developed a model in which the pressure that is conducive to the emergence of context-dependent communication was combined with the environmentally constrained interpretation of the signal by the receiver. In doing this, we complemented previous modelling work, which focused on how communicating agents can exploit shared information by means of inference. To the best of our knowledge, this is also the first work where environmentally constrained communication was modelled in conjunction with a vocabulary size cost. Our results point to the fact that context-dependent communication is likely a multilayered phenomenon, which should be analysed not only in terms of individual inferential capabilities, but also in terms of the agents-environment and agent-agent relations. In line with other authors of recent research on linguistic context dependence, we urge that out-of-context signal ambiguity should not be readily equated with proneness to error but rather understood as a pervasive property of real-life situated communication, resulting in communicative efficiency and flexibility.

This work was funded by the National Science Centre, Poland (OPUS 15 grant, 2018/29/B/HS1/00884). The experiments were partly run on the supercomputers of the Interdisciplinary Centre for Mathematical and Computational Modelling at the University of Warsaw (computational grant G86-1039). The metrics in the experimental results were tracked via Weights & Biases (Biewald,, 2020).

The authors declare that they have no competing interests.