Let’s Negotiate! A Survey of Negotiation Dialogue Systems

Haolan Zhan^$\heartsuit$, Yufei Wang^$\heartsuit$, Tao Feng^$\heartsuit$, Yuncheng Hua^$\heartsuit$, Suraj Sharma^{$\diamondsuit$}, Zhuang Li^$\heartsuit$,
Lizhen Qu^$\heartsuit$ Gholamreza Haffari^$\heartsuit$
^$\heartsuit$ Department of Data Science & AI, Monash University, Australia
^{$\diamondsuit$} California State University, Northridge, CA
{firstname.lastname}@monash.edu, [email protected]

Abstract

Negotiation is one of the crucial abilities in human communication, and there has been a resurgent research interest in negotiation dialogue systems recently, which goal is to empower intelligent agents with such ability that can efficiently help humans resolve conflicts or reach beneficial agreements. Although there have been many explorations in negotiation dialogue systems, a systematic review of this task has to date remained notably absent. To this end, we aim to fill this gap by reviewing contemporary studies in the emerging field of negotiation dialogue systems, covering benchmarks, evaluations, and methodologies. Furthermore, we also discuss potential future directions, including multi-modal, multi-party, and cross-cultural negotiation scenarios. Our goal is to provide the community with a systematic overview of negotiation dialogue systems and to inspire future research.

1 Introduction

“Let us never negotiate out of fear. But let us never fear to negotiate.”

John F. Kennedy

Negotiation is one of the crucial abilities in human communication that involves two or more individuals discussing goals and tactics to resolve conflicts, achieve mutual benefit, or find mutually acceptable solutions Fershtman (1990); Bazerman and Neale (1993); Lewicki et al. (2011). It is a common aspect of human interaction, occurring whenever people communicate in order to manage conflict or reach a compromise. Scientifically, one of the long-term goals of dialogue research is to empower intelligent agents with such ability. Agent effectively negotiating with a human in natural language could have significant benefits in many scenarios, from bargaining prices in everyday trade-in He et al. (2018) to high-stakes political or legal situations Basave and He (2016).

Refer to caption — Figure 1: The negotiation process involves a multi-turn interaction between agent and human. They exchange information about their deals and end up with accepting or declining deals.

Negotiation dialogue systems Lewandowska (1982); Lambert and Carberry (1992); Chawla et al. (2021c) is an emerging research field that aims to build intelligent conversational agents that can automatically negotiate with a human in natural languages, e.g., CICERO¹¹1https://ai.facebook.com/research/cicero/ from Meta AI. Agents negotiate with human through multi-turn interaction using logically reasoning Sycara and Dai (2010) over goals Zhang et al. (2020), strategies Zhou et al. (2020) and psychology factors Yang et al. (2021). As illustrated in Figure 1, negotiation dialogue agents interact with the human through multi-turn cycles. A successful negotiation process involves efficient information exchange, strategic discussion toward their goals, and a closing section.

Despite the significant amount of research that has been conducted on the task, there is a lack of a systematic review of the topic. In this work, we aim to fill this gap by reviewing contemporary work in the emerging field of negotiation dialogue systems, covering aspects such as benchmarks, evaluation, methodology, and future directions. In recent years, various benchmarks have been proposed for negotiation dialogue systems, ranging from bargaining Lewis et al. (2017) and game scenarios Asher et al. (2016) to job interviews Zhou et al. (2019) and items exchanging Chawla et al. (2021c). Our survey will provide an overview of these benchmarks and discuss how they have been used to evaluate the performance of negotiation dialogue systems.

Modeling the negotiation process for conversational agents also imposes challenges. Firstly, these agents must be able to reason about and employ various strategies in different situations. In addition to strategy modeling, it is also necessary to model the personalities (e.g., mind, emotion, and behaviors) of the negotiators. Thirdly, an effective policy learning method is essential for the successful use of language. To address these challenges, we can categorize existing solutions into three areas: (1) Personality modeling helps us understand negotiator’s preferences, (2) Strategy modeling enables agents to make reasonable decisions based on gathered information, and (3) Policy learning methods utilize information effectively to maximize results.

In summary, our contributions are three-fold: (1) To the best of our knowledge, we systematically categorize current negotiation dialogue benchmarks from the perspective of distributive and integrative, with each category based on different goal types of negotiation dialogue tasks. (2) We categorize typical evaluation methods and current solutions into an appropriate taxonomy. (3) We pointed out the current limitation and promising research directions in the future.

2 Backgrounds

2.1 Negotiation in Human

Humans negotiate everyday in their daily routines. Negotiation is used to manage conflict and is the primary give-and-take process by which people try to reach an agreement Fisher et al. (2011); Lewicki et al. (2011). Research on negotiation has been conducted for almost 60 years in the field of psychology, political science, and communication. It has evolved over the past decades from exploring game theory Walton and McKersie (1991), behavior decisions driven by the cognitive revolution in psychology Bazerman and Neale (1993), to cultural differences in the 2000s Bazerman et al. (2000). Negotiation research, however, is now forced to confront the implications of human/AI collaborations given recent advancements in machine learning Sycara and Dai (2010). Converging efforts from social scientists and data scientists who incorporate insights from both fields will be fruitful in maximizing expectations and outcomes during negotiation processes.

Negotiation is a process by which two or more parties attempt to resolve their opposing interests. Strategy of negotiation can be distributive, such as bargaining Fershtman (1990) and integrative, such as maximizing unilateral interests Bazerman and Neale (1993), both of which are used in various social situations such as informal, peer to peer, organizational, and diplomatic country to country settings. The implications for enhancing outcomes are thus large and important to understand. Research from psychology demonstrates that the negotiation process can be affected by psychological factors, such as personality Sharma et al. (2013), relationship Olekalns and Smith (2003), social status Blader and Chen (2012), and cultural background Leung and Cohen (2011).

2.2 Task Definition

The ability of an agent to maintain good communication skills as well as strategic reasoning capabilities is what makes a negotiation dialogue system unique from typical task-oriented and open-domain systems. Negotiation dialogue aims to interact with opponents in a strategic discussion to find acceptable solutions for both parties. A negotiation process involves a mixture of strategies such as debate, persuasion, adversary, and compromise. These strategies will change along with the negotiation flow and can be influenced by opponents’ personalities (e.g., thoughts, emotions, and behaviors). Therefore, strategy and personality are two main aspects to be modeled in negotiation dialogue systems. Formally, a negotiation dialogue task is defined as a tuple $(\mathcal{K},\mathcal{S},\mathcal{U},\pi,{g})$ . $\mathcal{K}$ refers to the pre-defined information which is prepared for the negotiation process, such as the negotiator’s preferences and demands. $\mathcal{S}$ refers to a trajectory $\{s_{1},s_{2},...\}$ , which is used to model the strategy transition process and provided for policy learning module (e.g., reinforcement learning). A series turn of dialogue interactions $\mathcal{U}$ can be viewed as $\{u_{1},u_{2},...\}$ are generated along with the negotiation process. A policy learning module $\pi_{\theta}(\mathcal{K},\mathcal{S},\mathcal{U})$ is utilized to learn an optimal deterministic policy which helps reach the negotiation goal $g$ .

3 Negotiation Datasets

DataSet	Negotiation Type	Scenario	# Dialogue	# Avg. Turns	# Party
InitiativeTaking (2014)	Integrative	Fruit Assignment	41	-	Multi
STAC (2016)	Integrative	Strategy Games	1081	8.5	Two
DealorNoDeal (2017)	Integrative	Item Assignment	5808	6.6	Two
Craigslist (2018)	Distributive	Price Bargain	6682	9.2	Two
NegoCoach (2019)	Distributive	Price Bargain	300	-	Two
PersuasionforGood (2019)	Distributive	Donation	1017	10.43	Two
FaceAct (2020)	Distributive	Donation	299	35.8	Two
AntiScam (2020b)	Distributive	Privacy Protection	220	12.45	Two
CaSiNo (2021c)	Integrative	Item Assignment	1030	11.6	Two
JobInterview (2021a)	Integrative	Job Interview	2639	12.7	Two
DinG (2022)	Integrative	Strategy Games	10	2357.5	Multi

Table 1: Negotiation dialogues benchmarks are sorted by their publication time. For each dataset, we introduce the negotiation type, scenario, the number of dialogues and corresponding average turns, and party attributes.

In this section, we summarize the existing negotiation datasets and resources. Table 1 shows all of the collected benchmarks, along with their negotiation types, scenarios and data scale. In this paper, we categorize these benchmarks based on their negotiation types, namely, integrative negotiation and distributive negotiation. The integrative negotiation is associated with win-win scenarios, and its goal is to develop mutual gain. On the contrary, the distributive negotiation is often associated with win-lose scenarios and aims to maximize personal benefits. In general, The distributive negotiation is more competitive than its integrative counterpart.

3.1 Integrative Negotiation Datasets

In integrative negotiations, there is normally more than one issue available to be negotiated. To achieve optimal negotiation goals, the involved players should make trade-offs for multiple issues.

Multi-player Strategy Games

The strategy video games provide ideal platforms for people to verbally communicate with other players to accomplish their missions and goals. Asher et al. (2016) propose the STAC benchmark, which is the player dialogue in the game of Catan. In this game, players need to gather resources, including wood, wheat, sheep, and more, with each other to purchase settlements, roads and cities. As each player only has access to their own resources, they have to communicate with each other. To investigate the linguistic strategies used in this situation, STAC also includes an SDRT-styled discourse structure. Boritchev and Amblard (2022) also collect a DinG dataset from French-speaking players in this game. The participants are instructed to focus on the game, rather than talk about themselves. As a result, the collected dialogues can better reflect the negotiation strategy used in the game process.

Negotiation for Item Assignment

The item assignment scenarios involve a fixed set of items as well as a predefined priority for each player in the dialogue. As the players only have access to their own priority, they need to negotiate with each other to exchange the items they prefer. Nouri and Traum (2014) propose InitiativeTalking, occurring between the owners of two restaurants. They discuss how to distribute the fruits (i.e., apples, bananas, and strawberries) and try to reach an agreement. Lewis et al. (2017) propose DealorNoDeal, a similar two-party negotiation dialogue benchmark where both participants are only shown their own sets of items with a value for each and both of them are asked to maximize their total score after negotiation. Chawla et al. (2021c) propose CaSiNo, a dataset on campsite scenarios involving campsite neighbors negotiating for additional food, water, and firewood packages. Both parties have different priorities over different items.

Negotiation for Job Interview

Another commonly encountered negotiation scenario is job offer negotiation with recruiters. Yamaguchi et al. (2021a) fill this gap and propose the JobInterview dataset. JobInterview includes recruiter-applicant interactions over salary, day off, position, company, and workplace. The participants are shown negotiators’ preferences and the corresponding issues and options and are given feedback in the middle of the negotiation.

3.2 Distributive Negotiation Datasets

Distributive negotiation is about the discussion over a fixed amount of value (i.e., slicing up the pie). In such negotiation, the involved people normally talk about a single issue (e.g., item price) and therefore, there are hardly trade-offs between multiple issues in such negotiation.

Persuasion For Donation

Persuasion, convincing others to take specific actions, is a necessary required skill for negotiation dialogue Sycara (1990); Sierra et al. (1997). Wang et al. (2019) focus on persuasion and propose PersuasionforGood, a two-party persuasion conversations about charity donations. In the data annotation process, the persuaders are provided some persuasion tips and example sentences, while the persuaders are only told that this conversation is about charity. The annotators are required to complete at least ten utterances in a dialogue and are encouraged to reach an agreement at the end of the conversations. Dutt et al. (2020) further extend PersuasionforGood by adding the utterance-level annotations that change the positive and/or the negative face of the participants in a conversation. A face act can either raise or attack the positive face or negative face of either the speaker or the listener in the conversation.

Negotiation For Product Price

Negotiations over product prices can be observed on a daily basis. He et al. (2018) propose CraigslistBargain, a negotiation benchmark based on a realistic item price bargaining scenario. In CraigslistBargain, two agents, a buyer and a seller, are required to negotiate the price of a given item. The listing price is available to both sides, but the buyer has a private price as the target. Then two agents chat freely to decide the final price. The conversation is completed when both agents agree with the price or one of the agents quits. Zhou et al. (2019) propose NegoCoach benchmark on similar scenarios, but with an additional negotiation coach who monitors messages between the two annotators and recommends tactics in real-time to the seller to get a better deal.

User Privacy Protection

Privacy protection of negotiators has become more and more vital. Participant (e.g., attackers and defenders) goals are also conflicting. Li et al. (2020b) propose Anti-Scam benchmark which focuses on online customer service. In Anti-Scam, users try to defend themselves by identifying whether their components are attackers who try to steal sensitive personal information. Anti-Scam provides an opportunity to study human elicitation strategies in this scenario.

4 Evaluation

We categorize the evaluation methods for negotiation dialogue systems into three types: goal-oriented metrics, game-based metrics and human evaluation. Table 2 lists the evaluation metrics that are introduced in our survey.

4.1 Goal-based Metrics

Goal-oriented metrics mainly consider the agent’s proximity to the goal from the perspective of strategy modeling, task fulfillment and sentence realization. Success Rate (SR) is the most widely used, which measures how frequently an agent completes the task within their goals. A similar metric Prediction Accuracy (PA) is to evaluate the agent’s strategy predictions or the outcome of negotiations, such as macro or average F1 score Wang et al. (2019); Dutt et al. (2020); Chawla et al. (2021c). For those scenario-related tasks, Yamaguchi et al. (2021a) present a task where the model is required to label the human-human negotiation outcomes as either a success or a breakdown, including area under the curve (ROC-AUC), confusion matrix (CM), and average precision (AP). Kornilova et al. (2022) propose a model-based evaluation based on Item Response Theory to analyze the effectiveness of persuasion on the audience.

In terms of language realization for negotiation dialogue, Hiraoka et al. (2015) employ a predefined naturalness metric (a bigram overlap between the system responses and the ground-truth responses) as part of the reward to evaluate policies in cooperative persuasive dialogues. Other classical metrics for evaluating the quality of response are also used, i.e., perplexity (PPL), BLEU-2, ROUGE-L, and BOW Embedding-based Extrema matching score Lewis et al. (2017).

Goal-based Metrics	SR, PA (2014; 2019; 2020; 2022); Average F1 score (2021c); Macro F1 score (2019; 2020); ROC-AUC, CM, AP (2021a); Naturalness (2015); PPL, BLEU-2, ROUGE-L, Extrema (2017)
Game-based Metrics	WinRate, AvgVPs (2017); Utility, Fairness, Length (2018); Average Sale-to-list Ratio, Task Completion Rate (2019)
Human Evaluation	Agent satisfaction (2015; 2017); Purchase decision, Correct response rate (2015) Achieved agreement rate, Pareto optimality rate (2017); Likert score (2018)

Table 2: Various Metrics used in the existing negotiation dialogues benchmarks.

4.2 Game-based Metrics

Different from goal-oriented metrics, which focus on evaluating the accuracy of strategies or actions, game-based evaluation provides a user-centric perspective through multi-turn interactions. Keizer et al. (2017) measure the bots’ negotiation strategies within the online game "Settlers of Catan" by proposing the metrics WinRate (the percentage of games won by the humans when playing with the bot opponents) and AvgVPs (the average number of victory points gained by the human players). He et al. (2018) present a task that two agents bargain to get the best deal using natural language. They use task-specific scores to test the performance of the agents, including utility (a score that is higher when the final price is closer to one agent’s expected price), fairness (the difference between two agents’ utilities), and length (the number of the sentences exchanged between the two agents). Zhou et al. (2019) design a task where a seller and a buyer try to achieve a mutually acceptable price through a natural language negotiation. They adopt different metrics to evaluate the dialogue agent, i.e., average sale-to-list ratio and the task completion rate. Besides, Cheng et al. (2019) propose an adversarial attacking evaluation approach to test the robustness of negotiation systems.

4.3 Human Evaluation

To evaluate the users’ satisfaction with the dialogue systems, human judgment is employed as a subjective evaluation of the generated output. Hiraoka et al. (2015) use a dialogue system as the salesperson to bargain with the human customers and have the users annotate subjective customer satisfaction (a five-level score), the final decision of making a purchase (a binary number indicating whether persuasion is successful), and the correct response rate in the dialogues. Lewis et al. (2017) employ crowd-sourcing workers to highlight that essential information when bargaining with dialogue systems, covering the percentage of dialogues where both interlocutors finally achieve an agreement, and Pareto optimality, i.e., the percentage of the Pareto optimal solutions in all the agreed deals. He et al. (2018) propose human likeness as a metric in evaluating how well the dialogue system is doing in a bargain. They ask workers to manually score the dialogue agent using a Likert metric to judge whether the agent acts like a real human or not.

5 Methodology Overviews

As shown in Figure 2, we categorize existing methods into Strategy Modeling, Personality Modeling, and Policy Learning. Strategy modeling methods help conversational agents utilize appropriate strategies in different situations. In addition, negotiation is not only about complex reasoning over strategies, but also influenced by psychology factors. Personality is one of the most important psychology factors that would affect agents’ decisions. Therefore, personality modeling helps agents perceive opponents. Besides, an effective policy learning method is indispensable for the successful use of language.

5.1 Strategy Modeling

In this section, we discuss strategy modeling for negotiation dialogue systems. Negotiation strategies include a range of tactics and approaches that people use to achieve their goals in negotiation processes. It can be categorized into three aspects: integrative (win-win) and distributive (win-lost), and multi-party.

5.1.1 Integrative Strategy

Integrative strategy (known as win-win) modeling aims to achieve mutual gain among participants. For instance, Zhao et al. (2019) proposes to model the discourse-level strategy using a latent action reinforcement learning (LaRL) framework. LaRL can model strategy transition within a latent space. However, due to the lack of explicit strategy labels, LaRL can only analysis strategies in implicit space. To resolve the problem, Chawla et al. (2021c) define a series of explicit strategies such as Elicit-Preference, Coordination and Empathy. While Elicit-Preference is a strategy attempting to discover the preference of the opponent, Coordination promotes mutual benefits by explicit offer or implicit suggestion. In order to capture user’s preference, Chawla et al. (2022) utilize those strategies using a hierarchical neural model. Besides, Yamaguchi et al. (2021b) present another collaborative strategy set to negotiate workload and salaries during the interview, which goal is to reach an agreement between employer and employee. It assists humans in becoming better negotiators during this process, e.g., communicating politely, addressing concerns, and providing side offers.

5.1.2 Distributive Strategy

Distributive strategy (known as win-loss) modeling focuses on achieving one’s own goals and maximizing unilateral interests more than mutual benefits. Distributive strategy can be used when you insist on your position or resist the opponent’s deal Zhou et al. (2019). For example, Dutt et al. (2021a) investigate four resisting categories, namely contesting, empowerment, biased processing, and avoidance Fransen et al. (2015). Each individual category contains fine-grain strategic behaviors. For example, contesting refers to attacking the message source, and empowerment implies reinforcing personal preference to contradict a claim (Attitude Bolstering) or attempting to arouse guilt in the opponent (Self Pity). Besides, Wang et al. (2019) design a set of persuasion strategies to persuade others to donate to charity. It contains 10 different strategies containing logical appeal, emotional appeal, source-related inquiry and etc. Li et al. (2020a) explore the role structure to enhance the strategy modeling. Dutt et al. (2020) further enhances the role modeling with facing act, which helps utilize strategy between asymmetric roles.

5.1.3 Multi-party Strategy

While previously mentioned work on integrative and distributive strategy modeling mainly relates to two-party, multi-party strategy modeling is slightly different. In multi-party situations, strategy modeling needs to consider different attitudes and complex relationships among individual participants, whole groups, and subgroups Traum et al. (2008). Georgila et al. (2014) attempt to model multi-party negotiation using a multi-agent RL framework. Furthermore, Shi and Huang (2019) propose to construct a discourse dependency tree to predict relation dependency among multi-parties. Besides, Li et al. (2021) disclose relations between multi-parties using a graph neural network. However, due to the limited access to multi-party datasets, strategy modeling on multi-party scenarios is under-explored.

5.2 Personality Modeling

Negotiation dialogue involves complex social interactions related to multiple disciplines, such as psychology, for understanding human decision-making. Personality is an important factor in the understanding human-decision process. We summarize those work modeling personality from three perspectives: Mind, Emotion, and Behavior modeling.

5.2.1 Mind Modeling

Mind modeling in negotiation dialogue systems encompasses several tasks, such as mind preference estimation and opponent response prediction. Mind preference estimation helps the agent infer the intention of the opponents and guess how their own utterances would affect the opponent’s mental preference. Nazari et al. (2015) propose a heuristic frequency-based method to estimate the negotiator’s preference. Langlet and Clavel (2018) consider a rule-based system incorporating linguistic features to identify user’s preference. A critical challenge for mind modeling in negotiation is that it usually requires complete dialogues, so it is difficult to predict those preferences precisely for partial dialogue. To make it applicable for those partial dialogues, which is widespread in real-world applications, Chawla et al. (2022) formulated mind preference estimation as a ranking task and proposed a transformer-based model that can be trained directly on partial dialogue.

In terms of opponent response prediction, He et al. (2018) firstly propose to decouple the modeling of the strategy of generation containing a parser to map utterances with dialogue acts and a dialogue manager to predict the skeleton of dialogue acts. Yang et al. (2021) further improve the negotiation system with a first-order model based on the theory of Mind Frith and Frith (2005) , which allows the agents to compute an expected value for each mental state. They provided two variance variants of ToM-based dialogue agents: explicit and implicit, which can fit both pipeline and end-to-end systems.

5.2.2 Emotion Modeling

Emotion modeling refers to recognizing the emotional change between negotiators. Therefore, explicit modeling of emotions throughout a conversation is crucial to capture reflections from opponents. To study emotional feelings and expressions in negotiation dialogue, Chawla et al. (2021a) explore the prediction of two important subjective goals, including outcome satisfaction and partner perception. Liu et al. (2021) provide explicit modeling on emotion transition engaged with pre-trained models (e.g., DialoGPT), to support help-seeker. Further, Dutt et al. (2020) propose a facing act modeling under the persuasive discussion scenarios. Mishra et al. (2022) utilize a reinforcement learning framework to engage emotion in the persuasive message.

5.2.3 Behavior Modeling

Behavior modeling refers to detecting and predicting opponents’ behaviors during the negotiation process. For example, fine-grained dialogue act labels are provided in the Craigslist dataset He et al. (2018), to help track the behaviors of buyers and sellers. Zhang et al. (2020) propose an opposite behavior modeling framework to estimate opposite action using DQN-based policy learning. Chawla et al. (2021b) explore early prediction between negotiators for the outcomes. Tran et al. (2022) leverage dialogue acts to identify optimal strategies to persuade humans for donation.

5.3 Policy Learning

Policy learning plays an important role in negotiation dialogue systems, by which the agent learns to choose a strategy and generate a response for the next step. Methods of policy learning can be roughly categorized into three types: reinforcement learning, supervised learning, and multitask learning.

5.3.1 Reinforcement Learning

Reinforcement learning (RL) is one of the most common frameworks chosen for policy learning. English and Heeman (2005) are the first to use RL techniques for negotiation dialogue systems. They employed a single-agent pattern to learn the policy of two opponents individually. But single-agent RL techniques are not well suited for concurrent learning where each agent is trained against a continuously changing environment. Therefore, Georgila et al. (2014) further advances the framework with concurrent progress using multi-agent RL techniques, which simultaneously model two parties and provide a way to deal with multi-issues scenarios. Besides, Keizer et al. (2017) propose to learn the action of the target agents with a Q-learning reward function. They further propose a method based on hand-crafted rules and a method using Random Forest trained on a large human negotiation corpus from Afantenos et al. (2012).

Most recent works try to equip RL with deep learning techniques. For instance, Zhang et al. (2020) propose OPPA, which lets the target agent behave given the system actions. The system actions are predicted and conditioned on the target agent’s actions. The reward of the actions for the target agent is obtained by predicting a structured output given the whole dialogue. Besides, Shi et al. (2021) use a modular framework containing a language model to generate responses, a response detector would automatically annotate the response with a negotiation strategy, and an RL-based reward function to assign a score to the strategy. Instead of the modular framework which separates policy learning and response generation, Gao et al. (2021) propose an integrated framework with deep Q-learning, which includes multiple channel negotiation skills. It allows agents to leverage parameterized DQN to learn a comprehensive negotiation strategy that integrates linguistic communication skills and bidding strategies.

5.3.2 Supervised Learning

Supervised learning (SL) is another popular paradigm for policy learning. Lewis et al. (2017) adopt a Seq2Seq model to learn what action should be taken by maximizing the likelihood of the training data. However, supervised learning only aims to mimic the average human behavior, so He et al. (2018) propose to finetune the supervised model to directly optimize for a particular dialogue reward function, which is defined as i) the utility function of the final price for the buyer and seller ii) the difference between two agents’ utilities iii) the number of utterances in the dialogue. Zhou et al. (2020) train a strategy predictor to predict whether a certain negotiation strategy occurred in the next utterance using supervised training. The system response would be generated conditioned on the user utterance, dialogue context, and the predicted negotiation strategy. In addition, Joshi et al. (2021) incorporate a pragmatic strategies graph network with the seq2seq model to create an interpretable policy learning paradigm. Recently, Dutt et al. (2021b) propose a generalised framework for identifying resisting strategies in persuasive negotiations using a pre-trained BERT model Devlin et al. (2019).

5.3.3 Multi-task Learning

Multi-task learning aims to jointly train the system with several sub-tasks and finally achieve satisfactory performance. Li et al. (2020b) propose an end-to-end framework that integrates several sub-tasks including intent and semantic slot classification, response generation and filtering tasks in a Transformer-based pre-trained model. Zhou et al. (2020) propose to jointly model both semantic and strategy history using finite state transducers (FSTs) with hierarchical neural models. Chawla et al. (2022) integrate a preference-guided response generation model with a ranker module to identify opponents’ priority.

6 New Frontiers and Challenges

Previous sections summarize the prominent achievements of previous work in negotiation dialogue, including benchmarks, evaluation metrics and methodology. In this section, we will discuss some new frontiers which allow negotiation dialogue systems to be fit actual application needs and applied in real-world scenarios.

6.1 Multi-modal Negotiation Dialogue

Existing research works in negotiation dialogue only consider text format as inputs and outputs. However, humans tend to perceive the world in multi-modal patterns, not only text but also audio and visual information. For example, the facial expression and emotions of participants in a negotiation dialogue could be important cues for making negotiation decisions. Further work can consider adding this non-verbal information into the negotiation process.

6.2 Multi-Party Negotiation Dialogue

Although some work sheds light on multi-party negotiation, most current negotiation dialogue benchmarks and methods predominantly focus on two-party settings. Therefore, multi-party negotiation dialogues are under-explored. Future work can consider collecting dialogues in multi-party negotiation scenarios, including General multi-party negotiation and Team negotiation. Specifically, General multi-party negotiation is a type of bargaining where more than two parties negotiate toward an agreement. For example, next-year budget discussion with multiple department leaders in a large company. Team negotiation is a team of people with different relationships and roles. It is normally associated with large business deals and highlights the significance of relationships between multi-parties. There could be several roles, including leader, recorder, and examiner, in a negotiation team Halevy (2008).

6.3 Cross-Culture & Multi-lingual Negotiation Dialogue

Existing negotiation dialogue benchmarks overwhelmingly focused on English while leaving other languages and backgrounds under exploration. With the acceleration of globalization, a dialogue involving individuals from different culture backgrounds participants becomes increasingly important and necessary. That is, there is an urgent need to provide people with a negotiation dialogue system with different cultures and multi-lingual. Further works can consider incorporating multi-lingual utterances and social norms among different countries into negotiation dialogue benchmarks.

6.4 Negotiation Dialogue in Real-world Scenarios

As discussed in Section 3, previous works have already proposed many negotiation dialogue benchmarks in various scenarios. However, we notice that most of these benchmarks are artefacts through human crowd-sourcing. Participants are often invited to play specific roles in the negotiation dialogue. The resulting dialogues may not perfectly reflect the negotiations in real-world scenarios (e.g., politics, business). Therefore, it could be a promising research direction to collect real negotiation dialogues. For example, one could collect recorded business meetings or phone calls.

7 Conclusion

This paper presents the first systematic review on the progress of the negotiation dialogue system. We thoroughly summarize the existing works, which cover various domains and highlight their challenges respectively. Besides, we summarize currently available benchmarks, evaluations, and methodologies. In addition, we shed light on some new trends in this research field. We hope this survey can facilitate future research on negotiation dialogue systems.

References

Afantenos et al. (2012) Stergos Afantenos, Nicholas Asher, Farah Benamara, Anaıs Cadilhac, Cedric Dégremont, Pascal Denis, Markus Guhe, Simon Keizer, Alex Lascarides, Oliver Lemon, et al. 2012. Modelling strategic conversation: model, annotation design and corpus. In Proceedings of the 16th Workshop on the Semantics and Pragmatics of Dialogue (Seinedial), Paris.
Asher et al. (2016) Nicholas Asher, Julie Hunter, Mathieu Morey, Benamara Farah, and Stergos Afantenos. 2016. Discourse structure and dialogue acts in multiparty dialogue: the STAC corpus. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2721–2727, Portorož, Slovenia. European Language Resources Association (ELRA).
Basave and He (2016) Amparo Elizabeth Cano Basave and Yulan He. 2016. A study of the impact of persuasive argumentation in political debates. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1405–1413.
Bazerman et al. (2000) Max H Bazerman, Jared R Curhan, Don A Moore, and Kathleen L Valley. 2000. Negotiation. Annual review of psychology, 51(1):279–314.
Bazerman and Neale (1993) Max H Bazerman and Margaret Ann Neale. 1993. Negotiating rationally. Simon and Schuster.
Blader and Chen (2012) Steven L Blader and Ya-Ru Chen. 2012. Differentiating the effects of status and power: a justice perspective. Journal of personality and social psychology, 102(5):994.
Boritchev and Amblard (2022) Maria Boritchev and Maxime Amblard. 2022. A multi-party dialogue ressource in French. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 814–823, Marseille, France. European Language Resources Association.
Chawla et al. (2021a) Kushal Chawla, Rene Clever, Jaysa Ramirez, Gale Lucas, and Jonathan Gratch. 2021a. Towards emotion-aware agents for negotiation dialogues. In 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), pages 1–8. IEEE.
Chawla et al. (2021b) Kushal Chawla, Gale Lucas, Jonathan May, and Jonathan Gratch. 2021b. Exploring early prediction of buyer-seller negotiation outcomes. arXiv preprint arXiv:2004.02363.
Chawla et al. (2022) Kushal Chawla, Gale Lucas, Jonathan May, and Jonathan Gratch. 2022. Opponent modeling in negotiation dialogues by related data adaptation. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 661–674, Seattle, United States. Association for Computational Linguistics.
Chawla et al. (2021c) Kushal Chawla, Jaysa Ramirez, Rene Clever, Gale Lucas, Jonathan May, and Jonathan Gratch. 2021c. CaSiNo: A corpus of campsite negotiation dialogues for automatic negotiation systems. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3167–3185, Online. Association for Computational Linguistics.
Cheng et al. (2019) Minhao Cheng, Wei Wei, and Cho-Jui Hsieh. 2019. Evaluating and enhancing the robustness of dialogue systems: A case study on a negotiation agent. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3325–3335.
Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186.
Dutt et al. (2020) Ritam Dutt, Rishabh Joshi, and Carolyn Rose. 2020. Keeping up appearances: Computational modeling of face acts in persuasion oriented discussions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7473–7485, Online. Association for Computational Linguistics.
Dutt et al. (2021a) Ritam Dutt, Sayan Sinha, Rishabh Joshi, Surya Shekhar Chakraborty, Meredith Riggs, Xinru Yan, Haogang Bao, and Carolyn Rose. 2021a. ResPer: Computationally modelling resisting strategies in persuasive conversations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 78–90, Online. Association for Computational Linguistics.
Dutt et al. (2021b) Ritam Dutt, Sayan Sinha, Rishabh Joshi, Surya Shekhar Chakraborty, Meredith Riggs, Xinru Yan, Haogang Bao, and Carolyn Penstein Rosé. 2021b. Resper: Computationally modelling resisting strategies in persuasive conversations. arXiv preprint arXiv:2101.10545.
English and Heeman (2005) Michael English and Peter A Heeman. 2005. Learning mixed initiative dialog strategies by using reinforcement learning on both conversants. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 1011–1018.
Fershtman (1990) Chaim Fershtman. 1990. The importance of the agenda in bargaining. Games and Economic Behavior, 2(3):224–238.
Fisher et al. (2011) Roger Fisher, William L Ury, and Bruce Patton. 2011. Getting to yes: Negotiating agreement without giving in. Penguin.
Fransen et al. (2015) Marieke L Fransen, Edith G Smit, and Peeter WJ Verlegh. 2015. Strategies and motives for resistance to persuasion: An integrative framework. Frontiers in psychology, 6:1201.
Frith and Frith (2005) Chris Frith and Uta Frith. 2005. Theory of mind. Current biology, 15(17):R644–R645.
Gao et al. (2021) Xiaoyang Gao, Siqi Chen, Yan Zheng, and Jianye Hao. 2021. A deep reinforcement learning-based agent for negotiation with multiple communication channels. In 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), pages 868–872. IEEE.
Georgila et al. (2014) Kallirroi Georgila, Claire Nelson, and David Traum. 2014. Single-agent vs. multi-agent techniques for concurrent reinforcement learning of negotiation dialogue policies. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 500–510.
Halevy (2008) Nir Halevy. 2008. Team negotiation: Social, epistemic, economic, and psychological consequences of subgroup conflict. Personality and Social Psychology Bulletin, 34(12):1687–1702.
He et al. (2018) He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018. Decoupling strategy and generation in negotiation dialogues. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2333–2343, Brussels, Belgium. Association for Computational Linguistics.
Hiraoka et al. (2015) Takuya Hiraoka, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2015. Evaluation of a fully automatic cooperative persuasive dialogue system. In Natural Language Dialog Systems and Intelligent Assistants, 6th International Workshop on Spoken Dialogue Systems, IWSDS 2015, Busan, Korea, January 11-13, 2015, pages 153–167. Springer.
Joshi et al. (2021) Rishabh Joshi, Vidhisha Balachandran, Shikhar Vashishth, Alan Black, and Yulia Tsvetkov. 2021. Dialograph: Incorporating interpretable strategy-graph networks into negotiation dialogues. In International Conference on Learning Representations.
Keizer et al. (2017) Simon Keizer, Markus Guhe, Heriberto Cuayáhuitl, Ioannis Efstathiou, Klaus-Peter Engelbrecht, Mihai Dobre, Alex Lascarides, and Oliver Lemon. 2017. Evaluating persuasion strategies and deep reinforcement learning methods for negotiation dialogue agents. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 480–484, Valencia, Spain. Association for Computational Linguistics.
Kornilova et al. (2022) Anastassia Kornilova, Vladimir Eidelman, and Daniel Douglass. 2022. An item response theory framework for persuasion. In Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 77–86. Association for Computational Linguistics.
Lambert and Carberry (1992) Lynn Lambert and Sandra Carberry. 1992. Modeling negotiation subdialogues. In 30th Annual Meeting of the Association for Computational Linguistics, pages 193–200.
Langlet and Clavel (2018) Caroline Langlet and Chloé Clavel. 2018. Detecting user’s likes and dislikes for a virtual negotiating agent. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, pages 103–110.
Leung and Cohen (2011) Angela K-Y Leung and Dov Cohen. 2011. Within-and between-culture variation: individual differences and the cultural logics of honor, face, and dignity cultures. Journal of personality and social psychology, 100(3):507.
Lewandowska (1982) Barbara Lewandowska. 1982. Meaning negotiation in dialogue. In Coling 1982 Abstracts: Proceedings of the Ninth International Conference on Computational Linguistics Abstracts.
Lewicki et al. (2011) Roy J Lewicki, David M Saunders, John W Minton, J Roy, and Negotiation Lewicki. 2011. Essentials of negotiation. McGraw-Hill/Irwin Boston, MA, USA:.
Lewis et al. (2017) Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017. Deal or no deal? end-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2443–2453, Copenhagen, Denmark. Association for Computational Linguistics.
Li et al. (2020a) Jialu Li, Esin Durmus, and Claire Cardie. 2020a. Exploring the role of argument structure in online debate persuasion. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8905–8912, Online. Association for Computational Linguistics.
Li et al. (2021) Jiaqi Li, Ming Liu, Zihao Zheng, Heng Zhang, Bing Qin, Min-Yen Kan, and Ting Liu. 2021. Dadgraph: A discourse-aware dialogue graph neural network for multiparty dialogue machine reading comprehension. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE.
Li et al. (2020b) Yu Li, Kun Qian, Weiyan Shi, and Zhou Yu. 2020b. End-to-end trainable non-collaborative dialog system. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8293–8302.
Liu et al. (2021) Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 3469–3483. Association for Computational Linguistics.
Mishra et al. (2022) Kshitij Mishra, Azlaan Mustafa Samad, Palak Totala, and Asif Ekbal. 2022. PEPDS: A polite and empathetic persuasive dialogue system for charity donation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 424–440, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Nazari et al. (2015) Zahra Nazari, Gale M Lucas, and Jonathan Gratch. 2015. Opponent modeling for virtual human negotiators. In International Conference on Intelligent Virtual Agents, pages 39–49. Springer.
Nouri and Traum (2014) Elnaz Nouri and David Traum. 2014. Initiative taking in negotiation. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 186–193, Philadelphia, PA, U.S.A. Association for Computational Linguistics.
Olekalns and Smith (2003) Mara Olekalns and Philip L Smith. 2003. Testing the relationships among negotiators’ motivational orientations, strategy choices, and outcomes. Journal of experimental social psychology, 39(2):101–117.
Sharma et al. (2013) Sudeep Sharma, William P Bottom, and Hillary Anger Elfenbein. 2013. On the role of personality, cognitive ability, and emotional intelligence in predicting negotiation outcomes: A meta-analysis. Organizational Psychology Review, 3(4):293–336.
Shi et al. (2021) Weiyan Shi, Yu Li, Saurav Sahay, and Zhou Yu. 2021. Refine and imitate: Reducing repetition and inconsistency in persuasion dialogues via reinforcement learning and human demonstration. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3478–3492.
Shi and Huang (2019) Zhouxing Shi and Minlie Huang. 2019. A deep sequential model for discourse parsing on multi-party dialogues. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7007–7014.
Sierra et al. (1997) Carles Sierra, Nick R Jennings, Pablo Noriega, and Simon Parsons. 1997. A framework for argumentation-based negotiation. In International Workshop on Agent Theories, Architectures, and Languages, pages 177–192. Springer.
Sycara and Dai (2010) Katia Sycara and Tinglong Dai. 2010. Agent reasoning in negotiation. In Handbook of group decision and negotiation, pages 437–451. Springer.
Sycara (1990) Katia P Sycara. 1990. Persuasive argumentation in negotiation. Theory and decision, 28(3):203–242.
Tran et al. (2022) Nhat Tran, Malihe Alikhani, and Diane Litman. 2022. How to ask for donations? learning user-specific persuasive dialogue policies through online interactions. In Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, pages 12–22.
Traum et al. (2008) David Traum, Stacy C Marsella, Jonathan Gratch, Jina Lee, and Arno Hartholt. 2008. Multi-party, multi-issue, multi-strategy negotiation for multi-modal virtual agents. In International workshop on intelligent virtual agents, pages 117–130. Springer.
Walton and McKersie (1991) Richard E Walton and Robert B McKersie. 1991. A behavioral theory of labor negotiations: An analysis of a social interaction system. Cornell University Press.
Wang et al. (2019) Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019. Persuasion for good: Towards a personalized persuasive dialogue system for social good. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5635–5649, Florence, Italy. Association for Computational Linguistics.
Yamaguchi et al. (2021a) Atsuki Yamaguchi, Kosui Iwasa, and Katsuhide Fujita. 2021a. Dialogue act-based breakdown detection in negotiation dialogues. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 745–757, Online. Association for Computational Linguistics.
Yamaguchi et al. (2021b) Atsuki Yamaguchi, Kosui Iwasa, and Katsuhide Fujita. 2021b. Dialogue act-based breakdown detection in negotiation dialogues. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 745–757.
Yang et al. (2021) Runzhe Yang, Jingxiao Chen, and Karthik Narasimhan. 2021. Improving dialog systems for negotiation with personality modeling. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 681–693, Online. Association for Computational Linguistics.
Zhang et al. (2020) Zheng Zhang, Lizi Liao, Xiaoyan Zhu, Tat-Seng Chua, Zitao Liu, Yan Huang, and Minlie Huang. 2020. Learning goal-oriented dialogue policy with opposite agent awareness. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 122–132, Suzhou, China. Association for Computational Linguistics.
Zhao et al. (2019) Tiancheng Zhao, Kaige Xie, and Maxine Eskenazi. 2019. Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1208–1218.
Zhou et al. (2019) Yiheng Zhou, He He, Alan W Black, and Yulia Tsvetkov. 2019. A dynamic strategy coach for effective negotiation. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 367–378, Stockholm, Sweden. Association for Computational Linguistics.
Zhou et al. (2020) Yiheng Zhou, Yulia Tsvetkov, Alan W Black, and Zhou Yu. 2020. Augmenting non-collaborative dialog systems with explicit semantic and strategic dialog history. In International Conference on Learning Representations.