Goal Driven Discovery of Distributional Differences via Language Descriptions

Exploring large corpora and generating discoveries from them can be ad hoc and laborious. For example, to compare the side effects of drug A and drug B, doctors might inspect two large corpora of patients’ self-reported reactions after taking each drug; based on ad hoc insights, they hypothesize that patients taking drug A more often “mentions feelings of paranoia”, and then validate this hypothesis by laboriously inspecting the two corpora. Since machines can automatically process a large amount of texts, we might hope for ML systems to facilitate exploratory analyses like the one above.

However, an ML task requires a unified input-output space and evaluation metric so that it can be automated, benchmarked, learned, and analyzed. To this end, we formalize one type of exploratory analysis problem as a natural language generation task: goal driven discovery of differences between text distributions via language descriptions (D5). As shown in Figure 1, the input to the D5 task is a “problem” comprising a description of a user-specified exploration goal (understanding side effects) and a corpus pair (text samples from the distributions of self-reported reactions after taking each drug). The output is a “discovery” represented as a natural language predicate (“mentions feelings of paranoia”). We evaluate a discovery with two criteria (Section 3): (1) validity: it should describe a true difference (Zhong et al., 2022); and (2) relevance to the goal (McGarry, 2005).

Since D5 is open-ended and aims at discovering unknowns, the most popular benchmark practice—comparing system-generated outputs with human-written references on a test set—is infeasible. We therefore design two evaluation strategies.

• •

  Diagnostic: we synthesized a dataset of D5 problems with known solutions, SynD5, to diagnose whether a D5 system can recover known differences between two synthetic corpora. This strategy is cheap and automated but might not reflect user utility in real applications.

• •

  Open-ended: we collected a dataset, OpenD5, by aggregating 675 open-ended D5 problems ranging across business, social sciences, humanities, health, and machine learning (Figure 2), comprising 4.4 million text samples in total across problem corpora. We then manually evaluated a subset of the output discoveries. This strategy is subjective and expensive, but useful for obtaining qualitative insights on more realistic applications.

These two strategies allow us to quantitatively evaluate and compare D5 systems. For example, we compared 1) the system from Zhong et al. (2022) designed to describe corpus-level differences without goals, and 2) a goal-conditioned variant that we develop in Section 4. We found language models successfully use the specified goal: the goal-conditioned variant is correct 12% more often on SynD5, and it produces relevant candidate discoveries 31% more often on OpenD5.

We envision OpenD5 to be a growing, diverse repository of open-ended D5 problems. They will not only help us evaluate D5 systems more reliably, but also allow the following operations:

To conclude, we show that D5 can be quantitatively evaluated, automated, analyzed, and learned. Like other ML tasks, it would benefit from a more diverse, authentic, and larger dataset. We hope future works can gather feedback from domain experts and curate an ever-larger dataset of D5 problems, thus accelerating exploratory analyses and facilitating scientific discoveries. ¹¹1Our code is released at https://github.com/ruiqi-zhong/D5 and our code to download OpenD5 is released at https://github.com/petezh/OpenD5. Given the limitations of our system, practitioners should interpret its outputs with caution and not use it to fully automate scientific discoveries.

We first introduce how each input problem is formatted. Then we discuss 1) how we synthesized SynD5, which is used for automatic diagnostic evaluation, and 2) how we collected OpenD5, which is used to investigate the practical value of D5 systems in open-ended applications.

For the goal of comparing the side effects of drug A and drug B, how do we evaluate a system-generated discovery that Corpus A “mention feelings of paranoia” more often? First, it needs to be valid, such that indeed more samples from Corpus A satisfy this predicate, which can be evaluated (approximately) objectively. Second, it needs to be relevant to the goal of understanding side effects, which depends on the user’s subjective judgement. We define validity and relevance below.

Let $\mathcal{D}^{\text{val}}_{A}$ and $\mathcal{D}^{\text{val}}_{B}$ denote the validation sets for Corpus A and B. We define the “validity” $V$ as

$$V(h):=\mathbb{E}_{x\sim\mathcal{D}^{\text{val}}_{A}}[T(h,x)]-\mathbb{E}_{x\sim\mathcal{D}^{\text{val}}_{B}}[T(h,x)].$$

(1)

Computing $V(h)$ is expensive since it requires human annotations $T(h,x)$ on a set of text samples even to evaluate a single discovery $h$. In practice, we do not have the budget to compute $V(h)$ on the entire validation split; therefore, we approximate this quantity by randomly sampling from Corpus $A$ and Corpus $B$. We use these samples to compute an empirical estimate of $V$, as well as a $p$-value for the null hypothesis that $V\leq 0$ using a one-sided t-test.

Therefore, we designed a procedure to evaluate relevance, where human or language model evaluators would score each discovery with \raisebox{-0.9pt}{2}⃝/\raisebox{-0.9pt}{1}⃝/\raisebox{-0.9pt}{0}⃝. The evaluators used the rubric below, which illustrates the meaning of each score with the essay example above:

• •

  \raisebox{-0.9pt}{2}⃝, relevant; e.g. the discovery “write in first person” is directly related to the writing style.

• •

  \raisebox{-0.9pt}{1}⃝, indirectly relevant; e.g. the discovery “use the word “I””, is not exactly a writing style, but can still inform the relevant underlying principle of “write in first person”.

• •

  \raisebox{-0.9pt}{0}⃝, irrelevant; e.g. the discovery “argue for abortion” is unrelated to the writing style.

To minimize biases while comparing two systems, the evaluators are blind to which system generates which discoveries.

To conclude, an ideal discovery would have a high $V$ value with a small $p$-value and achieve ratings of \raisebox{-0.9pt}{2}⃝ in relevance. In the next section, we will build a D5 system that addresses these criteria by first proposing goal-relevant candidate discoveries (hypotheses) and then automatically validate them.

We describe our D5 system, which maps from a corpus pair and an exploration goal to a set of natural language predicates. Our system is inspired by a two-stage model of how humans discover patterns in data: creatively brainstorming hypotheses and then rigorously validating them on the data (Ludwig & Mullainathan, 2022). Analogously, we first propose hypotheses conditioned on the exploration goal and a subset of samples from the corpus pair (Section 4.1). We then use a language model to approximately compute the validity of each hypothesis, and output the valid ones as the final discoveries (Section 4.2). Our system closely mirrors that of Zhong et al. (2022), except that we leverage the goal to propose more relevant hypotheses. Finally, we present a self-supervised learning algorithm to improve an LM’s ability to propose more valid hypotheses (Section 4.3); however, due to API access constraint, we cannot apply it to fine-tune gpt-3, so we provide a proof of concept experiment on Flan-T5 (Chung et al., 2022).

We show that both SynD5 and OpenD5 can be used to quantitatively evaluate D5 systems. Since SynD5 is automatic, we used it to compare a broad range of D5 systems and studied the contributions of three different factors: the quality of the proposer model (gpt-4 vs. text-davinci-003), the use of a validator, and the use of a goal. We then further investigated the effect of using goals under realistic applications through human evaluation on OpenD5.

We report the results in Table 1. We find that using the validator and the goals significantly improve the performance, and gpt-4 outperforms text-davinci-003 with goals and the validator ($p$ < 1% under a t-test). We conducted two additional robustness checks in the Appendix 10: (a) using text-davinci-003 instead of Claude-v1.3 to judge predicate equivalence, and (b) considering discoveries semantically similar to the references also to be correct; our conclusions do not change.

Finally, to improve the accessibility of our research, we ran the same experiments using gpt-3.5-turbo and flan-t5-xxl as our proposer, and report the results in Appendix Table 6. To show that our conclusions are general and not only apply to synthetically generated texts, we additionally constructed an extension of SynD5 with human-written texts by adapting the NYT dataset from Wang et al. (2023), where each text sample is a New York Times article with a topic and a location label: the topic dimension has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy); we then followed the same procedure described in Section 2.2 to create this extension of SynD5, and report our systems’ performance in Appendix Table 7. In all experiments, using the validator and the goal improves the performance.

We report the results in Table 2. Since this evaluation is subjective, the inter-annotator agreement is only moderate (Kappa=$0.56$); however, we can still robustly conclude that text-davinci-003 can leverage goals to propose hypotheses with higher average relevance rating, since this conclusion can be independently reproduced by every individual evaluator with $p<10^{-8}$. To make sure that the same conclusion can be robustly reproduced by external non-authors, we also evaluated the relevance of the hypotheses with Amazon Mechanical Turks, gpt-3.5-turbo, Claude-v1.3, and gpt-4. We report the results in Appendix Table 8 and found that our conclusion robustly holds under five different types of evaluators, including expert authors, external crowdworkers, and language models from different companies with different levels of capabilities.

Finally, we conducted similar experiments for the novelty and significance metrics in Appendix 14 and found that they both benefit from using goals as well. In the next section, we present example discoveries on OpenD5 to qualitatively understand what a D5 system can achieve.

To understand the utility and the limitation of a D5 system, we ran it on OpenD5, a set of realistic D5 problems, and analyze the output discoveries qualitatively.

ML models can also perform inductive reasoning in other modalities, such as vision. Hernandez et al. (2021) describes visual features that activate a neuron; Zhu et al. (2022) describes distribution shifts between the training distribution and the test distribution for images; and Eyuboglu et al. (2022) describes errors made by vision models. We hope future models can perform inductive reasoning in other modalities, such as sound (Aghajanyan et al., 2023) or physical senses (Thomason et al., 2016).

We thank Xiaochuang Han and Sam Bowman for their early discussions on this project. We thank Cathy Chen, Erik Jones, Jessy Lin, Alex Pan, Chenglei Si, Xi Ye, and Tianyi Zhang for their helpful feedback on the paper draft. We thank OpenAI and Anthropic for providing model access.

For each problem, we ran the proposer for  10 times on average; assuming each prompt to be at most 4000 tokens, we spent around $2.4 for each problem on OpenAI APIs if we use gpt-4 and text-davinci-003, and the cost would decrease to $0.8 if we use gpt-3.5-turbo. Notice that these estimates are computed based on the prices as of 05/14/2023, and we expect the price to further decrease in the future. We ran the Flan-T5 based validator for 2̃ hours on 1 80G A100 GPUs.

The total amount of computational resources spent in this research paper is around $2,500 in terms of OpenAI API and 3,000 hours of compute on A100 GPU with 80G memory.

The high-level description is in Section 2.2. Here we discuss the procedure that generated SynD5.

We consider three dimensions of differences: topic, genre, and style. For each, we generated 14/9/7 values, e.g., “celebrity love stories” and “sports team recruiting athletes for the topic attribute, “rap lyrics” and “screen play” for the style attribute, and “French” and “Spanish” for the language attribute. We then used GPT-4 and the Claude API to synthesize 54K text samples, where for each text sample we sampled a topic, genre, and style randomly, e.g. “Write a rap about a sports team recruiting athletes in French”. To synthesize a random SynD5 problem, we randomly sampled a distractor dimension (e.g. language) and a target dimension (e.g. topic), and for each dimension we sampled two random values (e.g. English and French for language, sports and art for topic).

For each problem, we sampled 10 texts for corpus $A$ such that all of them satisfy one sampled value for the distractor dimension (e.g. corpus $A$ is entirely in English), and 10 texts for corpus $B$ for to satsify the other distractor dimension (e.g. corpus $B$ is entirely in French). Then we set $V$ fraction of corpus $A$ to satisfy the reference target attribute, e.g. “is sports-related”, and $f$ fraction of corpus $B$ to satisfy the other value for the target dimension (e.g. “is art-related”). We chose $V$ uniformly at random from [0.6, 0.8, 1]. Finally, we provide $k$ example hypotheses from the target dimension other than the target dimension values for Corpus A and Corpus B, and we chose $k$ from [0, 2] uniformly at random. We then sampled 300 D5 problems in total from this distribution.

Table 4 shows the accuracy of different systems using text-davinci-003 as the judge for semantic equivalence. Table 5 shows the accuracy of different systems if we consider outputs semantically similar to the reference to be correct. Across all setups, we found that the conclusion reached in Section 5 still holds under these robustness checks.

To improve the accessibility of our research, we ran the same experiments with gpt-3.5-turbo and flan-t5-xxl, and report the results in Appendix Table 6. To show that our conclusions are general and not only apply to synthetically generated texts, we additionally constructed an extension of SynD5 with human-written texts by adapting the NYT dataset from Wang et al. (2023), where each text sample is a New York Times article with a topic and a location label: the topic dimension has 9 different values (e.g., politics, arts) and the location dimension has 10 different values (French, Italy); we then followed the same procedure described in Section 2.2 to create this extension of SynD5, and report our systems’ performance in Appendix Table 7. Under all experimental setups, using the validator and the goal improves the performance.

We prompt Claude v1.3 (Bai et al., 2022b) to judge whether the predicated predicate is similar to the reference. We consider a response that leads to a“yes” to be correct when we require the discovery to be semantically equivalent to the reference, and consider a response that leads to a “yes” or “related” to be correct when we require the discovery to be semantically similar to the reference.

“ Is text_a and text_b similar in meaning? respond with yes, related, or no.

Here are a few examples.
Example 1:
text_a: has a topic of protecting the environment
text_b: has a topic of environmental protection
and sustainability
output: yes

Example 2:
text_a: has a language of German
text_b: has a language of Deutsch
output: yes

Example 3:
text_a: has a topic of the relation between political figures
text_b: has a topic of international diplomacy
output: related

Example 4:
text_a: has a topic of the sports
text_b: has a topic of sports team recruiting new members
output: related

Example 5:
text_a: has a named language of Korean
text_b: uses archaic and poetic diction
output: no

Example 6:
text_a: has a named language of Korean
text_b: has a named language of Japanese
output: no

Target:
text_a: {predicate}
text_b: {reference}
output:”

To make sure that the conclusion that “using goal in the context can improve hypotheses relevance” can be robustly reproduced by external non-authors, we also evaluated the relevance of the hypotheses with Amazon Mechanical Turks, GPT-3.5-turbo, Claude-v1.3, and GPT-4. We report the results in Table 8 and found that the conclusion still robustly holds.

Not every valid discovery is meaningful. For example, if the goal is to understand the topical differences between news from 2008 (Corpus A) and news from 2007 (Corpus B), the discovery that Corpus A “contains news from 2008” is completely valid by definition but meaningless, since it provides only trivial information and is irrelevant to the goal of understanding topical differences.

McGarry (2005) surveyed a list of desirable properties for discovery, and we condensed them into three submetrics to rate how meaningful a discovery is based on the exploration goal: 1) relevance, 2) novelty, and 3) significance. We evaluate these independently of validity and assume that the discovery is already valid. For example, the discovery that “something can travel faster than light” is meaningful if true, even though it is highly implausible.

We rate each submetric with \raisebox{-0.9pt}{0}⃝, \raisebox{-0.9pt}{1}⃝, or \raisebox{-0.9pt}{2}⃝, where higher is better. We show the evaluation instructions below and present our rating on text-davinci-003 proposed hypotheses.

We present the full details of how we generated the hypotheses with the language model. The process roughly contains four stages: 1) obtaining representative samples for each corpus, 2) sampling hypotheses from GPT-3, 3) rewriting hypotheses, and 4) optionally plugging in example hypotheses.

Here we provide a high-level description of how the data was collected. For each problem in OpenD5, we used our proposer to produce a list of hypotheses. We automatically judged each hypothesis on a subset of samples from the research split using GPT-3 text-davinci-002 (Ouyang et al., 2022), Flan-T5 (Chung et al., 2022), and a model trained with RLHF from Bai et al. (2022a). We created the input distribution for training by combining and equally weighting the following $3\times 2=4$ distributions: the subset of $(h,x)$ pairs that GPT-3/Flan-T5/“RLHF” considers Yes or No to be the most likely answer. We then collected averaged turker ratings for in total 3138 $(h,x)$ pairs and used them to fine-tune Flan-T5 to create the validator (Chung et al., 2022).

To test cross problem generalization capability of our D5 system, whenever we applied our D5 system to a problem in OpenD5 in Section 6.1, we used a validator that is NOT fine-tuned on the $(h,x)$ pairs from this problem. We achieved this by keeping track of which problem each $(h,x)$ pair comes from and split all the $(h,x)$ pairs into three folds based on the problems; whenever we applied our D5 system to a problem, we used the validator trained on the two folds that do not contain this problem.

Our system in total produces 3296 discoveries on OpenD5. However, we do not have enough budget to validate every finding, since estimating $V$ is expensive (Section LABEL:sec:soundness). Therefore, from the remaining 3296 discoveries, we manually selected 21 discoveries that 1) the authors think are relevant enough, 2) are representative of potential use cases, 3) do not require expert knowledge for Turkers to judge, and 4) are likely to achieve a small $p$-value with fewer than 200 samples from $\mathcal{D}^{\text{val}}_{A}$ and $\mathcal{D}^{\text{val}}_{B}$. We then estimated their validity based on the procedure described in Section LABEL:sec:soundness by using fewer than 200 samples from the validation split and calculated the $p$-values.²²2We determined the number of samples s.t. $V^{\prime}$ can achieve a $p$-value of $0.005$. Estimating $V$ for these discoveries costs $\sim$$1500. Since we are testing multiple discoveries and each of them can be statistically significant merely due to chance, we keep 13 discoveries with $V$ that are significantly non-zero with $p$-value below 7%, a threshold determined by the Benjamini Hochberg’s procedure with a false discovery rate of 10%. In other words, fewer than 10% of the discoveries presented are false discoveries in expectation.

We still face many challenges in building a broadly useful system. We describe technical challenges that machine learning researchers can tackle in Appendix 19.1 and organizational challenges that require domain experts in Appendix 19.2.

Since the problems in OpenD5 are open-ended, our system could potentially produce discoveries with higher validity scores than our current system. Therefore, we design a self-supervised learning algorithm to improve an LM’s ability to propose more valid hypotheses, using the principle that it is easier to validate a discovery than to generate one.

However, since we cannot fine-tune instruction-tuned GPT-3, we can only experiment with Flan-T5 (Chung et al., 2022), an open-sourced instruction-tuned model that might only work well for easier “mini-problems”. As a proof of concept, we tested our algorithms for describing groups of four samples, where each group comes from a text cluster. As an overly simplified example, we will give the LM the prompt “Group A: 1. dog 2. cat 3. pig 4. cow. Group B: 1. phone 2. laptop 3. desk 4. cup” as an input and the LM can output “mentions an animal” as a hypothesis.

We qualitatively compare the discovery generated by our D5 system to the top-5 unigram features extracted by Naive Bayes, a traditional exploratory analysis method. The Naive Bayes method is effective when the target difference can be saliently reflected by individual words. For example, “yo” implies a rap genre, “die” implies a language of Deutsch, and [“rank”, “higher”, “univeristy”] hints at the topic of “college ranking changes”. Additionally, compared to black-box neural networks, such a method is fully interpretable.

In comparison, D5 can directly generate a semantically coherent description for the target difference, saving users’ time to guess the underlying correlation by inspecting the top unigram features. Additionally, it can capture differences that are hard to detect at a word level; for example, “the genre of biblical scripture” is mainly reflected in its sentence structure rather than individual words. Finally, D5 only describes goal-related differences, while Naïve Bayes picks up on any discriminative feature; for example, when identifying the topical differences between a English and a Deutsch corpus, Naïve Bayes fails catastrophically and only picks up common determiners such as “the” or “die” instead of topic words, since they are the most useful feature at telling which sample comes from which corpus. Given the respective strength of D5 and traditional exploratory methods, we envision D5 to serve as a complementary method to traditional methods.

(This section describes an interesting research direction we did not have time to fully pursue.)

A screenshot of the annotation interface can be seen in Figure 6.

Many of our datasets come from the following sources: the Computational Models of Social Meaning class from Columbia Universityhttp://www1.cs.columbia.edu/~smara/teaching/S18/, the ACL Anthology https://aclanthology.org, , and Kaggle datasets with an NLP tag. https://www.kaggle.com

abc-headlines. We collect headlines published by ABC news, an American news company from Kulkarni (2018). ABC headlines are directly downloaded from Harvard Dataverse. The year is extracted from the publication date field. Samples are constructed from the headline text. The data is downloadable from https://doi.org/10.7910/DVN/SYBGZL with license CC0 1.0.

ad-transcripts. We collect ad scripts from a variety of industries from Hartman (2019). Ad transcripts are directly downloaded from Kaggle. The top eight industries by frequency are selected. Newlines are replaced with spaces. The dataset is downloadable from https://www.kaggle.com/datasets/kevinhartman0/advertisement-transcripts-from-various-industries with license CC0 Public Domain.

admin-statements. We collect statements of administration policy from American presidents from Progress (2022). Administration statements are extracted from a collection hosted on GitHub. Extraneous symbols are removed and samples are split by paragraph. The dataset is downloadable from https://github.com/unitedstates/statements-of-administration-policy#statements-of-administration-policy and origin files have a Creative Commons Attribution 3.0 License.

ai2-natural-instruction. We collect a learning-from-instructions dataset released by the Allen Institute for AI from Mishra et al. (2022). Natural instruction tasks are directly downloaded without modification. The dataset is released under an Apache-2.0 license.

airline-reviews. We collect reviews of airlines collected from the review website Skytrax. Airline reviews for airlines, airports, and seats are downloaded from a public GitHub repository. Names of aircraft, airlines, countries, and traveler types are standardized. Ratings of 1, 4, or 5 on a scale of 5, and 1, 5, 8, or 10 on a scale of 10 are kept. This dataset can be downloaded via https://github.com/quankiquanki/skytrax-reviews-dataset.

aita. We collect posts on the “Am I The Asshole” Subreddit, an online forum people ask others whether they were in the wrong from O’Brien (2020). Posts from r/AmITheAsshole are downloaded from a praw scrape of Reddit. Topic areas are chosen based on common themes in posts and coarsely defined based on manual keywords. Each post can belong to multiple topic areas. The dataset can be downloaded at https://doi.org/10.5281/zenodo.3677563.

all-the-news. We collect news articles collected from various outlets between 2015 and 2017 from Thompson (2019). News articles are downloaded directly from the Components website. The titles are used as text samples.The dataset can be downloaded at https://components.one/datasets/all-the-news-articles-dataset .

amazon-reviews. We collect Amazon reviews collected from various product categories from Ni et al. (2019). Amazon reviews are downloaded from a 2018 crawl of the website. The first 100,000 review texts are treated as the text sample. The dataset can be downloaded at https://nijianmo.github.io/amazon/index.html .

armenian-jobs. We collect job postings in Armenia from Udacity (2017). The Armenian job postings dataset is downloaded from a snapshot on GitHub. Different IT jobs are manually coded and time intervals are defined in order to balance sample availability. The dataset can be downloaded at https://www.kaggle.com/datasets/udacity/armenian-online-job-postings .

boolq. We collect a reading comprehension dataset of yes/no questions from Clark et al. (2019). Boolean questions are downloaded directly as is. The dataset can be downloaded at https://github.com/google-research-datasets/boolean-questions with license CC-SA-3.0.

clickbait-headlines. We collect headlines across time from the Examiner, a clickbait news site from Kulkarni (2020a). The Examiner headlines are directly downloaded from Kaggle. The year is extracted from the publication date field. Samples are constructed from the headline text. The dataset can be downloaded at https://www.kaggle.com/datasets/therohk/examine-the-examiner, with license CC0: public domain.

convincing-arguments. We collect arguments on a variety of topics annotated for convincingness from Habernal & Gurevych (2016). Annotated arguments are downloaded from the GitHub repository. Arguments are sorted by rank. The bottom 400 are treated as “unconvincing”, the top 200 are treated as “convincing”, and the next 200 are treated as “somewhat convincing.” The dataset can be downloaded at https://github.com/UKPLab/acl2016-convincing-arguments, with license CC-BY 4.0.

craigslist-negotiations. We collect dialogue from Craigslist negotiations, an online seller platform from He et al. (2018). Craigslist negotiations are downloaded from Huggingface. Sequences which contained a “quit” intention or “reject” intention are categorized as failures; those which contained an “accept” intention are categorized as successes. The mid-price is defined as the mean price of the items sold. Within each category, the items are sorted by mid-price. The top half is treated as high-price and the bottom half is treated as low-price. This dataset can be downloaded at https://huggingface.co/datasets/Hellisotherpeople/DebateSum with MIT license.

debate. We collect evidence compiled for American competitive policy debate, published online by debate camps from Roush & Balaji (2020). The train split is downloaded from Huggingface. For each sample, we use the abstract as the text. Arguments are categorized by type, debate camp of origin, and topic/specific argument. For topics, we use domain knowledge to list relevant keywords for each topic and include any sample with a file name that includes any keyword. A single sample can belong to multiple topics. This dataset can be downloaded at https://huggingface.co/datasets/Hellisotherpeople/DebateSum with MIT license.

dice-jobs. We collect American technology job postings on dice.com from PromptCloud (2017). Job postings are downloaded from Kaggle. Posts from the six most popular companies are categorized by company. We remove miscellaneous characters and blank descriptions. We additionally apply our splitting procedure to reduce description length. This dataset can be downloaded at https://www.kaggle.com/datasets/PromptCloudHQ/us-technology-jobs-on-dicecom under CC BY-SA 4.0 .

diplomacy-deception. We collect dialogue from games of Diplomacy, which involves deception from Peskov et al. (2020). Diplomacy dialogues are downloaded from GitHub (all splits). The data are ASCII encoded and newlines are removed. Each message and label is treated as a sample. This dataset can be downloaded at https://huggingface.co/datasets/diplomacy_detection under unknown license.

echr-decisions. We collect facts of cases heard before the European Court of Human Rights from Chalkidis et al. (2019). Decisions are downloaded from a public archive. A random sample of 500 decisions is selected from the files. The samples with any violated articles are categorized as “violation,” while the rest are categorized as “no violation.” This dataset can be downloaded at https://paperswithcode.com/dataset/echr under unknown license.

essay-scoring. We collect essays from students from ess (2012). Essays are downloaded from a GitHub repository. Only essays from set 5 are considered. Essays with a score of at least 3 are categorized as good essays, while essays with a score less than 3 are bad essays. This dataset can be downloaded at https://www.kaggle.com/c/asap-aes under unknown license.

fake-news. We collect fake and legitimate news from Pérez-Rosas et al. (2017). Fake news articles are downloaded from the author’s website. Full articles are treated as text snippets. This dataset can be downloaded at http://web.eecs.umich.edu/~mihalcea/downloads.html#FakeNews under CC-BY-4.0.

fomc-speeches. We collect Federal Open Market Committee (FOMC) speeches from 1996-2020, which describe Federal Reserve policy from Mish (2020). Fed speeches are downloaded from Kaggle. The macro indicator data are merged in on the year and month. Full speech text is split by paragraph and categorized by speaker, year, and macroeconomic indicator. This dataset can be downloaded at https://www.kaggle.com/datasets/natanm/federal-reserve-governors-speeches-1996-2020 under unknown license.

genius-lyrics. We collect lyrics collected from Genius.com before 2020 from Lim & Benson (2021). Genius lyrics are downloaded from Google Drive. The lyrics are merged with song metadata and treated as samples. We categorize lyrics by hand-selecting popular artists, common genres, time periods, and view counts (over 1M views is high, 500k-1M is medium). This dataset can be downloaded at https://www.cs.cornell.edu/~arb/data/genius-expertise/ under unknown license.

happy-moments. We collect self-reported happy moments and demographic characteristics from Asai et al. (2018). The HappyDB dataset is downloaded from the official GitHub repository. Demographic data is cleaned and merged into happy moments. Happy moment descriptions are treated as samples and are categorized by type of happy moment, country of origin, and other demographic features. This dataset can be downloaded at https://github.com/megagonlabs/HappyDB under unknown license.

huff-post-headlines. We collect headlines from the news outlet Huffington Post from Misra & Arora (2019) and Misra & Grover (2021). Huffington Post headlines are downloaded from Kaggle. The short description of each article is treated as a sample and tokenized at the sentence level. This dataset can be downloaded at https://rishabhmisra.github.io/publications/ under CC-BY-4.0.

immigration-speeches. We collect congressional and presidential speeches that mention immigration from 1880 to the present from Card et al. (2022). Immigration speeches are downloaded from the replication package. The speech text is preprocessed to remove extraneous spaces. We engineer features corresponding to time periods, well-known speakers, other significant time periods, the racial group under discussion, and the geographic area within the United States. This dataset can be downloaded at https://github.com/dallascard/us-immigration-speeches/releases.

kickstarter. We collect names of startups on kickstarter.com from Mouillé (2017). We download a 2018 crawl from Kickstarter from Kaggle. The project name is treated as the text sample. This dataset can be downloaded at https://www.kaggle.com/datasets/kemical/kickstarter-projects?select=ks-projects-201612.csv under CC BY-NC-SA 4.0.

microedit-humor. We collect funny sentences generated by making one-word edits to normal statements from Hossain et al. (2019). The Microedit dataset is downloaded from the author’s website. We make the relevant edit to each text sample and treat the edited text sample as the data point. We bin the mean annotator grade into 4 and denote each as unfunny, neutral, funny, and very funny, respectively. This dataset can be downloaded at https://paperswithcode.com/dataset/humicroedit.

mnli. We collect a collection of sentence pairs annotated with textual entailment information from a range of genres from Williams et al. (2017). The MNLI corpus is downloaded from the official website. We treat the premise and hypothesis as text samples. This dataset can be downloaded from https://cims.nyu.edu/~sbowman/multinli/, most of which are under the OANC license.

monster-jobs. We collect American job postings on monster.com. Jobs on Monster.com are downloaded from Kaggle. Job descriptions are treated as samples and split at the paragraph and sentence level. We keep and categorize jobs from seventeen large cities. This dataset can be downloaded from https://www.kaggle.com/datasets/PromptCloudHQ/us-jobs-on-monstercom under CC BY-SA 4.0 .

movie-tmdb. We collect movie plot summaries from TMDB from Kaggle (2018). TMDB movie overviews are downloaded from Kaggle. We keep only English movies and bin popularity by deciles. The top decile is considered “hits,” the 70-80th percentiles are considered “average,” and the 30-40th percentiles are considered “bad.” This dataset can be downloaded from https://www.kaggle.com/datasets/tmdb/tmdb-movie-metadata21.

movie-wiki. We collect movie plot summaries collected from Wikipedia from Robischon (2019). Wikipedia movie summaries are downloaded from Kaggle. This dataset can be downloaded from https://www.kaggle.com/datasets/jrobischon/wikipedia-movie-plots under CC BY-SA 4.0.

news-popularity. We collect news headlines posted on social media platforms from Moniz & Torgo (2018). Headlines are downloaded from a reproduction package. The headline and title text are cleaned, and the title is treated as the text sample. The 100 most positive and negative or popular and unpopular articles on each topic are used as distributions. This dataset can be downloaded from https://archive.ics.uci.edu/ml/datasets/News+Popularity+in+Multiple+Social+Media+Platforms.

nli-benchmarks. We collect training examples from various natural language inference (NLI) datasets from Liu et al. (2022). NLI benchmarks are downloaded from a public collection on Google Drive. We examine the premise and hypothesis separately as samples. This dataset can be downloaded from https://github.com/alisawuffles/wanli.

npt-conferences. We collect Non-Proliferation of Nuclear Weapons (NPT) conference transcripts from Barnum & Lo (2020). NPT conference notes are extracted from the accompanying replication package. Text is split by paragraph, and only paragraphs longer than 50 characters are preserved. Text is split into three time ranges: pre-2008, 2008-2012, and post-2012. This dataset can be downloaded from https://journals.sagepub.com/doi/full/10.1177/0022343320960523.

open-deception. We collect arbitrary lies and truths from any domain generated by crowdworkers from Pérez-Rosas & Mihalcea (2015). Open domain lies are downloaded from the public dataset and lie texts are split into lies and truths. This dataset can be downloaded from https://web.eecs.umich.edu/~mihalcea/downloads.html#OpenDeception.

open-review. We collect submissions to ICLR, a machine learning conference from 2018 to 2021. Open review abstracts are accessed via the openreview API. We query for abstracts from the 2018-2021 ICLR blind submissions. Abstracts are classified based on rating: $>=7$ (“great”), 5-6 (“good”), and $<=4$ (“bad”). This dataset can be downloaded from https://openreview.net/.

parenting-subreddits. We collect posts from various parenting-related subreddits, which are text-based forums on the site Reddit from Gao et al. (2021). Posts from various subreddits are downloaded from the paper’s GitHub repository. We clean the text and split the posts according to the topic(s) each post is tagged with. This dataset can be downloaded from https://github.com/SALT-NLP/Parenting_OnlineUsage.

poetry. We collect poems from PoetryFoundation.com from Bramhecha (2019). Poems are downloaded from a 2019 scrape of the PoetryFoundation website from Kaggle. The text is cleaned and split according to subject tags and authorship. This dataset can be downloaded from https://www.kaggle.com/datasets/tgdivy/poetry-foundation-poems under GNU Affero General Public License.

political-ads. We collect political ads observed by Facebook users from pol (2021). Ads are downloaded from the Ad Observer website, which maintains an aggregate of all collected ads. We extract targeting metadata from the targeting field and define splits according to age, gender, location, interests, time, and political lean. This dataset can be downloaded from https://adobserver.org/ad-database/.

qqp. We collect questions from Quora.com from Quora (2017).

rate-my-prof. We collect reviews of lecturers from RateMyProfessor.com from He (2020). We download a sample of RateMyProfessor.com reviews from an online repo. We clean the text and guess the gender of the reviewed lecturer from the first name using the gender-guesser package. Due to data availability, we consider only male and female names. To improve the quality of the classification, we remove any posts which use pronouns from the opposing sex (e.g. “him”). This dataset can be downloaded from https://data.mendeley.com/datasets/fvtfjyvw7d/2 under CC BY 4.0 .

radiology-diagnosis. We collect impressions and medical histories of radiology patients from Pestian et al. (2007). Radiology diagnoses are downloaded from a GitHub copy of the original task dataset. We parse the metadata to retrieve the diagnostic code, decision type, impression, and patient history. Referencing the associated ICD codes, we convert codes to colloquial diagnoses (e.g. 786.2 denotes cough). We treat the histories and impressions as samples and split them according to diagnosis and level of consensus.

reddit-humor. We collect jokes posted on the Reddit forum r/Jokes, a message board for sharing jokes from Weller & Seppi (2020). Jokes are downloaded from the dev and test splits of the dataset. We clean the text and split the dataset according to whether they are labeled as funny. This dataset can be downloaded from https://github.com/orionw/rJokesData under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

reddit-stress. We collect stress-related posts on Reddit from Turcan & McKeown (2019). We split the post text based on which subreddit they are posted on (related to PTSD, anxiety, or stress generally). Reddit posts are downloaded from https://github.com/gillian850413/Insight_Stress_Analysis, and we recommend following the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

reuters-authorship. We collect articles from various Reuters authors from Liu (2011). The articles are split according to the author. Reuters articles are downloaded from the UCI repository https://archive.ics.uci.edu/ml/datasets/Reuter_50_50.

riddles. We generated several riddles. The 3000 most common English words are manually copied from a website. Words with between 5 and 8 characters are kept. We create two popular riddles. First, we split words based on whether they have a duplicate character. We exclude any words with multiple “doubles” or more than 2 of any character. Second, we split words based on whether they have the letter T.

scotus-cases. We collect facts from cases heard by the Supreme Court of the United States (SCOTUS) from Alali et al. (2021). Supreme Court cases are downloaded from a GitHub repository. We identify state/federal parties by manually defining keywords. We split based on the winning party, the identity of each party, and the type of decision. We then define several time periods and relevant political eras and split decisions accordingly. Finally, we split according to the ruling’s policy area and how it changes over time. The dataset can be downloaded from https://paperswithcode.com/paper/justice-a-benchmark-dataset-for-supreme-court under CC-BY-SA.

short-answer-scoring. We collect short answers from students from sho (2013). Short answers are downloaded from a GitHub mirror of the dataset. We consider only responses to essay set 1. The two scores are averaged and binned into good ($>=$ 2.5), medium (1.5-2.5), and bad ($<$1.5). The dataset can be downloaded from https://www.kaggle.com/c/asap-sas.

snli. We collect a collection of sentence pairs annotated with textual entailment information from images from Bowman et al. (2015). The dataset can be downloaded from https://nlp.stanford.edu/projects/snli/ under CC BY-SA 4.0.

squad-v2. We collect reading comprehension questions crowdsourced from Wikipedia articles from Rajpurkar et al. (2018). The dataset can be downloaded from https://rajpurkar.github.io/SQuAD-explorer/ under CC BY-SA 4.0.

stock-news. We collect top news headlines on Reddit, an online message board from Sun (2017). Headlines are downloaded from a GitHub mirror. We clean the text and divide the samples based on whether the DOW rose or fell that day. The dataset can be downloaded from https://github.com/ShravanChintha/Stock-Market-prediction-using-daily-news-headlines under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

suicide-notes. We collect posts from r/SuicideWatch and r/depression, two forums on Reddit fromHe (2021). The post title and body are combined to form the text samples. Samples are split based on whether they were posted in a suicide-related Subreddit. The dataset can be downloaded from a github: https://github.com/hesamuel/goodbye_world, under Reddit License and Terms of Service, and users must follow the Reddit User Agreement and Privacy Policy, as well as remove any posts if asked to by the original user.

times-india-headlines. We collect headlines from Times of India news from Kulkarni (2022). Headlines are downloaded from a Dataverse mirror. We use the first 1000 headlines in each year as samples. The dataset can be downloaded from https://www.kaggle.com/datasets/therohk/india-headlines-news-dataset under CC0 Public Domain.

trial-deception. We collect testimonies from witnesses in real trials from Pérez-Rosas et al. (2015). Trial testimonies are downloaded from the author’s website. The testimonies are divided based on whether they are considered truthful. The dataset can be downloaded from https://web.eecs.umich.edu/~mihalcea/downloads.html#RealLifeDeception.

un-debates. We collect speeches from debates at the United Nations from Baturo et al. (2017). Debate transcripts are downloaded from the Dataverse reproduction package. Samples are divided based on the country and year of the snippet. First, we isolate samples from Russia, China, and the United States and specify 3 time periods of interest. Next, we divide all samples by the decade. Finally, we create distributions for 19 countries of interest. The dataset can be downloaded from https://doi.org/10.7910/DVN/0TJX8Y under CC0 1.0 .

unhealthy-conversations. We collect expert-annotated unhealthy conversations from Price et al. (2020). Conversation transcripts are downloaded from the official GitHub repository. For each annotated attribute, we split the dataset based on whether that form of unhealthy conversation is present in the sample. The dataset can be downloaded from https://github.com/conversationai/unhealthy-conversations under CC BY-NC-SA 4.0.

urban-dictionary. We collect definitions from UrbanDictionary.com, a crowdsourced English dictionary from Kulkarni (2020b). Urban Dictionary entries are downloaded from Kaggle. Definitions are split into groups representing the top 1, 5, and 10 percent of definitions ranked by both upvotes and downvotes; we sample 10,000 from each and create a control distribution by randomly sampling 10,000 definitions from all entries. The dataset can be downloaded from https://www.kaggle.com/therohk/urban-dictionary-words-dataset under CC0 Public Domain.

wikitext. We collect text snippets from Wikipedia from Merity et al. (2016). The Wikipedia snippets are loaded from HuggingFace. We remove any samples that are empty or start with ’=’ (which represent headings); samples are tokenized at the sentence level and used for clustering. The dataset can be downloaded from https://huggingface.co/datasets/wikitext under CC BY-SA 3.0.

yc$-$startups. We collect descriptions of companies that were part of the Y Combinator startup incubator from Bhalotia (2022). YCombinator company descriptions are downloaded from a 2022 scrape on GitHub. Only companies with long descriptions are preserved. Companies are split according to founder characteristics, year, “top company” designation, operating status, and location. The dataset can be downloaded from https://www.kaggle.com/datasets/benhamner/y-combinator-companies.