Why Antiwork: A RoBERTa-Based System for Work-Related Stress Identification and Leading Factor Analysis

Mental health problems are increasingly being recognized as a critical issue in modern society. Conditions like depression and burnout can lead to sleep disorders, insomnia, and even suicide Brooks et al. (2011); Kalmbach et al. (2018); Maslach et al. (2001). Various factors contribute to mental health problems, and one prevalent factor in today’s world is the toxic work environment. Low wages and high pressure at work are two common reasons why people dislike their jobs. Moreover, there are additional detrimental behaviors in the workplace, such as harassment, bullying, and ostracism Limm et al. (2011); Rasool et al. (2021); Luo et al. (2008). Therefore, our goal is to thoroughly examine the impact of toxic workplaces to effectively address the numerous mental health problems associated with them.

Antiwork subreddit is an online forum associated with critiques of work and labor movements Codrea-Rado (2021); Todd (2021); Rockcellist (2019). While some workers are able to stand out and protect their rights, many individuals can only express their dissatisfaction online. While they might feel good after explicitly expressing their negative emotions, it is not truly helpful in relieving their mind if the MH problem is not solved McKenna and Bargh (1999). On the contrary, it might cause emotional contagion, which amplifies their negative emotions and cause mental health problems Kramer et al. (2014).

Recent years have seen a rapid increase in antiwork subreddit O’Connell (2022). COVID-19 and its economic damages to the world increase unemployment and employers are harsher and harsher to their employees Saladino et al. (2020); Osofsky et al. (2020); Chakraborty (2020). The antiwork subreddit has become a popular place for people to share their unpleasant working experiences. A large portion of posts are negative, showing signs of depression and hopelessness that are likely to develop into MH problems Hermida-Carrilo et al. (2023).

Researchers have used behavioral and linguistic cues from social media data, such as Twitter, Reddit, and Facebook, to study the mental health status of users Chancellor and De Choudhury (2020). Neural networks and deep learning methods Gkotsis et al. (2017) are increasingly popular in recent years to predict the mental health status of the user behind their posts, while traditional machine learning techniques such as supervised learning, principal component analysis and support vector machines (SVM) are still the popular choice for simpler NLP tasks. In De Choudhury et al’s work De Choudhury et al. (2016), the goal is to identify individuals who are at risk for suicide. Thus, De Choudhury et al focused on the most distinguishable characteristics of the two groups. If one starts to post in r/SuicideWatch within a range of time after they post in other MH subreddits, they are identified as suicide prone. In this paper, we focus on insights into the antiwork causes and characteristics. We want to find out what the contributing factors are so that we can make the proper attempts to eliminate the root of antiwork sentiments. Therefore, traditional machine learning techniques might not be enough to find out more in-depth results due to their limited inference ability. More advanced techniques, such as LLM, can be applied to handle more complex feature extraction. Together with topic modeling, its result can be more interpreted and thus we can gain deeper insights.

Latent Dirichlet Allocation (LDA) is a common technique used for topic modeling Nikolenko et al. (2017); Tong and Zhang (2016); Řehůřek and Sojka (2010). The idea behind LDA is that each document is a combination of several topics, while topics are represented by words. Thus, by finding the relationship between words and topics, each document can be clustered into topics based on the words in it. This simple and fast topic modeling technique shows great success in qualitative analysis. In our study, we also use LDA to conduct topic modeling and find out the most important aspects that workers feel uncomfortable with.

As antiwork ideation is most frequently implied in people’s discussions of workplace context, we pick five subreddits that have high relevance, including “r/antiwork”, “r/recruiting”, “r/recruitinghell”, “r/work”, and “r/jobs”. We obtain the raw data via Academic Torrents ¹¹1https://academictorrents.com/, consisting of a total of 329,830 posts and 400,000 comments (also referred to as “posts” for simplicity)²²2This work does not distinguish between Reddit posts and comments., with a time range of 03/27/2009 - 12/31/2022. The detailed data distribution among different subreddits is shown in Table 4 and Fig 6.

The raw datapoints we obtain remain post-level. We hence convert them into more structured, user-level datapoints by grouping the posts that belong to each user and sorting them in chronological order (see Fig 1). Moreover, we discard datapoints where either “selftext” or “title” field is missing.

We take a similar approach as De Choudhury et al. (2016) to create user groups {“antiwork”, “neutral”} - using subreddits as a proxy. Specifically, we decide that a user has an “antiwork” propensity if there exists a r/antiwork post (chronologically) after a post from “neutral subreddits” (“r/recruiting”, “r/recruitinghell”, “r/work”, and “r/jobs”); a user tends to be “neutral” if the condition above is not met and all the posts come from “neutral subreddits”. Similar to De Choudhury et al. (2016), this labeling schema serves as an efficient way of large-scale user modeling which would nevertheless induce noises. To mitigate the potential error of a user being labeled “neutral” but demonstrating “antiwork” emotion outside the “time periods” De Choudhury et al. (2016), in our approach we do not confine the time window for different subreddit posts to appear. We instead observe the whole time range and only focus on whether there exists a temporal order. We plot the schema of labeling in Fig 8. The post history (time normalized within [0,1]) for users from different classes is also visualized in Fig 7.

This ends up with 855 antiwork users and 83872 neutral users. To obtain a balanced class ratio and average post number of each class, we sample 1000 neutral users with Gaussian probabilistic function $\mathcal{N}(\mu_{\text{antiwork}},\sigma^{2}_{\text{antiwork}})(\cdot)$ on the average post number. Before training, we further split the data into training (0.75) and testing (0.25) sets. The final statistics are shown in Table 1.

We sanitize the “title” and “selftext” fields by replacing URLs and numbers with special tokens “url” and “@”), and emojis with their meanings (with the emoji.demojize package Kim and Wurster (2023)). For URLs and numbers, we manually examine them and verify that they do not contribute to contexts for understanding antiwork.

The training process for our model is set up as given the posts of a user, a label of this user is predicted. The models we experiment with include:

We further conduct separate analyses on the posts of different user classes with statistical methods.

While our objective is to uncover underlying causes of workplace dissatisfaction and employee unhappiness, we realize that our solution does have ethical implications. A classifier that can detect if a person is a member of the antiwork movement has the potential to be misused to stop strikes, fight against unions or falsely accuse workers of being against the company. To prevent this form of misuse, we will not be making the actual model or complete dataset available. We only share the anonymized antiwork labeled data to prevent training a classifier, and we do not share the trained classifier, but rather, just the word attribution utility. By doing so, we wish to prevent the misuse of this technology, since it can no longer classify individuals, but can be used to identify potential problems in the workplace. We also understand that although the posts used in our dataset are public, users may not be willing to have their personal information tracked and analyzed. To mitigate the problem, we have removed all personally identifiable information in the dataset and de-identified and paraphrased all the texts before presenting them. We encourage any further research involving this form of data to do the same.

In our work, we send RoBERTa-generated encodings to RNN to learn the difference between the posts sent by antiwork users and neutral users. The model achieves 80% overall accuracy, and performs well on both classes. Our study shows the great potential of LLM in social media analysis. As an emergent technology, LLM takes advantage of its large pre-train dataset and deep neural network, providing a fairly comprehensive understanding of natural language. Concatenated with classification layers, LLM would be extremely powerful in terms of feature extractions and predictions. This could be an effective model to process articles like social media posts on Reddit, and provide insights into them. We suggest future works to investigate the usage of LLM on more social phenomena analysis tasks like this.