Throwaway Accounts and Moderation on Reddit

People use different types of identity on SMPs, ranging from real names to anonymity. SMPs like Facebook have a strict real-name policy in their terms and conditions that require users to use their authentic names. Although this real-name policy is controversial and some users use fake names and/or identities to get around it [17], Facebook actively detects fake accounts using algorithms and encourages its users to report them [18]. On the other hand, many other SMPs, including Reddit, rely on a pseudonymity model where users create a screen name when they access the sites. Still, other SMPs (e.g., 4chan) use an anonymity identity policy. In this model, all posts are anonymous in the sense that they are not associated with a user-created screen name. A common limitation across these models is the lack of flexibility in identity choice. For instance, Facebook exclusively offers real names, preventing pseudonymous or anonymous participation. This inflexibility may not align with user preferences, as individuals may desire different levels of identity disclosure depending on the context or content sensitivity [19].

Reddit does not provide a formal anonymity feature that allows users to use it anonymously. However, we observe that users still have the desire to engage on Reddit anonymously in certain situations. The community norm, in this case, is for Reddit users to engage in a workaround approach where they use a throwaway account. Throwaway accounts on Reddit are temporary user accounts used only for a specific time and purpose. Reddit users use throwaway accounts to share secrets or discuss sensitive information without being identified and with the ability to walk away from further discussion [20]. As a result, many throwaway accounts are only used once [20], and almost all (96.3%) throwaway accounts on Reddit are used three or fewer times [1].

Marx argues that “to be fully anonymous means that a person cannot be identified according to any of the seven dimensions of identity knowledge.” [21, pg. 100] Those dimensions are: 1) legal name 2) locatability 3) pseudonyms that can be linked to legal name and/or locatability–literally a form of pseudonymity 4) pseudonyms that cannot be linked to other forms of identity knowledge–the equivalent of “real” anonymity (except that the name chosen may hint at some aspects of “real” identity, as with undercover agents encouraged to take names close to their own) 5) pattern knowledge 6) social categorization 7) symbols of eligibility/non-eligibility. Marx further argues that we have true anonymity only when “no aspects of identity are available (being uncollected, altered, or severed).” [22, pg. 104]

Absolute anonymity on the Internet is difficult to achieve, even for experts, and some argue, may not even be possible [23]. Although throwaway accounts on Reddit do not perfectly fit the dimensions of anonymity Marx describes (people may still self-disclose in their posts/comments when using throwaway accounts), prior work [1, 20, 24] suggests that throwaway accounts can serve as proxies for true anonymity, as they provide an acceptable level of anonymity. Thus, following prior studies (e.g.,  [3, 4, 25], we use the terms “identified” and “anonymous” to distinguish between ordinary accounts and throwaway accounts on Reddit.

Our work is not the only work that investigates throwaway accounts on Reddit. For example, other work finds that factors including the perception of anonymity and gender affect the ways people use throwaway accounts on Reddit [1]. Within subreddits that focus on marginalized groups (e.g., Asian Americans and Pacific Islanders focused subreddits), conservative people are more likely to post using throwaway accounts [26]. Using an anonymous identity via a throwaway account can bring certain benefits, such as enabling users to express views and thoughts more freely, especially about sensitive topics such as mental health [27]. Furthermore, the use of throwaway accounts does not hinder the quality of social support users receive from other Redditors [27]. Similarly, throwaway accounts allow users to share intimate content but are not associated with aggressive or unsupportive comments [4]. On Reddit, comments posted using throwaway accounts are more likely to receive a response, receive longer responses, and receive responses with higher Karma scores (users get Karma if their content gets upvoted) than those posted using identified accounts [28].

Besides all these benefits, however, online anonymity also has drawbacks, such as resulting in a lack of consequences for deviant behavior [29]. Deviant behavior such as trolling is common on SMPs such as Reddit [30]. According to the social identity theory of deindividuation [31], people in groups (e.g., an SMP such as Reddit) tend to lose selfhood and thus lose self-control over behavior. In this theory, anonymity could lead to this loss of control. This loss of control could then lead to deviant behavior, which may violate the community norms that are likely to be targets of moderation on Reddit, and are likely to be removed by moderation  [32]. While anonymous editors on Wikipedia are more prone to violating policies related to edit warring, particularly on discussion pages [33], Reddit users engage with the platform for a wider range of activities beyond information sharing and editing [34, 35]. So far, we don’t yet know if the use of throwaway accounts on Reddit will have any relationship with content removal by moderation. Thus, we extend prior research in this area by asking RQ1.

On SMPs, content moderation fits into two categories: distributed moderation and centralized moderation [32]. Distributed moderation is a moderation mechanism that distributes moderation tasks to site users. For example, many social Q&A sites use a voting system that users can upvote and downvote each others’ Q&As [36, 37]. Then the voting outcome “determines how prominently any content is displayed on the site”, which “allows the community to collectively decide its threshold for what content is acceptable and which issues need to be articulated and discussed.” [32]

Unlike distributed moderation which allows users to conduct moderation activities, centralized moderation uses moderators to handle moderation tasks. For example, SMPs like YouTube actively recruit human moderators to do moderation tasks on their sites [7]. Reddit adopts both distributed moderation and centralized moderation approaches. On the one hand, users can upvote and downvote other users’ posts/comments. This voting system influences the visibility and prominence of content on the platform’s interface. On the other hand, there are a certain number of users that voluntarily serve as human moderators for each subreddit. On Reddit, only centralized moderation will lead to content removal. The distributed moderation (users’ votes) will not directly lead to content removal. Since our research questions are around content moderation that leads to content removal, in this work, we focus on the latter–centralized moderation on Reddit.

Besides human moderators, Reddit also enables moderation bots⁴⁴4https://www.reddit.com/r/TheoryOfReddit/wiki/bots/, which are popular as a class of automated moderation tools supporting Reddit moderators [38]. Among these moderation bots, the most widely used one is named “AutoModerator” [39]. AutoModerator is an open-sourced site-wide tool set up by human moderators to facilitate the moderation tasks, especially repetitive ones [39]. Human moderators can also develop bots using Reddit’s API to perform similar and/or customized tasks. For both moderation bots and human moderators, there is a variety of list of moderation actions they could do on Reddit⁵⁵5https://support.reddithelp.com/hc/articles/15484284113172. These moderation actions could lead to different outcomes. Some of them might lead to the removal of user content. TeBlunthuis, Hill, and Halfaker found out that on SMPs like Wikipedia, human moderators can focus on social signals but overlook the actual misbehavior [40]. They also found that utilizing algorithmic flagging (which is similar to the moderation bots on Reddit) can reduce such bias and increase moderation fairness. Likewise, on Reddit, human moderators can focus on social cues like the usernames rather than the actual content users post. It is unclear that if the content posted by throwaway accounts is more likely to be removed by moderation by human moderators than by moderation bots. Thus, we ask RQ2.

To answer both of our research questions, we collected a subset of moderated and removed content. Using publicmodlogs and Reddit’s API, we collected the most recent 500 moderation logs, which contained moderation actions for both posts and comments of all 340 subreddits in mid-October of 2019. We wrote a Python JSON parser to parse the JSON objects returned by Reddit’s API. Using this approach, we were able to collect the following information about each moderation log: the moderation ID, date and time the moderation happened, the moderation action, the name of moderators, the name of users whose post/comment was moderated, and the content of each post/comment. We collected 36,514 moderation logs in total. The dates of these logs ranged from mid-July to mid-October of 2019.

We additionally collected a subset of unremoved content. Using the Pushshift API⁶⁶6https://pushshift.io, we collected all the unremoved posts and comments of all 340 subreddits within the period of those moderated logs (mid-July, 2019 to mid-October, 2019). We wrote a Python JSON parser to parse the JSON objects returned by the Pushshift API. Pushshift API stores comments and posts from Reddit in real-time as long as they are not removed by moderation bots (e.g., AutoModerator, the moderation of which occurs immediately without delay⁷⁷7https://support.reddithelp.com/hc/articles/15484574206484). This means those posts and comments that human moderators eventually removed are also included in Pushshift. For this content, the original text returned by the API will be replaced with [“removed”] [32]. Since we already collected those comments using publicmodlogs when we collected moderated content, we removed these duplicates from our dataset in this step. Since not all moderation actions lead to content removal, we also filtered the duplicates in this step. Using this approach, we were able to collect the following information on the unremoved content: unremoved post/comments, the author of each post/comment, date and time, and the content of each post/comment. As a result, we collected 527,763 unremoved posts and comments in total.

People want to be anonymous online, including on SMPs, for many reasons. First, people’s past experiences and life situations lead them to choose to be anonymous online [5]. In the same study, 93% of the participants have the experience of being anonymous for social activities, and 57% have participated in special interest groups anonymously. Being anonymous online, especially when interacting with highly sensitive content (e.g., highly sensitive subreddits on Reddit), may provide people more control over personal information disclosure, protect their personal safety, provide more emotional benefits, and make them feel free to express opinions [5]. On Reddit, people may use throwaway accounts, especially in highly sensitive subreddits, to look for information or social support. For example, in the subreddits related to sex abuse on Reddit, “anonymous commenting enables commenters to share intimate content such as reciprocal disclosures and supportive messages” [4]. People also feel more comfortable disclosing highly sensitive information (e.g., pregnancy loss) on identified SMPs (e.g., Facebook) after participation anonymously on Reddit [56]. SMPs like Reddit that allow throwaway accounts should make it easy for users to switch between identities without logging in and out. SMPs like Reddit may also consider using other approaches to let users engage anonymously more easily. Currently, Reddit users have to use some sort of a workaround (e.g., creating throwaway accounts) to stay anonymous. This might discourage or prevent users from engaging with SMPs. For example, people who are less knowledgeable about SMPs may not be aware of how to create a throwaway account. Prior work shows that people who have concerns about how social media can adversely affect their relationships with others will use SMPs less frequently [57]. The use of throwaway accounts/anonymous accounts could reduce these concerns.

On the other hand, there are also downsides to allowing anonymity online. Friedman and Resnick [58] highlighted the issue of “cheap pseudonyms”, where the ease of creating new online identities with minimal effort or cost can encourage misbehavior without the risk of harming one’s reputation. This issue may be even more prevalent on SMPs, where obtaining new identities often requires just a few simple registration steps. For instance, studies have shown that removing the option for anonymous participation can improve the quality of comments on online news sites [59] and online communities of practice [60].

Similarly, in our study, we find that content posted by throwaway accounts is more likely to be removed. The online disinhibition theory describes a phenomenon where people may have a lack of restraint on the Internet, which may lead to toxic behavior [61]. In this theory, anonymity is one of the factors that could lead to toxic disinhibition. Similarly, in the social identity theory of deindividuation [31], anonymity is one of the factors that could lead people to lose self-control over behavior in groups (e.g., SMPs like Reddit). Based on these theories and our empirical study, moderators need to exercise heightened vigilance toward throwaway accounts or anonymous users, as theoretically, people may use throwaway accounts or anonymous identities deliberately to misbehave or troll on SMPs. However, the solution is not to disallow throwaway accounts and anonymous identity on SMPs as this would make it very difficult for people who have privacy needs and want to seek information online, especially around highly sensitive topics. While this study does not definitively resolve the ongoing debate regarding throwaway accounts and anonymous identities, we propose that a robust and effectively implemented moderation mechanism may offer a viable solution. In an ideal scenario, moderation can allow users with privacy needs to participate anonymously while ensuring the timely removal of harmful content and dissuade users who deliberately misuse anonymity.

Many users do not perceive the removal of their content by moderation as justified [62]. Additionally, aside from publicmodlogs, Reddit’s moderation system lacks transparency [41]. However, in our study, we find that content removal decisions made by human moderators on Reddit are generally fair. Unlike the moderation bots, which have pre-defined rules and strictly enforce them, the moderation work by human moderators is more flexible. Although different moderators may take different moderation actions, especially in grey areas, human moderators are well aware of their tasks’ subjective nature [63]. Our results reveal that most of the moderation actions are still done by human moderators on Reddit, especially for the moderation actions that are not just removing a post or a comment.

Human moderation has human costs [15, 16], including that it is time-consuming and takes a psychological toll. For example, 40% of the participants (journalists and human rights workers) from a survey study reported that “viewing distressing eyewitness media has had a negative impact on their personal lives” [64]. As a human moderator, someone may spend hours each day viewing those distressing eyewitness media to support their online communities. Moreover, on Reddit, human moderators all voluntarily take on moderation tasks. On other SMPs such as Facebook, human moderators are paid, but the compensation is low and does not stop human moderators from presenting with PTSD-like symptoms [65]. All of these could lead to the grievances of human moderators: “the work of moderators and the precarity of their position with company policies” [66]. The grievances of human moderators may cause other further issues, such as the Reddit Blackout of July 2015 [67]. During the Reddit Blackout of July 2015, human moderators of NSFW (highly sensitive) subreddits were more likely to blackout than human moderators of SFW (less sensitive) subreddits [66]. Human moderators of highly sensitive subreddits are probably seeing more distressing eyewitness media than human moderators of less sensitive subreddits, which may cause more human costs such as more severe PTSD-like symptoms or grievances.

Another aspect of cost is that doing more moderation work may increase human moderators’ bias regarding the content they encounter. For example, trolling, which “includes flaming, griefing, swearing, or personal attacks, including behavior outside the acceptable bounds defined by several community guidelines for discussion forums” [6], is a common behavior on SMPs including Reddit [68]. Trolling, by definition, violates the community norms on Reddit [32], will very likely to be removed by moderation. Human moderators, who see a lot of trolling posts on a regular basis, may also be susceptible. Seeing trolling posts could cause their own misuse of authority (e.g., retaliation), although this seems dependent on personality traits [69]. This could potentially increase the bias of human moderation.

Volunteer governance continues to be the common approach to managing social relations (e.g., human moderation) in online communities [70]. Besides the positive effects of shielding a community from undesirable content, the content removal can also help to improve the comment’s author’s future behavior [71]. However, as we discussed above, human moderation has human costs. Alternative moderation approaches such as the moderation bots on Reddit could participate more in content moderation to reduce the dependency on human moderation. Currently, on Reddit, the most popular moderation bot - AutoModerator, can only “remove or flair posts by domain or keyword”.¹¹¹¹11https://support.reddithelp.com/hc/articles/15484574206484 Although we find that more active subreddits are more likely to have moderation bots remove content, the human labor of moderation is necessary [72]. It is challenging to completely replace human moderators with moderation bots due to the complexity of language and community norms. Especially for grey areas, human moderators might be more flexible than moderation bots. In fact, we find that more sensitive subreddits are more likely to have human moderators remove content. However, we argue that SMPs should take better care of human moderators and develop effective tools to support human moderators for content moderation. For example, an AI-backed sociotechnical moderation system can already detect close to 90% of the comments that would be removed by human moderators [73], which may also help with the gray areas. It can significantly reduce the costs to and of human moderators. But, humans need to remain in the loop to ensure moderation bots are helping and not removing content that human moderators would want to remain.