Enclosed Loops: How open source communities become datasets

Dependabot is a tool that maintains source code dependencies and mitigates security vulnerabilities in the software supply chain (Github, 2023). Founded in 2017, Dependabot was acquired by GitHub in 2019, and has since been fully integrated into GitHub’s platform (He et al., 2022). Further, Dependabot alerts are on by default for public repositories on GitHub (Github, 2023; McDonald, 2021). Security Advisories are synchronized from the National Vulnerability Database(national institute of Standards and Security, 2023), and repository owners can also raise security vulnerabilities in their code. If an advisory falls within one of GitHub’s supported ecosystems, the advisory gets verified by the GitHub Security Team. While other dependency management bots exist, Dependabot’s integration on GitHub makes it the most widely used. In 2019, 67% of bot-created pull requests came from the original and GitHub native versions of Dependabot (Wyrich et al., 2021).

The rust programming language makes an aggressive commitment to stability across updates to the language and compiler (Turon and Madsakis, 2014; Matsakis, 2014). Crater, originally called taskcluster-crater was introduced in 2015 as a tool to help guarantee stability by “compiling and running tests for every crate on crates.io (and a few on GitHub) (Team, 2015). Currently, there are 103,318 crates on crates.io (2023-01-29). Additionally, every single public repository on GitHub with a Cargo.lock file(Bos, 2022) is tested. Brian Anderson developed the initial version of Crater as “a tool to run experiments across parts of the Rust ecosystem.” (Team, 2015). Crater runs weekly or more (Bos, 2022). When new errors cause the compilation to fail, Rust’s compiler team may revert changes to the compiler.

Copilot is a cloud-based “artificial intelligence” assistant for writing code. Copilot is installed as a plugin compatible with a number of code editors, where it acts as a form of advanced auto-complete. It is currently based on OpenAI’s Codex model(Chen et al., 2021). Codex is a fully trained GPT3 model fine-tuned on 54 million public software repositories hosted on GitHub, filtered down to 159GB of Python source code(Chen et al., 2021). It is unclear what data the deployed model was fine-tuned on, likely much more. More than 1.2 million developers joined Copilot’s free technical preview in 2021. Within 1 month of moving to a $10/month subscription model, 400,000 users signed up(Iversen et al., 2022).

Within open source ecosystems, common wisdom suggests that by keeping individual packages up to date and security holes patched, the ecosystem as a whole is made safer (Okafor et al., 2022). Dependabot attempts to keep users’ code secure by helping keep everyone’s code secure and up to date.

In describing all the ways the Rust language is tested to maintain stability and safety one maintainer described how the existence and centralization of Github and Crates.io “allow us to treat the entire world of open source Rust code as our test suite” (Anderson, 2017). Those tests assist Rust language developers in maintaining the state of the compiler while keeping updates to the language timely. By allocating time and compute toward such a goal, they are demonstrating their commitment to stability(Turon and Madsakis, 2014) for all users of the language, within the open source and outside of it.

Copilot’s stated goal is to help developers focus on “bigger” problems, leaving the tedious parts of coding to an automated system (Inc., 2022). Copilot’s unstated goal is also to be a subscription-based product for GitHub to derive a profit (Iversen et al., 2022). Like nearly all social platforms from the 2010s, the free hosting of content is a means financial gain; this is no different for GitHub. Furthermore, Copilot collects telemetry data as discussed in section 3.4. This serves the other two goals by acting as a data flywheel (Larsson, 2021; Xia and Zhang, 2023) to improve future versions of Copilot making the tool more attractive as a product.

GitHub scans user code and crosschecks it against a shared database of CVEs (Mann and Christey, 1999). IN addition, GitHub is a numbering authority capable of adding vulnerabilities to that shared database. Dependabot is primarily an interface to that database. The code for the bot is open-sourced but individuals cannot add to the database. They can only make suggestions that are reviewed by the GitHub security team.

The rust language centralizes its own package management creating a data source for Crater to run on. Additional packages can be sourced from GitHub because of its dominance over code hosting. Crater currently runs on an AWS c5.2xlarge with 2 terabytes of storage(Team, 2015). This costs $0.34/hour with runs taking a few days. The servers used by crater are restricted to certain rust team members, however the costs are nominal and individuals could presumably implement their own system.

Without the centralization of GitHub, collecting the massive datasets necessary for training a Codex-like model would likely be much more difficult. GitHub already hosts the code, it is technologically trivial for them to use it as they wish. Additionally, if data-set size is to be the bottleneck in improvements in transformer-like models(Hoffmann et al., 2022) it is necessary to ingest maximal amounts of source code regardless of licensing or authors preferences. Training large models is financially costly, while those numbers are not released based on Allal et al. (2023) it could be  $40-$60,000 if compute is priced similarly to AWS EC2 P3 instances. There would be significant costs to developing and running the infrastructure for running inference as well. The tools existence and underlying technology cannot be cleaved from the social dimensions of its development (Whittaker, 2021). Copilot and similar models are fundamentally products of large well funded institutions.

Released under the prosperity public license, dependabot’s source code is available online as is the database alerts are based on. The tool is self-activating and on by default for public repositories. The license creates strong limitations for commercial use by a offering a restricted trial for such uses. Dependabot does not create copyright infringement or risks as its operational function to maintain the sustainability of a given repository. Dependabot’s technical processes would be classified as transformational use in that its use enhances the copyrighted code towards the goal of protecting packages from security risks.

Crater is licensed under both the MIT and Apache 2.0 licenses. The interests of the programmers who publish Rust code to crates.io and GitHub don’t have expressive interests that could be violated by its reuse. Because these are public-facing repositories, regression testing doesn’t violate terms of use as defined by the licenses. Further, this interest does not produce copyright issues from a fair-use perspective as it does not reduce the market value of the code that is posted. Instead, the code is made more valuable by ensuring that it continues to work. The Rust maintainers make noncommercial use of packages available online, and the functional processes Crater carries out are transformative. From a community perspective, there are no normative compelling reasons why a programmer would object to their code being scraped. Testing and stability do not invade an interest in personhood, nor do they create losses to the developer. They do not create unearned benefits to other parties. Rather, testing using code at scale benefits the community of which the programmer is part. It doesn’t interfere with their authorship interest in the expression in the code; it doesn’t interfere with their incentives to create it.

Whereas licenses enable others to obtain rights to copy and use software, terms of service agreements are a legal contract that outline a service relationship between a customer and a corporation. The Copilot case presents a copyright conundrum: a code recommendation system that generates copyright infringing works. However, there can be no secondary copyright liability without primary infringement, which means that Copilot is not itself liable unless it is used to infringe. Under U.S. copyright law, there are no copyrightable interests in infringing works. Therefore, Copilot as a recommendation tool that copies code produces a significant source of risk to its users. Further, Copilot is trained on code from repositories on GitHub, released under numerous open source licences but provides no attribution information. Arguably, contradicting its own normative standards for sharing code. GitHub itself admits that the text of the GPL was contained in the training data 700,000 times (Ziegler, 2022). GitHub executives also argue that the tool satisfies fair use because the output belongs to the operator.

Ongoing litigation will determine the legality of laundering open source to sell as a subscription. Regardless of legality, a number of projects have left GitHub for at least violating the spirit of their license and their trust. One complaint asks why the models treat open source code as a free commodity but were not also trained on the proprietary code of GitHub and its owner Microsoft (M. Kuhn, 2022) if there are no copyright concerns with the system.

Users interact with Dependabot when alerts are raised in their repository. Those alerts are only visible to maintainers otherwise they would risk outing vulnerable projects. While alerts are on by default for public repositories, users can opt out and turn alerts off. Users can also further engage with Dependabot by raising vulnerabilities, in their own packages or others. These features are well documented on GitHub. The safer each individual package is the safer the broader open source ecosystem is describes a feedback loop between users individual security concerns and the security of all packages.

There is relatively little user interaction with crater by normal member’s of the community. It is unclear how widespread its existence is to casual users of the programming language. While all code on crates.io is subject to crater experiments users could avoid publishing their code to the package manager. Similarly, only projects on GitHub with a Cargo.lock file are scraped and included in Crater tests. If those files are never pushed to a public repository they will be opted out of crater runs. Though users may not interact with crater directly there exists a positive feedback loop between the open source ecosystem and the tool. Stability is a core commitment by the maintainers to the rust community (Turon and Madsakis, 2014) and a major draw to users (Zeng and Crichton, 2019) The better the Rust Foundation can keep that commitment the stronger the draw of the language, the more the ecosystem produces, providing more data for Crater to function. Even beyond the broad feedback loop, one rust maintainer described in some cases where code tested by Crater was itself unsound “ [they] often inform the crate[package] maintainer and sometimes even help them fix it” (Bos, 2022).

Of the tools listed Copilot has the tightest interaction loop between users and the least transparency in a feedback loop between the community and the tool. By implementing it as a plugin for a wide variety of code editors it adds functionality to the context where developers are comfortable working(Barke et al., 2022). Copilot leverages textual norms of the medium of code to emphasize the “pair programming” modality (Bird et al., 2022). Users report spending less time searching online for answers at the cost of understanding their own code less (Bird et al., 2022). Github does collect telemetry data (though users can opt out) on which code suggestions have been accepted, edited, or rejected.

While these may be used to update the underlying model, feedback loops between the underlying model and the open source platform are limited temporally to updates to the underlying model. The authors of the Codex paper note that“the model may make suggestions for deprecated methods. This could increase open-source developers’ incentive to maintain backward compatibility, which could pose challenges given that open-source projects are often under-resourced”. One of the key innovations of working in public was the rapid iteration cycle that allows productive feedback loops to develop. Copilot is unlikely to be updated fast enough to produce those loops.

A significant number of users pushing Codex-like generated code to public repositories reinforces the use of popular languages and open source packages. The long term effect entrenches the disparities in package usage rates. The authors of Codex note how differential import rates might:

increase the dominance of an already influential set of individuals and organizations in the software supply chain. … Where a user may have done an Internet search before for ’which machine learning package to use’ or ’pros and cons of PyTorch vs. Tensorflow’ they might now just type ’# import machine learning package’ and trust Codex to do the rest. Users might be more inclined to accept the Codex answer under the assumption that the package it suggests is the one with which Codex will be more helpful. As a result, certain players might become more entrenched in the package market and Codex might not be aware of new packages developed after the training data was originally gathered.

At a sufficient scale of users this may hinder newer projects and libraries from gaining a foothold and developing their own community. An algorithmic and computational mono-culture.