An Ethical Highlighter for People-Centric
Dataset Creation

Creation encompasses the activities involved in producing a dataset, including sourcing, assembling, and cleaning data, as well as assigning labels. Typically, image datasets comprise images and their associated labels. These labels could be names of the people pictured, attributes (e.g., age), or potentially any other descriptions. More generally, we can think of labels as data in a textual modality associated with data in a visual modality (for now we exclude annotations like segmentation or depth maps). Our inductive analysis of this creation component revealed ethical concerns over violations of privacy and property rights, offensive data content, and subject autonomy.

In order to amass large numbers of images of people, some dataset creators have previously captured photos and videos without subjects’ knowledge or consent, two important aspects of privacy. For example, the creators of Duke MTMC [9] and Brainwash [10] were criticized for assembling datasets without notice or consent. Duke MTMC, created in 2014, comprised live footage of students on campus. In the same year, the Brainwash dataset used a webcam to capture images of customers in the San Francisco Brainwash cafe. In response to these criticisms, both datasets were ultimately taken down by their authors, respectively in May and June of 2019. The authors of Duke MTMC acknowledged that they had deviated from IRB-approved protocols by filming outdoors and releasing data without protections [11].

Even if dataset creators aim to capture faces in natural environments, they must grapple with the question of consent and determine methods to mitigate other privacy concerns. The creators of UnConstrained College Students (another dataset created on a college campus) suggested that some of its value was from featuring subjects that were “photographed [at long-range] without [students’] knowledge”, rather than being “posed” [12]. Images collected “in the wild” are considered to be particularly useful across a variety of naturalistic application domains. From the perspective of these creators, a lack of subject awareness (leading to a lack of consent) has been a feature, rather than a bug. It is also important to keep in mind that even when subjects give their consent to be included, they do not necessarily consent to all possible uses of a dataset—creators should be careful to specify the scope of the dataset’s purposes to subjects.

Another common method for gathering raw image data is to find and download images of people from the internet. For this method, it seems to be common practice to assume that “public persons” cede the right to privacy. For example, the creators of MS-Celeb-1M [13] started by assembling a list of 1M ostensible celebrities, selecting a subset of 100K identities and scraping their images from search engines. However, the dataset creators’ definition of “celebrity” was very broad—the original list was not limited to individuals who consider themselves as public figures, but was rather one million notable people on the internet, including a number of private persons. Consequently, when the dataset was released, it emerged that many faces were not those of celebrities, but instead of non-public persons (including vocal privacy advocates and journalists) who had not consented to having their faces included in the dataset.

By composition, we refer to properties of the dataset, spanning content (e.g., data units or elements comprising the dataset) such as visual images and text-based labels, mappings among elements expressed in different modalities (e.g., labels to images), and higher-order, macro attributes of the dataset such as demographic representativeness. Our analysis revealed that the composition of a dataset may be a source of ethical harms through, e.g., bias and unfairness, representational harm, and offensiveness.

Sources of offensive associations have been traced to labels used to generate ImageNet—namely, a database of words and semantic relations called WordNet [14]. In their analysis of the ImageNet subtree, Yang et al. (including ImageNet team members) found that of the 2,832 people categories within the subtree, 1,593 were potentially offensive categories [2]. Databases like Wordnet are often used to build datasets and are adopted across the industry. Other datasets built on WordNet, such as 80 Million Tiny Images [15], have similarly inherited offensive associations from WordNet. However, in this case, the $32\times 32$ images featured content that would be too small to manually perceive and audit [16]. Indeed, datasets like ImageNet have included images that are offensive or portray certain sub-populations in perjorative ways. In order to prevent such offensive images and associations, there is a need for more thorough auditing of both labels and imagery [17, 18].

Distribution is concerned with how creators make a dataset available, as well as that dataset”s terms of use and disclaimers. Our analysis revealed that the distribution of a dataset presents a source of ethical harms when it impedes accountability and violates subject autonomy.

This question of enforcement is important; creators need to consider the potential for misuse of their dataset and what they are in a position to enforce. As illustrated above, it is clear that disclaimers are important to have, but are not enforceable alone. Therefore, in the case of potentially sensitive data, dataset creators should consider reviewing requests on a case-by-case basis, putting forth a good faith effort to reject requests whose intent does not match the purpose of the dataset.

Many datasets do not clearly communicate their limitations nor how they should be used. We have seen some efforts to standardize documentation and increase dataset transparency, notably Datasheets for Datasets [6, 19], as well as an emerging culture of reflective practice to this end within academia and industry. In the past year, we have seen the addition of disclaimers to two existing datasets: the Labeled Faces in the Wild website was updated with a disclaimer about its potential lack of representation, and the VGGFace2 website similarly cautioned that their “distribution of identities… may not be representative of the global human population” and that users should “be careful of unintended societal, gender, racial and other biases”. While these examples indicate a shift towards greater transparency and acknowledgement of limitations in datasets, they also show that more work remains. Is it enough for dataset owners to qualify the use of datasets, or is it necessary to actively restrict access to and enforce the responsible use of datasets? What role do reflective and interrogatory processes (such as the framework we provide) have in leading to more responsible dataset practices?

Purpose answers the question, “Why?” Philosophers sometimes refer to this component as “teleology,” which involves explaining a phenomenon not in terms of what caused it but what motivated it; teleology covers a range of questions, e.g., what is the dataset for; what are its intended uses; for what purposes is it optimized?

Purpose is an incredibly rich source of ethical concern. A direct challenge to the moral legitimacy of a dataset’s own purpose is one such concern. For example, a detractor may assert that the purpose for creating a face dataset being to tell apart gay individuals does not meet a moral threshold [20]. Another way that purpose may stir up ethical concerns is in its relation to other characteristics of a dataset. One cannot overemphasize the potential for ethical discord that may follow. A typical instance is that of a face dataset, not specifically optimized for facial recognition, but used for this purpose. Though it may serve well in some capacities (e.g., face synthesis), it may result in bias, or representational harm to certain minority groups, when used to train a recognition algorithm [21, 22, 23]. Similarly, creators of face datasets considering policies for distributing or providing access to them would want to understand how these policies would apply in relation to certain purposes (e.g., as tools for surveilling subpopulations [24]).

Setting aside cases where a purpose itself is deemed morally reprehensible, purpose (or teleology) stands apart from the other components in that it is frequently relational in nature. By this, we mean that ethical issues emerge from mappings—between purpose and properties of creation, distribution, or composition. By implication, fastidious users of our framework will not consider their work complete until they perform a systematic pass of the other three components with a clear sense of a dataset’s teleology.