ViSAGe: A Global-Scale Analysis of Visual Stereotypes in
Text-to-Image Generation

The annotators are presented with an attribute and a statement of the form of ‘The attribute [attr] can be visually depicted in an image.’ The attribute presented in the sentence is selected from a list of all unique attributes (noun/noun phrases, and adjective/adjective phrases) from SeeGULL. The annotators are required to assign a Likert scale rating to each attribute indicating the extent to which they agree or disagree with the above statement w.r.t. to the given attribute term. We note that the visual nature of attributes lie on a spectrum. While there may be clearly defined attributes on either extremes, the ‘visual’ and ‘non-visual’ attributes in the middle are more subjective. For our subsequent analysis, we exclude all attributes where any annotator expressed uncertainty or disagreement regarding the visual nature. Consequently, we deem terms where all annotators at least agreed about their visual nature, as ‘visual’ resulting in a selection of 385 out of the original 1994 attributes. Please refer to the Appendix 7.1.1 for details. We recruited annotators based out of the United States, and proficient in English reading and writing. For each attribute, we get annotations from 3 annotators that identify with different geographical origin identities - Asian, European, and North American. Annotators were professional data labelers and were compensated at rates above the prevalent market rates.

Using the identified visual attributes (Appendix 7.1.1), we filter out the associated ‘visual stereotypes’ in SeeGULL. Figure  2 shows the global distribution of visual attributes. It also contains examples of visual stereotypes associated with different countries. We observe that around 30 identities, including Omani, Ukranian, Swiss, Canadian, Mongolian, etc., just have a single visual attribute associated with them. Australians (63) have the highest number of visual attributes associated with them followed by Mexicans (46), Indians (34), New Zealand (31), Ethiopians (27), and Japanese (20). The numbers in bracket indicate the number of visual attributes associated with the identity group in SeeGULL. We use these visual stereotypes for identifying their prevalence in images in the next step.

The annotation task aims to examine whether well-known visual stereotypes about identity groups are reflected in the generated images. We generate 15 images per identity group and present it to the annotators. Each image is accompanied by two sets of attributes: (i) a set of visual attributes stereotypically associated with the identity group (Figure 2), and (ii) an equal number of randomly selected visual non-stereotypical attributes, i.e., attributes that are not stereotypically associated with the identity group. We combine the two sets and randomly show 5 attributes in succession alongside each image, until every attribute has been covered. An additional option of ‘None of the above’ is also displayed with each set. The annotators are asked to select all attribute(s) that they believe are visually depicted in the image. Additionally, they are also asked to draw bounding boxes to highlight specific regions, objects, or other indicators that support their selection of the visual attribute within the image. If annotators believe that no attributes are visually represented, they can choose ‘None of the above.’ Overall, we get annotations for 2,025 identity-image pairs and 40,057 image-attribute pairs. Similar to  3.1, we recruited annotators proficient in English reading and writing, residing in United States but identifying with different nationalities. We get annotations for each attribute-image pair from 3 annotators.

Annotating stereotypes in images at scale can often times be resource-intensive and time-consuming. Therefore, we also investigate the feasibility of using automated techniques for stereotype detection in images. We use the already existing CLIP (Contrastive Language-Image Pre-training) embeddings combined with the BART model to generate image captions. An image is mapped into the CLIP embedding space, and the embeddings are then scored against a cache of n-grams. The top-k embeddings are then sent to BART to generate candidate captions. We get top 50 captions for each image for the same set of the 15 images per identity group previously generated. Building on the already identified visual stereotypes, we identify the stereotypes in captions by performing a string match. To further understand how uniquely a stereotypical attribute $attr_{s}$ is present in the caption of the images of an identity group, we compute a salience score of the attributes w.r.t. the identity group $S(attr_{s},id)$. We use a modified tf-idf metric as follows: $S(attr_{s},id)=tf(attr_{s},id)\cdot idf(attr_{s},C)$. The function $tf(attr_{s},id)$ represents the smoothed relative frequency of attribute $attr_{s}$ for the identity group $id$; and the function $idf(attr_{s},C)$ denotes the inverse document frequency of the attribute term $attr_{s}$ in all the captions ‘C’, reflecting the importance of the attribute across all the captions. A higher salience score of an attribute, indicates a more unique association of the attribute with the identity group. For each identity group, we then extract the most salient associated visual stereotypes as present in the captions.

We use the identified visual stereotypes for evaluating whether already known stereotypical attributes are visually represented in the images of different identity groups. We only consider identity groups that have more than one associated visual stereotypes for our analysis. We find the likelihood $\mathbb{L}(attr_{s},id)$ of a stereotypical attribute $attr_{s}$ being present in any image of the identity group $id$ as follows. If a stereotypical attribute $attr_{s}$ was shown $n$ times per identity group across all 15 images and selected $k$ times by the annotators, then $\mathbb{L}(attr_{s},id)=\frac{k}{n}$. Figure 3 presents examples of the most likely stereotypes associated with certain identity groups as annotated by different annotators. We observe that the attributes with the highest likelihood for an identity group, also align with the known stereotypes present in SeeGULL. For example, ‘dark’, ‘thin’, ‘skinny’, and ‘underweight’, the most likely attributes depicting Sudanese individuals, are also known stereotypes in SeeGULL. The attribute ‘poor’ which has a likelihood measure of 0.5 for Bangladeshi, i.e., approximately 50% of images representing Bangladeshis contained a representation of ’poor’, is also an annotated stereotype in SeeGULL. Similarly, attributes ‘beaner’, ‘brown’, ‘festive, ‘dark skin’, and ‘sombrero’ are stereotypes in SeeGULL for Mexicans, and are also present in the images representing a Mexican person. An Indian person is often represented as ‘dark, ‘brown’, and ‘religious’ which are its associated visual stereotypes in SeeGULL. We note that the terms ‘brown’, ‘black’ etc., in this work, as identified in images do not imply race, and instead are about appearance, or appearance-based observed race Hanna et al. (2020); Schumann et al. (2023), or even the colors themselves.

It is crucial to understand if representations of some identity groups tend to be more stereotypical than others. To analyze this better, we compute the ‘stereotypical tendency’ $\theta_{id}$ for each identity group. We measure the mean likelihood of a stereotype being present in any image representing an identity group $\mathbb{L}({stereo},id)$ and compare it with the likelihood of a randomly selected non-stereotypical attribute being depicted visually in the same set of images $\mathbb{L}({random},id)$.

Let $\mathbb{L}(attr_{s},id)$ denote the likelihood of a stereotypical attribute being present in an image, and $\mathbb{L}(attr_{r},id)$ denote the likelihood of a randomly selected non-stereotypical attribute being present in the image corresponding to an identity group $id$. We select an equal number, $k$, of stereotypical and random attributes for any given identity group for a fair comparison. The likelihood of an image being stereotypical for a given identity group is, then, denoted as $\mathbb{L}({stereo},id)=\frac{1}{k}\sum_{i=1}^{k}\mathbb{L}({attr}_{s}^{i},{id})$, where $i$ denotes the $i$-th stereotypical attribute associated with the identity group. Similarly, the likelihood of a random attribute being present in the image of the identity group is given by $\mathbb{L}({random},id)=\frac{1}{k}\sum_{j=1}^{k}\mathbb{L}({attr}_{r}^{j},{id})$, where $j$ denotes $j$-th attribute selected randomly with the identity group. Computing the ratio between the two for an identity group

$$\theta_{id}=\frac{\mathbb{L}(stereo,id)}{\mathbb{L}({random},id)}$$

gives the ‘stereotypical tendency’ of the generated images. Higher the ratio, greater is the likelihood of its stereotypical representation.

We observe that on average, the visual representation of any identity group is thrice as likely to be stereotypical than non-stereotypical, i.e., the visual stereotypical attributes associated with an identity group are thrice as likely to appear in their visual representation when compared to randomly selected visual non-stereotypical attributes. We also compute $\theta_{id}$ for all 135 identity groups. We observe that images representing Togolese, Zimbabwean, Swedes, Danish, etc., only contained stereotypical attributes when compared to random attributes, i.e., $\theta_{id}$ is infinite or N/A; whereas images representing Nigerians were 27 times more likely to contain stereotypical attributes than randomly selected non-stereotypes. (Please refer to the Tables 1 and 2 in the Appendix for all scores).

Visual representation of some stereotypes can be more offensive than others. For example, the stereotypical depiction of an identity group as ‘poor’ can be considered more offensive when compared to the visual depiction of a stereotype ‘rich’. We investigate the offensiveness of images and whether certain identity groups have a more offensive representation compared to others.

Using the offensiveness rating of stereotypical attributes in SeeGULL, we infer an overall offensiveness score $O(id)$ for representation of identity groups. Let ${O}(attr_{s},id)$ denote the offensiveness score of a stereotypical attribute $attr_{s}$ associated with an identity group $id$ in SeeGULL. The mean offensiveness score for each identity group is then

$$O(id)=\frac{1}{n}\mathbb{L}(stereotype,id)\cdot\sum_{i=1}^{n}{O}(attr_{s}^{i},id)$$

where $i$ denotes the $i$-th stereotypical attribute $attr_{s}$ identified as being present in the image representing an identity group by the annotators. Figure 8 visualizes the normalized offensiveness score for different identity groups. We observe that the representations of people from countries in Africa, South America, and South East Asia, are comparatively more offensive. Jordanians, Uruguayans, Gabonese, Laotian, and Albanians have the most offensive representation; whereas Australians, Swedes, Danish, Norwegians, and Nepalese have the least offensive representation.