Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications

The first step of our system is to recognize UI elements in a given UI screenshot. In our work, we use Faster-RCNN (Ren et al., 2015) to localize and recognize UI elements in UIs, and then merges the detection in this step with the results from Text Content Extraction module to form more accurate detections.

To extract text content in UI, we adopt a mature OCR model, PaddleOCR (developers, 2022), which supports over 80 language, and achieves high accuracy. We use their pretrained model for English language, and obtain the bounding boxes of text lines and text contents in the UI screenshot. Instead of inputting the detected UI element from the previous step one by one, we directly feed the whole UI screenshot to the OCR engine to save computing time and to retrieve miss-detected text elements. After we obtain the text lines (coodinates and text contents), we merges these text detections with the detections from FasterRCNN.

Merging with FasterRCNN detections: UI elements normally do not overlap with each other except one case when an ImageView element is served as a background of other elements. Therefore, we first sort the UI elements by their size in ascending order. For each UI element, we calculate the interaction over union (IoU - $interactionarea/unioinarea$) values with each detected text line. If the IoU value over a predefined threshold, we consider it as a match, and add the corresponding text content to the UI element. As one UI element may contain several text lines, we allow one UI element to match with several text lines. However, one text line can only be matched with one UI element. After we iterate all UI element, the unmatched text lines will be considered as TextView elements that is missed by our UI Element Detection module.

In addition, we notice that the bounding boxes of text elements from UI Element Detection are not very precise while OCR engines can extract highly precise bounding boxes of the text lines. Leveraging this feature, we further refine the detected bounding boxes by replace the bounding boxes of UI text elements with the merged bounding box of the matched text lines.

Understanding the semantics of icons are important to recognise the existence of dark patterns. For example, star icons will appear in the Nagging patterns, and the triangle Ad information icons will always appear in ad-related dark patterns. Hence, for UI elements detected as ImageView or ImageButton elements, we trained a ResNet-18 model (He et al., 2016) to classify the icons into finer semantic categories.

In addition, we notice that some advertisement-specific icons, such as and , are normally not included in the view hierarchy, which means that our localisation model is “trained to miss them”. However, these icons are necessary to detect ad-related dark patterns like Disguised Ads. Actually, most advertisements will have the exact same ad-specific icons. The reason is that many advertisement network companies³³3https://digitaladvertisingalliance.org/participating, such as Google Inc., Facebook, Microsoft and AT&T, join the adChoices⁴⁴4https://youradchoices.com/ self-regulatory program, which aims to establish and enforce responsible practices for online behavioural advertising and give consumers enhanced transparency and control. By joining this program, their advertisement is required to provide the adchoice icon, i.e., a triangle info icon , to indicate that the ads are recommended based on their online behaviours. Therefore, we adopt the template matching technique (Brunelli, 2009) to find these missing ad-specific icons.

Finer details on the status of UI elements are necessary as explored in Section 4. Therefore, to recognise the status of UI elements, we trained another light-weighted ResNet-18 (He et al., 2016) model.

Color can impact the visual saliency to manipulate user perception. To extract the background and text color of UI elements, we use color histogram to first obtain the most frequently color as the background color. We then obtain the foreground color by computing the euclidean distance between the rest color with the background color, and find the one with farthest distance. The rationale inside is that we assume the designers will make the background and foreground colors having highest contrast ratio so that end-users could clearly separate the foreground content from the background.

Apart from the properties of UI elements, their relationships are also important, especially for False Hierarchy tricks, which need to compare the visual saliency among related UI elements. Normally, the app providers or designers will manipulate the color and size of buttons or text elements to highlight or dampen some options. Therefore, we use density-based spatial clustering of applications with noise (DBSCAN) algorithm (Ester et al., 1996), which is a density-based clustering algorithm, to perform elements grouping, and only consider text elements and buttons. The advantages of this clustering algorithm is that we do not need to pre-define the number of clustering.

The basic idea is that based on the element features, it can group UI elements in a high-density region, while marking those UI elements in low-density region as outliers. To define whether an area is of high-density or of low-density, we need to define a distance function and two thresholds, i.e., a distance threshold $\alpha$ and a density threshold $\beta$. We consider four features, including element types, size (width and height), coordinates and their text content, based on which we compute the distance between any two elements, to group relevant UI elements. After relevant elements are grouped together, we examine their color to see if they have big visual differences.

We utilized the Rico dataset to train our element localization model (i.e., FasterRCNN) as it provides bounding boxes and types for contained GUI elements. We selected 15 types of Android GUI elements (e.g., Button, ImageView, TextView) as our target objects⁶⁶6We provide the list of UI element and examples in the supplementary materials. We filtered out any UIs that do not contain these elements, resulting in a dataset of 50,524 UIs. We splitted the dataset into training, validation, and testing sets (80:10:10) and ensured that UIs from one app are only included in a single split to prevent potential data leakage.

We initialised the model parameters using the pretrained model of COCO dataset (Lin et al., 2014) and finetuned all parameters using the processed Rico dataset. The model was trained on a Nvidia 1080 GPU for 160 epochs with a batch size of 8, an initial training rate of 0.001, and weights are updated by an Adam optimizer(Kingma and Ba, 2014). To remove duplicate detection, we applied non-max suppression to choose the most confident detection from the overlapping detection. We set the interaction over union (IoU) threshold to 0.5, and found the best confidence threshold using the validation dataset, which is 0.65.

To train our icon recognition model, we used the annotations from Liu et. al (Liu et al., 2018). In total, they identified 135 icon types and labelled 73,449 icons. We removed duplicate icons by comparing the color histogram of icons, and considered icon types that appear at least 200 times in our training dataset. Note that icon types that appear less than 200 times are considered as an additional Other type. After that, we noticed some icons are noisy data or misclassified. Therefore, we manually examined all data and removed these wrong data. As a result, we had 81 target icon types, such as star and close icons, and one other type⁷⁷7All icon types can be seen in supplementary materials.. Note that while for now, we only require the identification of several icons, we still train a general icon recognition model for future extension. In addition, as the bounding boxes of detections may not be as accurate as the ground truth bounding boxes, we performed some data augmentation techniques to enhance the robustness. Specifically, we randomly added some noises to the groundtruth bounding boxes to simulate the potential inaccurate detections. All icons were splited into training, validation and testing dataset using the same method as the element localization dataset. This means that if an icon’s UI is in the training dataset, the icon will also be placed in the training dataset.

We rescaled the icon to a input size of $224\times 224$. We initialised the ResNet-18 model parameter using the pretrained model on ImageNet dataset and finetuned it on a Nvidia 1080 GPU for 100 epochs with a batch size of 128 and an initial training rate of 0.001. The model weights were updated by stochastic gradient descent optimizer.

For our status recognition model, we extracted UI elements with a type of “Checkbox”, “Switch” or “Toggle Button” from the semantic dataset. After we obtained the data, we then manually annotated the status of them as the Rico semantic dataset does not contain such information. Same as the icon classification model, we also performed data augmentation by randomly adding some noises to the bounding boxes. As a result, we obtained 1,845 checked elements and 1,383 unchecked elements. As some non-checkbox elements may be wrongly detected as checkboxes, we trained a 3-class model to deal with the problem. Therefore, we randomly extracted 3,093 other elements as a negative class for the model to mitigate the potential classification errors inherited from element localization step. This dataset was splited following the same strategy as in previous datasets.

The model was trained by inputting a RGB image of input size $224\times 224$ on NVIDIA 1080 GPU. The optimizer was SGD and the initial learning rate is 0.001. We used a batch size of 128 and trained the model for 100 epochs.

Baseline: We considered four baselines: UIED, FRCNN (nontext), FRCNN (nontext) with PaddleOCR results and our ablation model (FRCNN-all). UIED (Chen et al., 2020c) is a state-of-the-art UI element detection technique that combines deep learning and traditional methods to achieve precise localization of UI elements. For text detection, it uses the EAST (Zhou et al., 2017) model and for non-text element detection, it employs a novel old-fashioned method with a top-down coarse-to-fine strategy based on the unique characteristics of UIs and UI elements, along with a CNN classifier to classify the elements. However, our task does not require such high precision in element detection, as end-users will examine the results manually. Therefore, we also considered the Faster RCNN model trained to detect only non-text UI elements, referred to as FRCNN (nontext) for brevity, and used PaddleOCR to detect text elements and merged them with the non-text element detections, called FRCNN (nontext) $+$ PaddleOCR. Lastly, we compared an ablation version of our proposed technique, FRCNN (all), to evaluate the performance of the FasterRCNN-only method.

Metric: We used precision, recall and F1-score to evaluate the models. Precision is defined as $TP/(TP$+$FP)$, recall is $TP/(TP$+$FN)$ and F1 score is computed by $(2\times Precision\times Recall)/(Precision$+$Recall)$. A True Positive (TP) refers to a detected element matching the ground truth box and its type is right. A False Positive (FP) refers to a detected element that does either not match a ground-truth object or fail to predict the right element type. TP is determined based on the interaction over union (IoU) value of two boxes. The IoU value is calculated by $interaction\_area/union\_area$ of the two boxes. If a detected instance has a IoU with the groudtruth box over a preset threshold, and is predicted as a right class, we consider it as a match. We used the regular IoU threshold 0.5 for our task (Ren et al., 2015; White et al., 2019), as the final results would be examined to a human. In addition, we note that OCR engine (both EAST and PaddleOCR) will detect precise bounding box for text elements that the groundtruth bounding box may contain it and have a low IoU value. Therefore, if a text element is contained by a groundtruth text box, we also consider it as a match.

Results: Table 3 shows the performance of all models. Our model performs better than most baselines in most metrics, achieving 0.49 in F1 in non-text element detection, 0.62 in F1 in text element detection and 0.58 in F1 for all elements. Under a loose requirement of precision, deep learning models are able to achieve better performance on non-text element detection than UIED, especially in UIs with noisy background. For text-element, PaddleOCR performs similar to EAST used in UIED, both reaching around 0.50 in F1. After merging deep learning results with PaddleOCR, the performance of both FRCNN(nontext) in detecting non-text elements slightly degrades. Surprisingly, as we can see from the table, FRCNN(all) can achieve better performance than FRCNN(nontext)$+$PaddleOCR in detecting non-text elements, around 0.06 higher in F1 score. We investigated the reasons and found that OCR engines can detect texts on advertisements, and the groundtruth annotations normally do not contain the advertisement components and elements, which degrades the OCR engines. Our final model, UIGuard confirms the usefulness of the OCR engine, achieving 0.61 in precision, 0.55 in recall and 0.58 in F1 in detecting all elements.

For icon recognition module, we did not setup a baseline as this kind of method is mature. Therefore, we adopted the accuracy metric to evaluate our performance. Our model achieves 0.97 accuracy in testing dataset, and 0.95 in detections from our system. Details of the classification results for each icon type can be seen in supplementary materials.

Similarly, we also used accuracy metric to evaluate of our status recognition module. Our model achieves 0.99 accuracy in testing dataset and 0.94 accuracy in detections from our element localisation model. Details of the classification results for each type can be seen in supplementary materials.

Table 4 shows the detailed results for each dark pattern type, and overall results for each strategy and for all detections. We considered two weighted average metric: (1) macro average with equal weights for all categories, and (2) micro average with weights based on category object counts. Overall, UIGuard achieves macro/micro average precisions of 0.83/0.82, recall of 0.82/0.77, and F1 score of 0.82/0.79, respectively, which demonstrates that our system can effectively and accurately detect dark patterns in mobile UIs. To avoid the potential bias from cases with limited examples, we also reported the results after excluding these cases (n¡10) and the macro/micro results are 0.82/0.84 for precision, 0.76/0.77 for recall, and 0.79/0.80 for F1. In terms of strategies, UIGuard also achieves around or over 0.8 for all metrics in Sneaking, Interface Inference and Forced Action strategies.

In terms of the detailed dark pattern types, UIGuard reaches around or over 0.80 in F1 score in 7 types. Our system performs well in detecting certain types of dark patterns, such as NG-Pop-up AD, II-Preselection - Checkbox selected, II-AM-General Types - small close buttons, which shows the efficiency of our proposed system. However, UIGuard performs less well in detecting other types, such as NG-Nag to rate and NG-Nag to upgrade. While it remains good recall rates, it detects many false positives, which leads to a low performance in F1 scores. We manually checked the reason of false positives in these two types. As seen in Figure 3a and Figure 3b, our tool detects in-context rating and upgrade. However, based on the definition of these two dark pattern types, it has to appear as a pop-up window. Some additional techniques such as detecting the existence of pop-up windows can mitigate these kinds of false positives. In addition, the definition of the Disguised AD dark pattern is ambiguous, leading to poor performance in detecting it. For example, as seen in Figure 3c, we can see UIGuard detect the potential existence of dark patterns as it locates the ad icon. However, as this part of UI is clearly different from other parts as the content is quite different with a big picture, we do not consider it as a disguised ad. Similar situation applies to Figure 3d. As people may have different understandings of what is a dark pattern, a more personalized detector that allows users to control which dark patterns they want to be alerted to may be beneficial in addressing this issue.”

We then conducted ablation experiments to understand the usefulness and necessity of each module in our system. To do this, we considered the text-only model as the Base Model ($+$Text-only), and gradually added $+$Icon Semantic , $+$Template Matching, $+$Status Recognition and $+$Color & Grouping modules one by one. We calculated the same evaluation metrics for each ablation and noted the trend after the addition of each module.

As we can see in Table 5, Base Model performs well in detecting Sneaking and Forced Action dark patterns because these types of dark patterns rely heavily on text content. In comparison, Base Model does not perform well in detecting Nagging and Interface Inference types, as pure text information is not enough for these types of dark patterns. After adding the Icon Semantic module, we can see an increase in performance in the Nagging type from 0.09 to 0.28 in F1. The Template Matching boosted performance further, as many dark patterns are advertisement related, and identifying the existence of advertisement can be very useful in detecting ad-related dark patterns. This also means the basic text content and icon semantic modules are insufficient for these types, and our template matching could mitigate errors inherited from the element detection module. The Status Recognition module specifically deals with checkbox-related dark patterns. While maintaining the precision, its recall goes higher from 0.75 to 0.80 for the overall performance. Finally, after adding the Color & Grouping module, our system gains the ability to recognise dark patterns related to element relationships and color. Our final performance achieves 0.83 in precision, 0.82 in recall and 0.82 in F1 score. The ablation experiments show the usefulness and necessity of each module in our system.

To ensure a diverse range of participants, we recruited individuals of various ages, genders, and levels of education through social media platforms such as LinkedIn, Twitter, and alumni networks. All participants were required to be at least 18 years old and have experience using mobile phones. There were no incentives or reimbursements offered for participation in the study.

The study was conducted in the form of an online survey ⁹⁹9An example procedure can be seen in (Chen, 2023), which consists of five steps (see Figure 4). Before the survey starts, we gave a short introduction of our user study and asked for the consent from the participants to attend this study. After obtaining their consent, the first step collected demographic information from participants, including their gender, age range, level of education, career, frequency of mobile app usage and their knowledge of dark pattern.

In the second step, participants were asked to evaluate the existence of malicious UI designs in a set of $n$ UIs. They were provided with the definition of a dark pattern (i.e., “A malicious user interface is defined as a user interface using tricks that make you do things that you didn’t mean to”), and were asked to interpret it themselves. If they identified any malicious designs, they were asked to click on the location of the dark pattern on the UI, provide a reason for their identification, and rate the severity and difficulty of the issue on a 5-point Likert scale. Reasons for their ratings were optional to control the length of the study. The severity is defined as how much impact it will have if they are tricked. The hardness for recognising the issue is defined as how hard it is to recognize the issue (e.g., how much time it will cost, how carefully the end-user should be).

In the third step, participants were shown the detection results from the tool for each of the k UIs from step 2 (similar to the output in Figure 2), and were asked to evaluate whether they identified the same issues as the tool. If not, they were asked to provide a reason for their discrepancy, rate the severity and difficulty of the issue, and provide their reasons (optional). The participants’ own evaluations of the UIs were also shown to them so that they could recall what they recognise in Step 2. The reason that we ask the participants to compare their detections with the detection results from our tool, rather than the ground-truth results, is two-fold. First, we want to understand the usefulness of our proposed tool by learning the reasons why people could not identify the issues by themselves. Second, we want to evaluate whether the participants can learn to critique the results from our system and learn something from imperfect AI, as AI systems are not always 100% correct. This practice can be beneficial during the process of comparing their results with our detections.

Step 4 was a practice session, in which we asked the participants to evaluate another $m$ UIs with similar issues as the UIs in Step 2 and asked the same questions as in Step 2. This step aimed to examine whether participants have learned the concept from the short session in step 3, and to understand the educational effect of the tool.

In the final step, participants were asked to provide overall feedback about the usefulness, presentation, advantages, and disadvantages of the tool, and to rate their knowledge of dark patterns again. They were also given the opportunity to provide any other comments. A brief version of the survey is provided in the supplementary materials. Note that our study passes the ethical assessment from our organisation, and all participants gave their consent on using their answers in our paper.

For our user study, the participants was required to evaluate $2*n+m$ user interfaces. To control the time consumed by each study (around one hour), we sampled around 20 UIs for Steps 2 and 3, as well as for Step 4. In these sampled UIs, we aimed to include both correct and wrong detections from our tool in order to effectively test its usefulness. Additionally, we also included some benign UIs to evaluate the participants’ understanding of ethical and unethical user interfaces. Given the total of 14 target dark pattern types, we sampled 1-2 instances per type. As such, we adopted a probability-based sampling strategy to ensure a fair representation of different levels of performance for each type of dark pattern from our tool. To align the UIs in Step 4 with Steps 2 and 3, we sampled UIs that were not included in Step 2 and had a similar issue. To minimize bias and ensure representative, we ensured that each UI was sampled from different apps. As a result, we sampled 23 UIs for Step 2 and 3, and 20 UIs¹⁰¹⁰10Excluding UIs that have no dark patterns but are detected by our tool for Step 4.

To conduct the user study, we used the Qualtrics platform ¹¹¹¹11https://www.qualtrics.com/. Our pilot study indicated that the full survey (in which users need to evaluate in total 66 UIs) would take approximately one hour to complete. As we did not offer any financial incentives to participants, we decided to split the survey into four 15-minute segments. For each segment, we ensured that the UIs used in Step 4 were aligned with those used in Step 2. However, we also provided the option for participants to complete the full one-hour version of the survey if they preferred. The main distinction between the full survey and the 15-minute versions lies in the number of UIs that participants evaluate in Steps 2, 3, and 4. In the full survey, participants assessed all 23 UIs for Steps 2 and 3, and all 20 UIs for Step 4. Conversely, in the 15-minute versions, participants only evaluated a smaller subset of UIs, 6-7 for Steps 2 and 3, and 5-6 for Step 4. The UIs utilized in the 15-minute versions collectively encompass the UIs employed in the full survey. All questions and steps except the UIs are the same.

In total, 58 people participated in our user study. Of these, 24 finished the full 1-hour version and 34 completed a 15-min questionnaire. As seen in Figure 5, the participants were diverse in terms of gender, age, occupation, education level, and mobile phone usage. 24 of participants were female, 33 were male and 1 preferred not to say. The ages of the participants ranged from 18 to over 65, with 12 in the 18-24 age range, 39 in the 25-34 age range, 4 in the 35-44 age range, 2 in the 45-54 age range, and 1 over 65. Of these participants, 10 were students, 6 were software engineer or developers, 16 were researchers from different domains (e.g., software engineering, Human Computer Interaction (HCI), UI/UX, social, finance and AI), 19 preferred not to say. The rest were freelance, construction worker, data analyst, export operator, or accountant. The education levels of the participants varied, with 2 having a degree below a bachelor’s degree, 14 having a bachelor’s degree, 27 having a master’s degree, and 15 having a doctoral degree. In terms of mobile phone usage, 1 participant used their phone less than 5 times per day, 10 used it 5-10 times per day, and 47 used it more than 10 times per day. In terms of knowledge of dark patterns, 20 participants were unaware of dark patterns, 19 had a little knowledge, 15 had some knowledge, 3 knew very well, and 1 considered themselves an expert. These results show that our participants were diverse. A detailed demographic distribution of each survey can be seen in Section A.

When evaluating the presence of malicious UI designs in step 2, participants identified a total of 474 instances. Upon manual review, we found that 163 (34.4%) of these instances matched with our target dark patterns, resulting in a recall rate of 0.185. In detail, for each dark pattern type, we showed the number of correct detection and recall in Table 6.  ¹²¹²12Note that we do not report the number of instances that participants spotted for each type because we cannot tell which type they are.

Among all dark pattern types, participants performed the best in detecting II-AM-Disguised AD, achieving 0.625 recall rate. From their provided reasons, we found that most participants can sense that the advertisements which mixed in the main content are malicious design, but only a few of them can clearly articulate the exact reasons. However, for other types, recall rates for other types were lower than 0.3, with particularly low rates for NG-Pop-up to upgrade, II-Preselection-No Checkbox, FA-Social Pyramid and Pay to avoid AD (all below 10%). The main reason for these low recall rates was lack of knowledge or awareness of these patterns, with some participants stating that these patterns are common and not a big deal. Additionally, some participants believed that FA-Social Pyramid was not a malicious design because users have the option to invite a friend or not.

We then reported the average severity and hardness ratings for each dark pattern type and the overall results in Table 4. For both scores, we reported two average scores in terms of whether the participants successfully identified the malicious design in Step2 (TP) or were showed by our tool in Step 3 (FN). Generally, for most dark pattern types, participants had similar severity scores. For some dark pattern types, including NG-Pop up to upgrade, SN-Forced Continuity, II-AM-False Hierarchy, FA-General Types-Pay to avoid Ad, the average severity scores for TP were much lower than the scores for FN. This result indicates that participants who identified the dark patterns on their own considered them to be more severe than those shown to them by our tool. However, both groups agreed that dark patterns can cause frustration, waste time, and lead to financial loss. There was more variance in the ratings of difficulty. For NG-Pop up AD, II-AM-Disguised AD, II-AM General Types-small close buttons and FA-General Types-watch AD to unlock feature, FA-General Types-Pay to avoid Ad, participants who identified these instances independently perceived that it was easier to spot than other participants who can not. The only reverse pattern lies in SN-Forced Continuity, which was perceived as more difficult by those who were shown it by the tool.

In step 3 of our user study, we included 5 instances where our tool provided incorrect detections of dark patterns. We wanted to see if participants would be able to recognize these incorrect detections and learn from the correct ones. Out of the 92 reasons provided by participants for why they did not spot the wrong detections, only 12 (13%) recognized that the AI was wrong, 4 (4%) questioned the detection but were unsure, and the remaining 76 (83%) accepted the incorrect detections and blamed themselves for not spotting them. This highlights the challenge of distinguishing between accurate and inaccurate results from AI systems and the importance of assisting users in evaluating the output of AI systems critically. Fortunately, with our knowledge-driven dark pattern checker, which essentially uses a heuristics-based classifier, we can easily trace the reasoning behind its decisions. For example, in Figure 2, we can see that the system identified point 2 as a false hierarchy instance because it detected two related buttons, one with a grey background and the other with a white background, and the button labeled “Install” was highlighted while the button labeled “No Thanks” was difficult to notice. Providing this information to end-users can help them understand the decision made by the model and determine if it was correct or not.

In order to evaluate the effectiveness of our approach, we compared the results from Step 2 with those from Step 4. In Step 4, participants identified 570 instances of malicious UI design in the UIs presented to them. Of these, 453 (79.5%) matched our target dark patterns, yielding a recall rate of 57.8%. Overall, the recall rates for most dark pattern types were significantly higher in Step 4 compared to those in Step 2, indicating that participants had learned from the short session in Step 3. However, the recall rates in Step 4 were still relatively low, suggesting that a short session may not be sufficient for participants to fully understand and avoid dark patterns. Powered with knowledge learned from Step 3, the hardness scores all went lower compared to the results in Step 2/3, while the severity scores were similar as in Step 2/3.

To further assess the impact of our approach, we conducted a Wilcoxon signed-rank test (Wilcoxon, 1992) to determine whether participants’ knowledge of dark patterns had increased. The null hypothesis was that participants’ knowledge would remain the same, while the alternative hypothesis was that participants’ knowledge would increase. The test yielded a p-value of $7.1\times 10^{-9}$, indicating that the null hypothesis could be rejected and that the difference in knowledge was statistically significant. This suggests that our approach was successful in increasing participants’ knowledge of dark patterns.

Figure 6 shows the boxplots of usefulness and presentation ratings of our tool. For both these two aspects, UIGuard obtained the scores of a median of 4 and the majority of them had a score falling in the range of 3 to 5, which demonstrates that our system is useful and the current presentation way is acceptable. Besides, among all 58 participants, 45 (77.6%) participants rated the usefulness at 4 or 5, and 42 (72.4%) of them rated the presentation at 4 or 5.

Generally, participants appreciated the usefulness and accuracy of the tool, and believed this tool could “help users detect many dark patterns that may not be easily found by users” and “avoid them being tricked”, and thus brought them better user experience”. They also thought they learned something from this study and their awareness on dark patterns was raised. Moreover, some considered this tool can be used as “a foundational training tool” to raise people awareness in dark patterns, “help app stores to examine apps” and “remind the engineers to enhance their design and think about the ethical concerns.” Apart from this, some said they get used to these tricks and could well recognise these tricks by themselves. One interesting thing is that one participant mentioned that “even if we know trick, we cannot avoid clicking it sometime.” In addition, some had different opinions on the concept of certain dark patterns like “Ads are the source of income for some free apps. Watching the ads out of users’ choices shouldn’t be identified as dark design.” In addition, some participants also expressed confusion over distinguishing true detections from false positives. In terms of presentation, most participants found the tool to be clear and informative, though some mentioned issues with visibility for elderly users.