Proposing an Interactive Audit Pipeline for Visual Privacy Research

As society progresses, humans are becoming more dependent on the accessibility and convenience that technology offers. Everyday a large amount of visual content is uploaded to Social Media Networks (SMNs) from billions of users across the globe, which can explain the large amounts of sensitive data that is available online. While these ecosystems have goals that revolve around helping people build connections with others; there are gaps in the methods used to protect the information for individuals and corporations who share or collect content [1, 2, 3, 4, 5, 6, 7]. The increased upload of images and videos emphasize the need for privacy protection and mitigation strategies for visual content. Visual privacy techniques extend from SMNs to connected networks, smart cities, lifelogging, and much more [8]. Various harms can occur as a result of sensitive information being displayed, which makes visual privacy a growing area of concern [2, 9, 3]. Existing technologies in the industry can show a disregard for protecting the information of individuals who share visual content or for individuals who are captured in the content [4, 6]. While research is being done to address these concerns, there exists a gap in understanding the overlap between fairness, privacy, and user-feedback for issues regarding visual privacy in the machine learning (ML) pipeline.

A need for visual privacy has emerged from SMNs and the integration of Internet of Things (IoT) devices that can expose sensitive information through visual content [10, 11, 12]. The constant sharing and storing of videos and images bring skepticism about individual privacy and rights [13, 14]. Researchers have created datasets, models, and deployed applications that they believe will provide privacy to its’ users [15, 16, 17, 18, 14, 19, 20]. Within these algorithms and systems, researchers should continually make decisions to assess the fairness, privacy, accessibility of the data and model in regard to the communities they serve. Bias can be curated from the data collection process, reinforced in the model’s training, and systematically imposed in the deployment phase [21]. With privacy and bias issues arising throughout the ML pipeline it provokes the question: does the impact from model development procedures outweigh the societal benefits?

The goal of this paper is two-fold. First, the aim is to understand visual privacy and fairness as their issues intrude into the pipeline and potentially impact the stakeholders and community where the pipeline is deployed. Secondly, we provide a comprehensive pipeline indicating fairness and privacy issues and propose an auditing strategy to reduce these affects in visual privacyresearch. In this paper, we observe the critical decisions that are often overlooked when deploying AI (§ 2). We further discuss several privacy, fairness, and ownership issues that can arise in the pipeline (§ 3, 4). We argue for the use of humanover-the-loop strategies to discover privacy and fairness issues in ML. We extend this technique to suggest two auditing processes: Fairness Forensics Auditing System (FASt) and Visual Privacy (ViP) Auditor (§ 5). Finally, reflect on the need to pursue research agendas that have harmful societal impacts (§ 6).

Visual privacy research provides technologies that can be used for multiple scenarios. ML algorithms and privacy-aware systems are developed to mitigate individual and platform risks in the realm of visual privacy. Algorithms and systems have been implemented for individuals in their daily lives [22, 23, 24, 25, 10] and on social media networks [26, 27, 20, 28]. To protect the visual privacy of individuals, researchers have suggested the use of concepts like obfuscation [18, 29] for mitigating objects and faces and privacy risk scores. The idea of visual privacy mitigation is centered around accessibility and caters to the individualistic concept of privacy [30, 7]. Most visual privacy systems use one or more of these five protection techniques [29]: intervention [20], blind vision [20], secure processing [19, 20], redaction [18], and data hiding [20].

Researchers [9] have explored visual content to understand how attackers can extract textual information, including credit card numbers, social security numbers, residence, phone numbers, and other information. The authors divide their privacy protection strategies into ones that control the recipient and others that control the content. They further conduct a user study to evaluate their effectiveness against human recognition and how those alterations can affect the viewing experience. In other words, researchers [25] wanted to build a privacy-aware system for blind users of social media networks. The dataset for this paper was collected via VizWiz, a mobile application that allows the participants to consent to their photos in this study. Before making this data public, the authors removed private objects to protect each user.

From these works, we notice the range of applications and the broad impact that they can have on society. When building these algorithms and systems, issues with fairness and privacy can seep into the pipeline. One of the most widely used models for computer vision is the Convolutional Neural Network (CNN) [31, 32, 33]. Wang et al. [34] studied bias on visual recognition tasks by a CNN model and provided the strategies for bias mitigation. Measuring social biases in grounded vision and language embedding leads into a direction of studying biases from a mixture of language and vision [35]. A comparison study using multiple visual datasets is performed to help to understand how biases could be in datasets and affect object recognition task [36]. There are significant disparities in the accuracy against darker-skinned females for the commercial gender classification systems [37]. Ethical concerns arise in facial processing technology. Two design considerations and three ethical tensions for auditing facial processing technologies are described in [38]. Specifically, auditing products need to be cautious about the ethical tension between privacy and representation. A framework for protecting users’ privacy and fairness has been proposed in Sokolic et al. [39]. The framework blocks harmful tasks, such as gender classification, which can generate sensitive information to certify privacy and fairness in the face verification task.

The accuracy and precision of these systems can depend on (1) the data collection process, (2) fairness forensics performed on the data, (3) human-over-the-loop techniques during model training, and (4) post-training evaluation. The researchers should also consider the effects of deployed models, the trade-offs for private and fair systems, methods to mitigate harms, and additional measures that can be added for accountability. The impact of our approach is that it addresses these existing limitations on traditional visual privacy systems and suggests auditing strategies for the ML pipeline. This suggested pipeline considers privacy and fairness issues at each phase of the pipeline. In an upcoming section, we explore the application of human-over-the-loop and extend that technique to incorporate FASt and ViP Auditor to allow researchers to create safe and fair systems.

We describe the ML pipeline as having three phases (Fig. 1). Phase 1 is the Data Preparation process. This phase includes considerations of (1) raw data sources, (2) data collection processes, (3) data storage, and (4) data cleaning processes that a researcher should explore before entering into the next phase. Data can come from anywhere and everywhere. With so many data source possibilities available, the researcher should consider which sources are relevant to them. The data collection process for researchers can include using existing image datasets, social media datasets, or web scraping methods in respect to the visual privacy research task. Once a dataset is collected, a researcher could employ data cleaning tasks (e.g., crowd-sourced labeling) to derive an optimal dataset and labels.

In Phase 2, shown in Fig. 1, we begin the Modeling process. The cleaned data from Phase 1 can be divided into three datasets: training, testing, and validation. Training data is used as input for the ML algorithm. After training with the researcher’s desired ML algorithm, the researcher receives a model to run testing and validation datasets on. The model provides the output of the performance with several metrics. This new information can be used for refining the model before entering the final phase of the pipeline.

The last phase of the proposed machine learning pipeline is Deployment. The Deployment phase uses real-world data as input for the selected model. The researcher or end-user will see the real-world applications and results of their selected model from Phase 2. This phase allows the researcher to evaluate their model’s performance and impact in the communities that they serve.

Efforts to implement technology that serves to mitigate harm are the motivation behind visual privacy research. In this section, we will discuss privacy and fairness issues that frequently occur with developing and deploying visual privacy systems. Examples of these issues are shown in Fig. 2 and Table 1. We suggest that when evaluating visual privacy systems, researchers should consider bias issues as they arise in the ML pipeline. As apparent from Figure Fig. 2 there are fairness issues involved with all stages of ML pipeline. This investigation will compromise of three over-arching visual privacy issues and describe how they could affect the ML pipeline. The question becomes are these models necessary, and do they serve to benefit the people or system that is affected by it?

ML models are constantly being updated once deployed to the real world; regular updates help to avoid and minimize costly errors. Differences in the time between error discovery and model correction for the deployed model are crucial. Systems should be able to respond to unexpected bias before, during, and after deployment.

It could be impossible to erase the damage caused by the aftermath of a system; however, stakeholders could start making a change now. One way to do this would be using an interactive ML approach, human-in-the-loop  [67, 68, 69, 70]. Training in the human-in-the-loop framework requires humans to make incremental updates to anticipate issues [71]. Traditional ML pipelines conduct training on their own without interference from humans. To debug these models, the researcher must begin a thorough investigation of the models’ predictions, parameters, and data after the learning has been completed. An interactive approach would allow a person in the model’s training process, which will reduce debugging and runtime. The human is able to check the learning for the model and coach the model to meet the desired result in a feedback cycle. Feedback cycles allow the researcher to provide positive feedback iteratively to the model after viewing the processes. This can allow the researcher to understand the possible bias and privacy issues in the model and mitigate it immediately. This approach can be extended to various ML research areas in fairness, computer vision, and privacy.

In traditional human-in-the-loop approaches, the human becomes a bottleneck for the feedback process. In light of this, we suggest using a human-over-the-loop approach [72]. Human-over-the-loop allows researchers to step into the pipeline as needed to perform corrections. This removes the necessity of a human approving each iteration of the model. With this feature integrated in the ML pipeline, the researchers should consider having multiple “humans” to monitor the training. This, in turn, can lower response times to resolve biases that may be imposed from “humans” during learning. Based on the human-over-the-loop technique, we propose the use of two interactive auditing strategies that can reduce fairness and privacy issues to allow researchers to conceptualize, develop, and deploy safer visual privacy systems.

Being mindful of the societal impacts, evaluation methods (i.e., FASt and ViP) and monitoring strategies (i.e., human-over-the-loop) have been presented as mitigation techniques to reduce errors in the ML pipeline and in the deployed system’s life cycle. However, there are no full-proof techniques for ensuring that the software is exempt from producing harm. For a stakeholder to know when to halt deployment implies that they have developed a plan for the system and require human intervention throughout the ML pipeline for proactive decision making. Monitoring for privacy and fairness issues and their potential to harm the community throughout the software’s life is an essential part of this. When evaluating the fairness of a model, a researcher can explore the model’s training data and performance metrics to decipher sub-trends and anomalies. From this evaluation, the researcher can generate an idea of what success can look like from their model.

It might also be helpful to pivot directions for the machine learning model to avoid going too far down a path that could prove disastrous for marginalized communities. There may be a point at which the model is beyond recognition. It may be worth completely re-imagining the ML pipeline or abandoning the effort altogether when it has strayed far from its’ intended goal. Before completely re-imaging or abandoning the model, the researcher could integrate human-over-the-loop techniques to improve the ML pipeline’s consideration for privacy and fairness. The decision of which route to go ultimately involves the researcherevaluating the trade-off between the safety for the impacted communities or the potential accomplishments of producing innovative software. Success should be inspired by the ability to impact society positively, not by a system’s ability to quickly solve an idea. Questioning when to stop in the ML pipeline should include prioritizing societial impact and the affect on marginalized communities.

Halting deployment on a project that has gone awry should be seen as a successful learning result, not as a failed project. If permissible, the stakeholder should consider opening up the research project or system for external review to cultivate a meaningful conversation around learning from the harms that development and deployment could have caused.

Researchers should closely monitor data preparation, modeling, and deployment processes to avoid harming communities and stakeholders. The decision making process for researchers can be challenging, but it is imperative to continually evaluate to improve the model’s learning process and the deployment outcomes for the communities they serve. When building a visual privacy model needing large amounts of data, it can be easy to obtain datasets that are widely distributed, but may not have been examined for discriminatory, private. or fairness issues. This work discusses privacy and fairness issues that frequently occur in the ML pipeline that could emerge at various phases. We also assert the need for responsible auditing systems to bring accountability into model training and the deployed system. To do this, we propose using human-over-the-loop strategies to introduce interactive auditing for fairness and privacy. With ML pipeline audits and engaged researchers, the evaluation and consideration given to project development and deployed systems can become a standard procedure. These proposed mitigation strategies are the first steps of a much needed effort to address privacy and fairness issues in the ML pipeline.

We believe that apart from suggesting an interactively auditable ML pipeline, future research should consider the trade-off between privacy, fairness, and model accuracy in these systems. In addition, we will develop a strategy for determining a failure rate for ML models to provide a mechanism for researchers to continually evaluate if their system is successful. These considerations warrant further investigation to determine the success and limitations of deployed systems.

The researchers are partially supported by awards from the Department of Defense SMART Scholarship and the National Science Foundation under Grant No. #1952181.