Towards Scalable Defense Against Intimate Partner Infiltration

An expanding body of research underscores how abusers exploit technology to harm their intimate partners [31, 32, 8, 14, 15, 16, 23]. The consequences are far-reaching, including financial harm [3], escalating to physical confrontations, and even resulting in homicide [28].

Among the tactics explored by  [15, 31, 35, 4, 6], three distinct types of technology-facilitated IPV have been identified: remote, proximate, and physical access. Remote tactics include actions such as distributed denial-of-service (DDoS) attacks, SMS bombing, unauthorized remote logins, and location tracking [15, 29]. Proximate tactics involve methods requiring closer physical presence, such as the use of spy cameras, router monitoring, or other forms of nearby surveillance [6]. Physical access tactics, on the other hand, encompass direct interactions with devices, such as installing spyware [8] or creepware [27], accessing phone records, or deploying keylogging tools to monitor keystrokes. These tactics rely on distinct attack vectors.

In this work, we introduce a taxonomy specifically focused on the physical access category of IPV, which we term Intimate Partner Infiltration (IPI). Physical access tactics are particularly challenging to detect, especially on smartphones, because abusers often have legitimate access to these devices and may already know or have registered credentials such as facial recognition profiles or passcodes [31]. This inherent proximity and familiarity make detection more complex, as these actions can easily blend into normal usage patterns. Additionally, addressing physical access tactics requires more careful and nuanced design approaches, as confronting such behaviors carries a higher risk of escalating the situation, potentially putting the victim in greater danger [23, 28, 3].

Based on the literature  [31, 35, 4, 6], we further refine and taxonomize IPI tactics into categories, actions, and subactions. The Benign category includes general, non-malicious behaviors that resemble normal device usage and serve as a baseline for comparison. Impersonation involves abusers pretending to be the victim by sending emails, messages, reviews, or comments to manipulate perceptions or harm reputations [31, 35]. Leakage refers to accessing or extracting private information, such as account settings, browsing history, messages, or files, and sometimes uploading content like photos or videos to violate privacy [6]. Modification includes altering device settings, changing account credentials, or modifying files, disrupting the victim’s sense of control and potentially causing emotional or psychological harm [4]. Lastly, Spyware involves the installation of software to covertly monitor the victim’s activities, representing one of the most invasive and persistent forms of IPI [35].

This taxonomy enables systems like AID to classify IPI behaviors with increasing granularity, distinguishing between categories (5-class), actions (9-class), and subactions (28-class), supporting more precise detection and intervention strategies.

Prior research on technology-facilitated IPV has mainly provided qualitative insights into victims’ experiences and perceptions of how perpetrators misuse technology. Freed et al.[16, 15] examined the ways in which abusers deploy GPS trackers and audio surveillance tools, gathering details from victims to infer perpetrator methods. Bellini et al. [4] identified online forums and communities as accelerants for Intimate Partner Surveillance (IPS), emphasizing how readily available strategies and tools exacerbate the problem. Tseng et al. [31] provided measurement-based evidence of technology-driven abuses, illustrating the scope of these issues. Other research has also emphasized the need for legal frameworks and defensive tools to combat IPV and address the widespread availability of spy devices [30, 6].

A range of mitigation solutions has emerged in response to these findings. Although basic tools exist to help users remove browser histories and delete digital traces to safeguard their privacy [2], they do not address the wide spectrum of potential IPI abuses. Instead, security clinics [18, 32] provide in-person consultations for survivors, offering tailored support to identify risks and rebuild a sense of safety. While these clinics have demonstrated significant promise, scaling their services beyond local communities presents considerable challenges. One major limitation is their reliance on specific physical locations, which restricts access for survivors outside those areas. Furthermore, these clinics require the involvement of technical experts who possess specialized knowledge to address the complex and evolving nature of technology-facilitated abuse. Recruiting and training such experts, as well as ensuring ongoing support, is resource-intensive and difficult to sustain at scale. This combination of geographic constraints and reliance on highly skilled personnel highlights the barriers to replicating models like the Clinics to End Tech Abuse (CETA) in New York City [9] and the Madison Tech Clinic (MTC) in Madison [22] in broader contexts. These challenges underscore the need for scalable, accessible, and resource-efficient approaches to combat IPI on a wider scale.

Another highly promising yet underexplored approach for detecting IPV is continuous authentication. From a technical standpoint, user authentication on smartphones falls into two categories: one-time and continuous methods. While one-time verification (e.g., PINs, facial recognition) initially prevents casual intruders, it can be insufficient in IPV settings where abusers may already know these credentials [31, 36]. Continuous authentication, drawing on biometric or behavioral data (e.g., motion sensors [12, 7, 21], touch interactions [13, 36, 38], or hybrid approaches [10, 1]), offers stronger security by persistently checking whether the current user matches the legitimate owner. Nonetheless, such systems have been designed primarily for strangers or unauthorized outsiders. When abusers are intimate partners, benign sharing (e.g., letting a child play a game) can resemble malicious infiltration, making it difficult to distinguish IPI behaviors from normal usage.

Our analysis in Table 3 highlights this challenge, revealing that close friends or partners demonstrate significantly more similar behavioral signatures than strangers when using the same device. We derive these results by computing the similarity scores of individuals with close relationships versus those with no connections. These scores are calculated based on the cosine similarity of their behavioral representations, which are extracted using our feature extractor (a pretrained AutoEncoder; see Section 4.3 for implementation details). This metric quantitatively supports our hypothesis that individuals with close relationships, such as partners, exhibit significantly more similar behavioral signatures than strangers when using the same device. This similarity complicates the design of a user-detection system. Moreover, survivors often become wary of technology after experiencing trauma [16], complicating the adoption of new security tools. Consequently, while improvements in continuous authentication and security clinics represent meaningful progress, more nuanced and automatic approaches are needed to address the subtle ways intimate partners can exploit physical access without triggering suspicion or harming the survivor further.

Our vision for AID is an “invisible” digital forensic tool that records a digital footprint of evidence of IPI on the victim’s smartphone. To accomplish this, AID runs continuously in the background, quietly collecting standard system traces from the phone for 1) user identification to verify whether the user is the legitimate owner and 2) behavior detection. Finally, based on our proposed taxonomy (Table 2), AID determines whether or not the behavior is potentially IPI related. If an IPI behavior is detected, the natural next step is to send an alert, but this may be visible to the attacker if s/he is still accessing the phone, or if the notification is not cleared before the attacker accesses the phone again. As such, we do not send any alerts, and instead record the event on the victim’s smartphone, leaving the analysis of detected IPI events for health and IPV professionals in security clinics. However, designing AID presents the following challenges:

Figure 1 shows AID’s workflow, which contains three phases: data collection, module inferencing, and analysis report generation.

During collection phase, AID samples and records time-series multi-modal data signals from the smartphone in the background (Table 4). AID collects 4 categories of modalities to obtain a comprehensive view of how the user is interacting with the phone. Among these, IMU and Sys play a critical role in the User Identification Module. Meanwhile, Int and App data are relevant to the Behavior Module for detecting suspicious behaviors associated with IPI. Intuitively, incorporating all data sources should result in the best performance due to more available information. However, our ablation studies (Section 5.5) reveal that there is a clear divide in the importance of each data type for each task; incorporating more information often reduces performance due to a lack of expressive power of a small model to generalize to redundant and higher dimensional inputs that results.

The User Identification Module employs a pretrained AutoEncoder to extract embeddings from user data, which are then used to train a Support Vector Machine (SVM) classifier that adapts to the target user. This adapted module performs identity verification, determining whether the current user is the authorized device owner. Meanwhile, the Behavior Detection Module utilizes a hybrid model combining Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNNs) to recognize and classify user activities into one of 27 IPI subactions, shown in Table 2. To ensure reliability, results from both modules are refined through a post-processing step, which enhances stability and accuracy.

These results are consolidated into a report that provides key details such as the time of detection, the app involved, the user’s status (e.g., owner or non-owner), the most likely behaviors detected (e.g., benign or suspicious actions in Table 2), and the overall risk assessment based on confidence measures output from each module. For instance, the report might highlight that a verified user performed benign activities on one app. At another point in time, the report may detect an anomalous user engaging in actions like "Alter Account Setting," leading to an "IPV Risk Detected" classification. The report remains securely stored on the device until it is analyzed by health, safety, or security professionals.

As we will see later in this section and in the evaluation, detecting phone behaviors in an “invisible” manner is much different than traditional human activity recognition that aim to detect physical body movements with the IMU on a smartphone, smartwatch, or other wearable. The signals available to us are limited, can only indirectly capture activities of interest, and different behaviors have similar traces; For example, changing an account password or searching for a YouTube video both involve typing and tapping on the screen. As such, AID logs the top-k most probable behaviors from non-owners for security professionals to analyze.

Next, we detail the implementation design of AID, and further discuss training mechanisms to improve adaptability to different users in Section 4.3 and safety mechanisms in Section 4.4. Section 4.5 discusses optimization strategies for AID, focusing on efficient computing for real-world use.

First, we pretrain an LSTM-based AutoEncoder using a corpus of diverse user data. This pretraining process allows the model to learn general patterns and representations of phone activity and user behavior. AutoEncoders are particularly suitable for this task as they excel at learning patterns and representations from unlabeled and limited amounts of data.

Second, when the owner installs the application for the first time, AID guides the owner through a short calibration procedure of 5 minutes, where the owner uses the device naturally. This session provides sufficient data fine-tuning the classifier to adapt to the owner’s unique behavioral patterns. The fine-tuning dataset is constructed by selecting a subset of the data newly collected from the owner and a subset of data from other users as negative examples. To ensure balance during classifier training, we maintain a 1:1 ratio between the selected samples from the owner and other users. We explore different methods for selecting and constructing the full fine-tuning dataset in our evaluation, specifically Section 5.2.

Finally, the backbone of our user identification module takes inspiration from KedyAuth[19], a state-of-art user authentication architecture for mobile platforms. We adopt a similar encoder-decoder structure as shown in Figure 2(a), except we replace the encoder backbone with an 8-head LSTM Layer (4 units per head), instead of using CNN layers, to better adapt to the multimodal time series data being processed. The outputs are concatenated and projected through a dense layer to produce a compact 16-dimensional latent representation. The decoder reconstructs the original input by first expanding the latent representation back to the input shape using a RepeatVector layer, followed by an LSTM layer for the reconstruction task. Once pretrained, the features extracted by the autoencoder are used as input into a lightweight SVM classifier with Radial Basis Function (RBF) kernel, to distinguish between use by the owner or a non-owner.

To detect IPI behaviors, we focus on interaction data (Int.) and app usage signals (App.) captured by the operating system (Figure 1). These modalities provide critical insights into how users engage with the device, such as navigating apps, interacting with the screen, or performing specific actions, which is indicative of IPI. We show through ablation studies in Section 5.5 that these two classes of signals enable the greatest performance, instead of relying solely on motion data, commonly used in traditional HAR.

Given the challenges of obtaining labeled data post-installation, we implement a server-side training architecture while deploying only the detection components on-device. Recent advances in HAR have explored various architectures, from CNNs[33] and LSTMs[24] to Transformers[11]. Inspired by the success of hybrid models in HAR[37], we develop an LSTM-CNN architecture for classifying IPI behaviors. As shown in Figure 2(b), our model processes input sequences through parallel paths: an LSTM layer with 64 units and a one-dimensional convolutional layer (kernel size=3, 64 filters). The CNN output undergoes max pooling and flattening before concatenation with the LSTM output. The combined features pass through two dropout-dense blocks for multi-class behavior classification.

Second, we leverage the Accessibility Service [17], a built-in function of Android OS that enables AID to launch without clicking on the application icon. This enables AID to begin running, while remaining absent from the “Recent Apps” page to enhance stealthiness and prevent abusers from terminating AID from here.

At the most basic level of awareness, the abuser only knows the existence of AID, but not its location on the device. In this case, AID is still feasible, as the abuser cannot prevent AID from running, and it is difficult to subconsciously mimic and change one’s phone’s habits.

A more informed level of awareness for the abuser is knowing the exact location of AID. The most direct response from the abuser might to delete AID, or force the phone to stop by altering background settings. Regardless of the approach, any attempt to interfere with AID already reveals a clear intention of IPV behaviors, which can be used to inform victims, security, and health professionals.

The most severe scenario, mentioned in 3, is a deterioration in the relationship, where physical violence is likely to occur. This scenario is beyond the scope of AID and requires external intervention, such as from police and law enforcement.

Each participant was instructed to interact with commonly used applications—Amazon, Gmail, Instagram, Slack, Spotify, and YouTube—to simulate everyday usage patterns (e.g., searching, changing passwords, or uploading media). Before starting, participants received a task list containing 44 actions (full list in appendix) designed to guide their interactions with these six applications. The task list (Table 11 lists all the tasks) aimed to cover a comprehensive range of scenarios typical for smartphone users, including both benign actions (e.g., watching videos or listening to music) and potentially IPI-related actions (e.g., attempting to modify an account password). The list is randomized to enhance generalization and reduce potential bias in the collected data. Before each task, a team member sets up the appropriate label for the task, after which the participant begins completing it. Once the task is finished, the team member assigns the label for the next action, repeating this process until all tasks are completed.

We intentionally introduced flexibility into the task instructions to allow participants to complete tasks in their preferred ways to capture increased behavioral diversity. For example, we did not prescribe specific postures for using the phone or dictate the exact steps to complete a task. In the case of modifying a password, participants could either navigate through multiple settings pages or use the app’s built-in assistant to reach the relevant page directly.

For the evaluation of the User Identification Module, we randomly select 95% of the data from 22 users as the pretraining set for the AutoEncoder. The remaining 5% is combined with data from 3 additional users, labeled as ’abuser’ (positive sample, denoted as class 1), to form the fine-tuning set. Two users are designated as the owner and the abuser. From the owner, the first 5 minutes of data are added to the fine-tuning set (as described in Section 4.3.), labeled as ’owner’ (negative sample, denoted as class 0), while the owner’s data after the initial 5 minutes and all of the abuser’s data are used to form the test set. For the evaluation of the Behavior Classification Module, we use a leave-one-user-out scheme, where data from 27 users serves as the training set, and the remaining user is used as the test set. This approach assesses whether the learned patterns can generalize to classify the behaviors of unseen users.

For all results, we used K-fold validation to consistency of our results. This involved setting up multiple permutations of pretraining users, tuning users, and test user combinations, and averaging the results across these permutations. Specifically, for the User Identification Module, we tested 12 combinations of owner-abuser pairs: 9 involving the previously mentioned participant pairs (e.g., couples or close friends) and 3 involving strangers to increase test diversity. For the Behavior Classification Module, we conducted 12 iterations of leave-one-user-out evaluations and averaged the results.

The AutoEncoder for user identification is pretrained on a self-supervised reconstruction task for a maximum of 100 epochs with a batch size of 512, using the Adam optimizer and a learning rate of 1e-3. Training incorporates early stopping with a patience of 5 epochs and a minimum improvement threshold of 0.0001. Mean Squared Error (MSE) is used as the loss function to reconstruct the input data. Similarly, the Behavior Classification model is trained for 50 epochs, batch size = 512, Adam optimizer, learning rate = 1e-3, and is optimized with Categorical Crossentropy as the loss function.

Interestingly, we observe a plateau or slight performance drop with certain configurations, such as a 5 Hz sampling rate and a 5-second window size. We hypothesize that while a larger window size and sampling provides more information, it requires a larger and more expressive model to generalize well. Our model has the smallest number of parameters, compared to existing authentication works, which limits its capability of generalizing to higher dimensional inputs. Despite these limitations, our results suggest that a highly expressive model is not strictly necessary. The best configuration performs admirably even at moderately low sampling rates, making the system computationally efficient and power-friendly. Additionally, the stability of F1 scores across varying configurations highlights the model’s adaptability, offering flexibility in deployment scenarios where power and processing constraints are critical.

We opt for the best configuration, i.e., a sampling rate of 10 Hz with a window time span of 1 second. Unless explicitly stated otherwise, all subsequent evaluations of the user identification module are based on this optimal configuration.

Compared to AuthentiSense, AID achieves a 6.4% improvement in F1 score. AuthentiSense employs a larger CNN-based siamese network trained using the triplet mining technique, which is highly demanding in terms of training data and prone to overfitting or inefficiencies. In contrast, AID benefits from its much smaller model size (25$\times$ smaller) and a more direct and efficient training approach. By leveraging unsupervised training with an AutoEncoder, AID learns feature representations directly from encoding and reconstructing raw data traces, leading to better generalization despite using significantly fewer parameters.

Similarly, AID outperforms KedyAuth, which also uses an unsupervised AutoEncoder and an SVM classifier. AID achieves a 12.2% improvement in F1 score and a 6$\times$ reduction in FPR, which is critical in IPV detection scenarios. KedyAuth’s CNN-based model is optimized for high-frequency data at 100 Hz but struggles to effectively handle lower-frequency data. In contrast, AID leverages a multi-head LSTM AutoEncoder, which excels at processing sequential data and extracting richer patterns from lower-frequency signals. Additionally, AID incorporates a post-processing technique (Section 4.3) that further smooths and stabilizes the results, reducing the FPR to just 5.2%, which is significantly lower than KedyAuth’s 33.1%.

When selecting based on chronological time, we simply use the first $n$ consecutive windows provided by the user. We believe that this scheme performed worse than random selection because the windows collected at a specific time is likely less diverse than randomly selecting windows across the entire calibration session, where users are likely performing a variety of different actions throughout.

When selecting based on similarity, we select windows from the owner and from other users that are similar to each other (measured by cosine similarity). The intuition is that it is more difficult to distinguish inputs that are more aligned, so incorporating them during the fine-tuning phase could potentially help the model distinguish these harder cases. To achieve a balance, we select windows in a specific ratio: 30% hard (most similar), 50% mid (moderately similar), and 20% easy (least similar). However, these cases are likely not completely indicative of the general behavior of the owner and would require more typical examples to generalize well. Randomly sampling provides a high chance of selecting diverse samples that both embody a user’s typical behavior and samples that may be difficult to distinguish from other users.

As shown in Figure 3, the peak F1 score is achieved with our random selection method using 12 windows for fine-tuning and remains consistently high as the number of windows increases. In comparison, similarity-based selection exhibits a similar trend as the random method but plateaus at a lower F1 score. Furthermore, it shows instability as the training set size changes, fluctuating from an F1 score of 0.960 with 20 windows to 0.902 with 24 windows, demonstrating its less stable performance compared to random sampling and highlighting its difficulty in finding diverse yet representative windows for adaptation. Similarly, the time-based method, constrained by its lack of diversity, consistently results in lower F1 scores across different training window sizes.

However, by slightly broadening our logging to the top-k, the rate we successfully “detect” or report the true behavior increases drastically. For instance, reporting the top-5 most probable subactions yielded an F-1 score of 0.895 and reporting the top-3 actions boosted the F-1 score to 0.923. These results highlight how certain IPI behaviors may exhibit similar patterns across multiple modalities, especially at finer-grained levels (e.g., subactions). This reinforces the need for systems like AID to account for ambiguity by ranking behaviors, rather than solely relying on single-point predictions, which still offers investigators or automated systems actionable insights.

The evaluation of the end-to-end (e2e) system focuses on its ability to combine User Identification Module and Behavior Classification Module to produce final predictions aligned with the goals of IPI detection. The labeling process for e2e performance is derived by integrating the outputs of these two levels. Specifically, User Identification Module determines whether the user is the owner or abuser, while Behavior Classification Module classifies the corresponding behavior into categories such as benign or IPI-related actions. By combining these outputs, the system assigns a single label to each instance that encapsulates both the identity of the user and the behavioral context.

For example, behaviors classified as "Alter Account" are considered safe when performed by the verified owner but are flagged as IPI risks when executed by an abuser. Similarly, benign behaviors by an intruder, such as general browsing, are marked as non-risk. This labeling framework ensures that the system captures the interplay between identity and behavior, allowing for nuanced and context-aware decision-making that is not possible by analyzing just one or the other.

The rationale behind this labeling approach lies in its structured integration of user identity and behavioral context, achieved through explicit rule-based definitions. As mentioned, FPR is crucially important due to the nature of IPI events. Trivial false alarms could lead to significant negative consequences for victims, such as undue distress, mistrust in the system, or unintentional exposure to their abusers. So it is important to handle interventions with care to avoid exacerbating emotional distress, particularly for at-risk populations like IPV survivors. The e2e labeling framework minimizes such risks by ensuring that both user identity and behavioral context are integrated into the decision-making process.

Table 7 shows the e2e performance of AID at Action level (5-class) behaviors in identifying both the correct user (owner vs. non-owner) and the IPI behavior. The best performance is achieved through top-3 behavior classification. With 0.981 F1 score and 0.040 FPR, the e2e system demonstrates great reliability when allowed to consider the top three ranked predictions, effectively minimizing false alarms. Narrowing the prediction scope to top-2 provides more confident results, with only a marginal trade-off in F1 score, precision, and FPR. Similarly, top-1 predictions achieve the highest certainty but at the cost of slightly higher FPR, underscoring the inherent trade-off between prediction completeness and certainty. This balance is especially critical in high-risk scenarios, where reducing false positives is paramount to avoid unnecessary distress or harm to victims, yet ensuring accurate identification of IPI behaviors remains equally important.

Additionally, the application of post-processing consistently enhances system performance across all configurations. For instance, in the top-1 prediction, post-processing reduces FPR from 0.095 to 0.087 and improves F1 from 0.951 to 0.956, reinforcing the importance of refinement steps to mitigate prediction errors. These results validate the effectiveness of AID in integrating user identity and behavioral context, demonstrating its capability to reliably classify behaviors at the action level.

Figure 5 shows the performance of AID with different backbones for the classifier. We adopt SVM as the backbone of the classifier, since it has the highest performance, with 0.98 F1 score. Among the deep learning-based models, the Dense classifier demonstrates an F1 score of 0.969, which is slightly lower than SVM but higher than the LSTM classifier. Out of all architectures, SVM is the least complex, least expressive, but less susceptible to overfitting. It’s high performance suggests that the encoder could extract highly relevant features, with little data, to distinguish between the phone owner and other users, which is helpful in our few-shot fine-tuning process. Other more expressive methods, such as Dense MLPs and LSTM, typically require more data, memory, and compute resources.