LegalGuardian: A Privacy-Preserving Framework for Secure
Integration of Large Language Models in Legal Practice

The integration of LLMs into legal practice intersects disciplines like computer science, computational linguistics, cryptography, privacy, and ethics. Preserving client confidentiality while leveraging LLMs requires understanding these fields. Researchers have explored techniques such as differential privacy, data sanitization, encryption methods, and federated learning to mitigate LLM privacy risks (Edemacu and Wu, 2024). In the legal domain, strict ethical obligations and regulations amplify these challenges, necessitating innovative solutions. This section presents related works most relevant to our study.

Cryptographic Techniques: Cryptographic methods like Multi-Party Computation (MPC) (Yao, 1982) and Homomorphic Encryption (HE) (Gentry, Halevi, and Smart, 2012) secure computations on sensitive data. MPC allows collaborative computation without sharing inputs (Lindell, 2020), and HE enables computations on encrypted data without decryption (Iezzi, 2020). However, these techniques often have high computational overhead and scalability issues, especially with high-dimensional free-text legal data. Latency from cryptographic operations can make real-time applications impractical (Yavuz et al., 2017), and implementing them with LLMs requires substantial architectural changes, challenging practical deployment.

Differential Privacy and Adversarial Training: Differential Privacy (DP) (Dwork, 2006) controls privacy loss by adding calibrated noise to data or query results, limiting the impact of any single individual’s data (Kifer and Machanavajjhala, 2011). Applying DP to free-text data is challenging due to natural language complexity (Li et al., 2021). Adversarial training (Li et al., 2023; Coavoux, Narayan, and Cohen, 2018) learns representations predictive for the main task but invariant to private attributes, aiming to prevent sensitive information leakage. However, these methods often trade off utility for privacy, degrading main task performance, especially when private information is intertwined with task-relevant features (Zhou et al., 2022).

Encryption-Based Methods: Techniques like EmojiCrypt (Lin, Hua, and Zhang, 2024) protect user inputs by substituting sensitive text with encrypted representations (e.g., emojis), obscuring PII while preserving structure for LLM processing. However, these methods risk the LLM misinterpreting encrypted tokens, reducing performance or causing unintended outputs (Edemacu and Wu, 2024). Non-standard tokens can introduce ambiguity, affecting model understanding, especially in precision-critical legal contexts.

Federated Learning: Federated Learning (FL) (McMahan et al., 2016) trains models across decentralized devices holding local data, keeping sensitive data on local devices—a benefit in privacy-sensitive domains like legal practice (Chalamala et al., 2022). In LLMs, FL could allow legal organizations to train models without exposing proprietary data. However, FL addresses privacy during training, not inference, where privacy breaches often occur with LLM chatbots. FL also introduces significant communication overhead and requires node synchronization, impractical in heterogeneous legal environments (Kairouz et al., 2021). Modifying LLM architectures for FL further limits its applicability.

Privacy in Legal Applications of LLMs: Applying LLMs in legal contexts faces challenges due to strict confidentiality and sensitive data. Research has adapted privacy-preserving techniques for legal needs, such as combining legal ontologies with NER to enhance sensitive information redaction (Cardellino et al., 2017). Domain-specific models like Legal-BERT (Chalkidis et al., 2020) improve understanding of legal terminology, aiding PII detection. However, these methods struggle with scalability and handling nuanced legal language. There is a lack of comprehensive solutions balancing privacy, efficiency, and usability tailored for legal practitioners using LLMs.

Data Sanitization: Data sanitization identifies and removes PII from user input before processing. Kan et al. (2023) proposed the PP-TS framework, using a local LLM to detect and mask sensitive attributes before sending data to a cloud-based LLM, involving pre-processing, LLM invocation, and post-processing. Similarly, Chen et al. (2023) introduced the ”Hide and Seek” (HaS) framework, requiring two models—a masker for anonymization and a reconstructor for de-anonymization—to protect privacy while maintaining utility. While effective in some contexts, these methods have limitations: training multiple models increases computational costs and complexity, making them less feasible for many legal practices. Additionally, PII detection accuracy and potential loss of contextual information can impact downstream task performance (Zhou et al., 2022).

Maintaining client confidentiality is a fundamental ethical obligation in the legal profession. To facilitate the integration of AI-enabled systems without compromising this confidentiality, we present LegalGuardian, a novel privacy-preserving framework specifically designed for legal practitioners. In this section, we provide a comprehensive overview of LegalGuardian’s development and evaluation, including a detailed case study on privacy-preserving techniques, the framework’s architectural design, the creation of a synthetic legal prompt dataset, and an assessment of its effectiveness in protecting attorney-client confidentiality.

To this end, first, we conduct a comprehensive case study on the application of privacy-preserving techniques within the legal domain, specifically addressing the protection of attorney-client confidentiality and privacy. This study aims to pave the way for the broader adoption of generative AI tools in the legal sector by tackling critical privacy concerns that hinder their integration.

Second, we develop a synthetic legal prompt dataset consisting of 50 synthetically generated prompts. These prompts are tailored to reflect realistic scenarios faced by immigration lawyers practicing in the United States. We aim to share this dataset with the research community to support further studies in this domain.

Third, we introduce LegalGuardian, displayed in Figure 1, the first privacy protection framework explicitly designed to safeguard attorney-client confidentiality in the era of generative AI. LegalGuardian employs NLP techniques to mask and unmask confidential PII within prompts, ensuring sensitive data remains secure during interactions with Large Language Model (LLM)-based chatbots.

Finally, we provide a thorough assessment of the proposed technique’s effectiveness in protecting confidentiality from both technical and legal perspectives. Technically, we evaluate the framework using metrics such as accuracy, entity-level accuracy, and contextual similarity. From a legal standpoint, we examine how our approach aligns with existing laws and ethical guidelines governing attorney-client privilege and confidentiality.

To effectively safeguard client confidentiality while utilizing LLMs in legal practice, we developed the LegalGuardian framework. Figure 2 illustrates the data creation and evaluation pipeline of our framework, which consists of four main components:

1. 1.

   Synthetic Data and Content Generation (Green Box): We generate realistic legal prompts that simulate various immigration-related scenarios, providing a comprehensive dataset for testing and evaluation.

2. 2.

   PII Masking Layer (Yellow Box): Using advanced NER models and local LLMs, we identify and mask PII entities within the prompts to prevent the disclosure of sensitive information.

3. 3.

   Secure Prompting Layer (Purple Box): With the PII securely masked, we interact with external LLMs using these sanitized prompts to perform downstream NLP tasks without risking privacy breaches.

4. 4.

   Evaluation Layer (Blue Box): We assess the effectiveness of the framework by evaluating its ability to preserve privacy while maintaining the functional utility of the outputs.

To simulate realistic legal scenarios for testing our framework, we generated a synthetic dataset specifically designed to reflect the workflow of an immigration lawyer practicing in the United States.

Initially, we synthesized realistic client and employer details, including names, addresses, nationalities, and other pertinent metadata such as visa types and practice areas. The Faker (Faraglia and et. al., 2016) library was employed to generate these details, ensuring diversity and authenticity within the dataset. Random selection methods were applied to probabilistically assign document titles and subfields within practice areas, thereby enhancing the variability and representativeness of the prompts.

Subsequently, dynamic prompts were constructed by integrating the generated entities and metadata into structured templates. These prompts were then processed using the Qwen-2.5 14B language model  (Qwen Team, 2024), producing 50 realistic prompts that reflect various immigration case scenarios, such as visa applications, asylum cases, and naturalization processes. Detailed prompt templates used in this process are provided in Appendix.

We selected 50 prompts for the study to balance the need for a representative dataset with the practical constraints of conducting detailed manual reviews for each prompt. This approach enabled the creation of a diverse and realistic dataset for evaluating the privacy-preserving capabilities of the framework.

To ensure the confidentiality of sensitive information within the synthetic prompts, we implemented a PII masking layer comprising two separate components to compare their results at the end.

Firstly, we utilized GLiNER (Zaratiana et al., 2023), specifically GLiNER_Multi_PII-v1 model, which is a NER model designed for flexibility and efficiency, capable of identifying custom entity types by leveraging BERT instead of being limited to predefined entity categories like traditional NER models. This choice was motivated by our task’s focus on PII detection, aligning perfectly with the model’s fine-tuned capabilities for recognizing and handling sensitive personal information. We chose GLiNER because our domain-specific task required assigning unique, custom entity names rather than relying on generic categories provided by other models, enabling more precise handling of legal-specific PII.

Secondly, we employed the Qwen2.5-14B language model (Qwen Team, 2024), prompting it to identify PII entities in a one-shot manner, as illustrated in Listing 1. Qwen-2.5 14B model is chosen because it is a state-of-the-art LLM that consistently ranks at the top of LLM leaderboards for its size category, making it an ideal balance between performance and computational efficiency. Among medium-sized open-source models (14B parameters), Qwen-2.5 demonstrated the best overall performance across benchmarks, solidifying it as the optimal choice for our application.

After effectively masking the PII in the prompts, we implemented a secure prompting mechanism—referred to as the Secure Prompting Layer (illustrated in the Purple Box of Figure 2)—to interact with an external LLM. This layer enables the use of masked prompts to perform downstream NLP tasks such as summarization, translation, or legal analysis without risking the exposure of sensitive information.

To rigorously assess the effectiveness of the LegalGuardian framework, we implemented a comprehensive evaluation layer focusing on several key metrics designed to measure both privacy preservation and utility:

1. 1.

   Masking Accuracy: We evaluated the precision and recall of the PII masking process to determine how accurately the framework identifies and masks sensitive entities.

2. 2.

   Semantic Consistency: We measured the semantic similarity between the outputs generated by the LLM when using the original (unmasked) prompts and the masked prompts. This assessment ensured that the masking process did not adversely affect the semantic content and overall utility of the outputs.

All data from the prompts, entity dictionaries, and outputs were systematically stored and organized using the Pandas library, and metrics were calculated using SpaCy (Honnibal et al., 2020), facilitating thorough analysis. Both automated metrics and manual reviews were employed to validate the framework’s effectiveness in preserving privacy while maintaining functional utility in legal NLP tasks.

The masking and unmasking accuracies of the Qwen and GLiNER models were evaluated using a manually curated dataset comprising 50 prompts for each model. For every prompt, detailed review tables were generated, documenting the detected entity categories, the expected categories, and the correctness of entity detection.

The evaluation focused on specific entity categories relevant to legal applications, including person, case_number, date_of_birth, address, company, tax ID, location, date, law office, and nationality. True positives (TP), false positives (FP), and false negatives (FN) were identified to calculate precision, recall, and F1 scores. Here, true positives represented correctly detected entities; false positives indicated entities incorrectly detected or over-detected; and false negatives corresponded to entities that were expected but missed by the models. In total, the dataset includes different 460 entities.

The overall results demonstrated that GLiNER achieved higher precision due to fewer false positives, highlighting its effectiveness in avoiding over-detection. In contrast, Qwen exhibited slightly better recall by successfully identifying more relevant entities, albeit with a higher rate of over-detection. The F1 scores reflected a balanced trade-off between precision and recall for both models.

Figure  3 compares the performance of two models, GLiNER and Qwen2.5-14B, based on three metrics: Precision, Recall, and F1 Score. GLiNER achieves a perfect Precision of 100% compared to 99% for Qwen2.5-14B, highlighting its stronger ability to avoid false positives. However, Qwen2.5-14B outperforms GLiNER in Recall (94% vs. 88%), indicating better identification of relevant instances. Similarly, Qwen2.5-14B exhibits a higher F1 Score (97% vs. 93%), reflecting its superior overall balance between precision and recall. This suggests that while GLiNER is more precise, Qwen2.5-14B demonstrates better all-around performance.

To gain a deeper understanding of the models’ performance across specific entity categories, we computed entity-level precision, recall, and F1 scores for both Qwen2.5-14B and GLiNER. This granular analysis was crucial to evaluate how effectively each model handled different types of sensitive information commonly encountered in legal contexts, such as person, address, date_of_birth, law_office, and other relevant categories. By examining these metrics, we identified variations in model behavior across entity types, revealing strengths and weaknesses in detecting particular categories.

Table 1 shows the detailed performance metrics for GLiNER and Qwen2.5-14B across various entity types, highlighting key distinctions in their strengths. GLiNER consistently excels in precision, particularly for structured entities like case_number and tax_id, achieving perfect scores in precision, recall, and F1. This underscores its capability to minimize false positives, making it highly effective for well-defined categories. Similarly, GLiNER maintained perfect precision for entities such as address and company, further demonstrating its reliability in handling structured information.

Qwen2.5-14B, on the other hand, exhibited higher recall for more context-dependent categories like person and address. This suggests its strength in identifying a broader range of relevant entities, even at the cost of a slight reduction in precision. For instance, in the person category, Qwen2.5-14B achieved a recall of 77% compared to GLiNER’s 72%, while maintaining equal precision of 100%. This trade-off highlights Qwen2.5-14B’s proficiency in broader entity detection, particularly for less structured contexts.

In table 1, both models show balanced performance for core entities such as date and location. GLiNER achieved a perfect F1 score for date, while Qwen2.5-14B delivered a perfect F1 score for location, reflecting their shared reliability for these crucial categories. Additionally, for entities such as law_office and case_number, both models attained perfect scores across all metrics, confirming their strong capabilities for well-defined, structured types.

Overall, we observe complementary strengths between the two models. While GLiNER’s precision-driven approach is advantageous for structured entity types, Qwen2.5-14B’s higher recall makes it better suited for detecting context-dependent or ambiguous entities.

To assess the impact of the masking and unmasking processes on the semantic integrity of the outputs, we conducted a semantic similarity analysis. This analysis evaluated how closely the unmasked outputs from the LegalGuardian pipeline aligned with the original outputs generated from unmasked prompts. The objective was to ensure that masking sensitive information did not compromise the quality or meaning of the outputs—a critical factor in maintaining utility while preserving privacy.

We employed three measures to compute semantic similarity:

• •

  Cosine Similarity: Implemented using the spacy-transformer-md model, this metric measures the similarity between sentence embeddings of the original and unmasked outputs. It captures the overall semantic alignment by comparing the vector representations of the texts.

• •

  Jaro-Winkler Distance: This string-based metric focuses on character-level similarities, making it particularly useful for detecting slight variations in the output text. It assesses how similar two strings are by considering the number and order of matching characters.

• •

  Levenshtein Distance: Also a character-level metric, the Levenshtein distance quantifies the minimum number of single-character edits—insertions, deletions, or substitutions—required to change one string into the other. It provides insights into the structural changes introduced by the masking and unmasking processes.

The evaluation process involved generating baseline outputs by inputting the original, unmasked prompts into the Qwen model. These baseline outputs were then compared with the unmasked outputs obtained from each model (GLiNER and Qwen2.5-14B) after processing through the LegalGuardian pipeline. By comparing these outputs, we measured the extent to which the pipeline preserved the semantic meaning of the text.

In Table 2, we present the semantic similarity analysis, comparing GLiNER and Qwen2.5-14B across key metrics. GLiNER outperformed Qwen2.5-14B in both Cosine Similarity (0.9808 vs. 0.9731) and Jaro-Winkler Similarity (0.8328 vs. 0.7601), demonstrating stronger semantic alignment and structural fidelity in its outputs.

Qwen2.5-14B, however, achieved a lower Levenshtein Distance (0.4017 vs. 0.4358), indicating fewer character-level changes, which may benefit tasks requiring minimal textual modifications.

Overall, GLiNER excels in preserving semantic and structural consistency, while Qwen2.5-14B offers advantages in minimizing textual alterations, highlighting their complementary strengths.