Overview
Research introduces a clinically grounded framework designed to evaluate the privacy characteristics of medical language models (LMs). This framework moves beyond typical evaluations that focus solely on the recovery of training text, instead assessing information disclosure under threat models structured along a graded axis of adversarial access. The evaluation spans from publicly inferable demographic information to leaked note fragments, measuring both verbatim memorization of patient-specific text and semantic leakage of sensitive diagnoses.
Research Context
Medical LMs have demonstrated the capacity to memorize and subsequently reproduce protected health information. Existing privacy evaluation methodologies for these models frequently concentrate on the recovery of text from their training datasets. However, these methods often do not fully account for the practical implications of disclosure under realistic adversarial scenarios, which require a more nuanced understanding of how sensitive information might be extracted or inferred.
Approach
The developed framework addresses the limitations of prior privacy evaluations by introducing a graded axis of adversarial access. This axis categorizes potential access levels, starting with information that is publicly inferable (e.g., demographics) and progressing to more specific and potentially compromised data such as leaked note fragments. Within each tier of adversarial access, the framework employs two primary metrics:
- Verbatim Memorization: This metric quantifies the exact reproduction of patient-specific text from the training data.
- Semantic Leakage: This metric assesses the degree to which sensitive diagnoses can be inferred or recovered by an adversary, even if not in verbatim form.
The framework was applied to a specific medical language model. This LM had been continually pretrained on a dataset comprising 378,000 clinical notes. The objective was to observe the model's behavior regarding privacy implications when exposed to real-world clinical data.
Findings
Application of the framework to the medical LM revealed several key findings:
- High Memorization from Routine Metadata: The input of routine encounter metadata—specifically, patient name, date of birth, visit date, provider name, and practice location—elicited high rates of verbatim memorization across a patient's historical timeline.
- Sensitive-Diagnosis Recovery: The framework indicated significant semantic leakage of sensitive diagnoses. For example, the Area Under the Receiver Operating Characteristic (AUROC) curve for abortion recovery was 0.91, and for HIV recovery, it was 0.82.
- Nuance in Exact-Match Memorization: While exact-match memorization was observed, the study noted that it could overstate the true extent of sensitive disclosure. Approximately 36% of the memorized tokens were identified as reflecting templated documentation rather than unique, highly sensitive patient-specific narratives.
Why This Matters
This work highlights the inherent risks associated with training language models on longitudinal clinical data. The framework provides a practical and reusable methodology for conducting contextual privacy evaluations of medical LMs, which can inform the development and deployment of these models in healthcare settings.