Overview
Federated Large Language Model (LLM) unlearning involves the removal of targeted knowledge when both forget and retain data are distributed across private clients and inaccessible to a central server. This process creates a distinct challenge for aggregation compared to standard federated optimization due to the conflicting objectives encoded within locally trained adapters. A particular issue identified is “magnitude explosion,” where unnormalized cross-client aggregation leads to an increase in the merged update magnitude, subsequently degrading retained utility, particularly under more intensive unlearning conditions.
To address this, the FIRM (Fisher-Informed Retention Merging) framework has been developed. FIRM operates as a single-round mechanism designed for stable and retention-aware server-side aggregation. It leverages client-local Fisher statistics, which serve as a unified sensitivity signal. These statistics are used to construct a Fisher-normalized base merge and to apply explicit control over parameters sensitive to retention.
Research Context
The core problem FIRM addresses lies in federated LLM unlearning, a scenario where data for both knowledge removal (forget data) and knowledge preservation (retain data) remains distributed across multiple private client devices. The server, in this setup, does not have direct access to this raw data. Consequently, locally trained adapters on each client inherently encode objectives that can conflict, differentiating their aggregation from conventional federated optimization techniques. The previously mentioned phenomenon of magnitude explosion is a critical failure mode in this context, specifically where aggregated updates, lacking normalization, increase in magnitude, leading to a deterioration of the model's ability to retain desired information, especially when aggressive unlearning is applied.
Approach
FIRM's methodology centers on using client-local Fisher statistics. These statistics are integrated to form a unified sensitivity signal. This signal guides two primary components:
- A Fisher-normalized base merge: This component forms the foundation of the aggregation process, incorporating the sensitivity information derived from Fisher statistics to normalize updates.
- Explicit control over retain-sensitive parameters: Beyond the base merge, FIRM introduces a mechanism to specifically manage parameters that are critical for retaining desired knowledge.
The framework's second-order retention projection mechanism yields a unique minimum-distance solution. Furthermore, the corresponding dual of this projection scales with the number of retain clients rather than with the model dimension, indicating a computational characteristic relevant to federated settings.
Findings
Experiments were conducted across three distinct datasets: WMDP, TOFU, and MUSE-HP. The LLMs utilized for these evaluations were Zephyr-7B and Llama-3-8B. The primary findings indicate:
- Utility gains at comparable forgetting rates were observed on the WMDP and MUSE-HP datasets.
- On the TOFU dataset, the performance showed metric-dependent trade-offs.
- Controlled comparisons specifically supported the benefits of combining the base merge component with the additional retention correction mechanism at selected operating points.
Why This Matters
The issues addressed by FIRM are pertinent to scenarios requiring targeted knowledge removal in LLMs while data remains private and distributed across clients. The degradation of retained utility under stronger unlearning, resulting from magnitude explosion, is a practical challenge. FIRM's approach to stable, retention-aware aggregation offers a framework for managing these conflicting objectives in federated unlearning settings.