ICANEWS

Engineering & Technology

Activating Directions for Mitigating Emergent Misalignment in Language Models Across Architectures

Research explored actionable activation directions for detecting and mitigating emergent misalignment in language models, specifically when models are fine-tune

Overview

Research explored actionable activation directions for detecting and mitigating emergent misalignment in language models, specifically when models are fine-tuned on insecure code. The investigation focused on whether this misalignment corresponds to a causally actionable activation-space direction shared across different model architectures. The study examined four instruction-tuned model families: Qwen2.5-1.5B, Gemma-2-2B, Llama-3.2-1B, and Ministral-3-3B, all identically fine-tuned.

Research Context

The context for this research involves the phenomenon of emergent misalignment in language models, specifically when these models are fine-tuned using insecure code. The internal structure of this misalignment is described as poorly understood. The research sought to determine if a common, causally actionable activation-space direction exists across various model architectures that correlates with this emergent misalignment.

Approach

The research methodology involved identically fine-tuning four instruction-tuned language model families: Qwen2.5-1.5B, Gemma-2-2B, Llama-3.2-1B, and Ministral-3-3B. To identify a common activation direction, a difference-in-means approach was utilized. The effectiveness of this direction was assessed by its ability to separate aligned and misaligned activations. Causal steering was then employed by subtracting this identified direction. A secure-code control was used to confirm the content specificity of the observed effects. Cross-architecture transfer was attempted using ridge regression maps.

Findings

  • Across the four identically fine-tuned instruction-tuned model families (Qwen2.5-1.5B, Gemma-2-2B, Llama-3.2-1B, Ministral-3-3B), a difference-in-means direction achieved 99.6% separation of aligned and misaligned activations at each model's final layer.
  • Causal steering, implemented by subtracting this identified direction, reduced code spillover by 21-51 points.
  • A secure-code control confirmed the content specificity of the observed reduction in code spillover.
  • Cross-architecture transfer, utilizing ridge regression maps, yielded large behavioral suppression, ranging up to 46 points.
  • However, cross-architecture transfer failed specificity controls, as random and orthogonal directions performed comparably in suppression.
  • The research identified a two-tier specificity structure: within-model directions demonstrated causal specificity and actionability.
  • Conversely, cross-model directions were found to be causally real but non-specific.
  • An asymmetric transfer topology was observed, where Gemma and Qwen acted as geometric donors, and Llama functioned as a receiver.

Why This Matters

These findings define the limits of linear cross-architecture correction for emergent misalignment. The results recommend a focus on within-model probing for auditing language models to identify and address misalignment effectively. This suggests that while some causal effects can transfer across models, precise and specific mitigation may require model-specific interventions.

Key Findings