Overview
This research investigates the controllability of political alignment in large language models (LLMs) through prompt-based steering. The study departs from typical LLM political audits that reduce alignment to a single point on a political compass, instead focusing on the extent and directionality of steerability in LLM responses. The methodology involved stress-testing prompt-based controllability across multiple ideological personas and a range of political items, utilizing several prominent LLMs.
Research Context
Existing political audits of LLMs often characterize a model's alignment as a singular point on a political compass. However, in practical deployment, a model's 'resting point' is less critical than its capacity for steerability. This steerability primarily operates via the system prompt, which can be a personalization layer set by a platform or induced from a user's history, rather than a manually written instruction.
Approach
The study implemented a dispersion-first stress test designed to assess prompt-based controllability. The experimental setup included:
- Targeted Personas: 12 distinct ideological personas, in addition to an unsteered baseline.
- Test Items: 70 items derived from the Political Compass.
- Replication: Ten replicates for each condition.
- LLMs Evaluated: Seven leading large language models: GPT-5, Claude, Grok, Gemini, DeepSeek, Kimi, and Qwen.
This comprehensive setup generated a total of 63,700 responses. The analysis focused on how contextual framing influenced variance along economic and society axes and observed differential shifts among the LLMs.
Findings
- Contextual Framing Dominance: Contextual framing, implemented through system prompts, explained approximately 88% to 93% of the variance observed on both the economic and society political axes. This indicates a high degree of instruction-adjustability in LLM responses.
- Model Identity Limited Influence: The identity of the LLM accounted for less than 3% of the observed variance, suggesting that prompt-based steering is a primary driver of political alignment shifts.
- Differential Model Response: LLMs did not respond uniformly to steering. Some models exhibited greater movement in their responses than others, while some demonstrated saturation under extreme framing conditions.
- Resolution of Conflicting Directional Steering: Previous audits reported conflicting results regarding directional steering. This study suggests these discrepancies can be resolved by acknowledging that baseline responses are often not centered. The effect is geometric, differentiating between displacement and proximity, rather than indicating differential compliance.
- Consistent Shifts under Authoritarian Prompts: When subjected to authoritarian prompts, models produced similar shifts on identical questions.
- Implications for Auditing: The findings indicate that political-coordinate audits should incorporate steerability assessments, reporting on dispersion, symmetry, saturation, and refusal floors in LLM responses.
Why This Matters
The study highlights that the capacity for LLMs to be steered ideologically through system prompts far outweighs inherent model identity in determining response variations. Understanding dispersion, symmetry, saturation, and refusal floors in steerability is critical for robust political auditing of LLMs, moving beyond static evaluations to dynamic assessments of how models respond to contextual influence.