Overview
This study investigates the capacity of small language models (LLMs) to perform introspection, specifically by detecting and reporting on perturbations within their internal activations. The research focuses on activation steering, which involves injecting concept vectors into a model's residual stream, and then assessing the model's ability to accurately report on these introduced perturbations. The work identifies limitations in prior evaluation paradigms for small models and proposes new confound-free methods for assessment. Furthermore, it introduces and evaluates Introspection Fine-Tuning (IFT), a supervised fine-tuning approach designed to enhance introspective abilities in LLMs.
Research Context
The investigation centers on whether small language models can internally monitor their own states and communicate changes. Previous work on evaluating such capabilities often employed a binary detection paradigm, where models were prompted to answer "Yes" or "No" regarding the detection of an injected thought. This study observes that for small models, this binary detection method is confounded, as steering biases these models towards affirmative responses irrespective of the actual content of the question. This confounding issue motivated the development of alternative evaluation paradigms.
Approach
To address the confounding issue in prior binary detection methods, the researchers proposed two confound-free evaluation paradigms:
- Sentence Localization: In this paradigm, models were tasked with identifying which of $N$ sentences had been perturbed. The chance level for this task is $1/N$.
- Strength Comparison: This paradigm required models to identify which of two sentences received a stronger perturbation (injection). The chance level for this task is $50\%$.
Six models from two distinct families, Llama-3.2 and Gemma-4, were evaluated using these paradigms. These models ranged in size down to 2 billion parameters. A key component of the research was the introduction of Introspection Fine-Tuning (IFT). IFT is a supervised fine-tuning method. It utilizes sentence-localization examples, which are constructed from the model's own perturbed forward passes, as training data.
Findings
- The binary detection paradigm previously used for evaluating introspection was found to be confounded in small models, as steering alone biased them towards affirmative responses.
- Using the confound-free evaluation paradigms, models as small as 2 billion parameters (2B) demonstrated introspective ability reliably above chance levels.
- Introspective ability generally increased with model scale across the evaluated Llama-3.2 and Gemma-4 families.
- However, the smallest model evaluated, Llama-1B, performed at or below chance levels in introspection tasks prior to fine-tuning.
- Introspection Fine-Tuning (IFT) significantly improved Llama-1B's sentence-localization accuracy, raising it from $9.6\%$ to $60.6\%$. This represents a $6\times$ improvement.
- The gains from IFT generalized zero-shot to the held-out strength-comparison task, with Llama-1B's accuracy improving from $30.2\%$ to $52.2\%$.
- IFT also resulted in improved introspection for models of 3 billion and 8 billion parameters.
- The application of IFT induced only negligible degradation on standard capability benchmarks.
Why This Matters
The findings suggest that introspective ability within language models is not solely determined by their scale but can be directly trained and enhanced. This training approach appears to unlock latent self-monitoring capacities. The implications of this for AI transparency and alignment are noted within the study.