Overview
This research introduces Skill-Conditioned Gated Self-Distillation (SGSD), a method designed to enhance Large Language Model (LLM) reasoning by integrating experience-derived skills. SGSD reframes skill-based self-distillation (SD) as a process of teacher hypothesis validation rather than unconditional imitation. The approach utilizes retrieved skill-mistake pairs, a multi-teacher pool, and a verifier mechanism to generate token-level supervision. A gated objective is employed to distill informative signals and suppress uncertain or extreme outputs.
Research Context
On-policy self-distillation (SD) is a technique used to improve LLM reasoning capabilities. These methods typically rely on teacher-side privileged information (PI) to convert sparse verifier outcomes into dense, token-level supervision. Existing SD methods generally assume the availability of trusted PI, such as reference answers or successful trace data. The current research explores an alternative source for PI: an experience-derived skill bank. This skill bank offers compact and reusable skills but also introduces the challenge of potential irrelevance or misleading information.
Approach
SGSD addresses the challenge of using potentially irrelevant or misleading skills from an experience-derived skill bank. The methodology involves several steps:
- Skill Retrieval: The system retrieves skill-mistake pairs from a skill bank.
- Multi-Teacher Pool Construction: A pool of multiple teachers is constructed based on these retrieved skills.
- Student Rollout Scoring: All skill-conditioned teachers score the same plain-prompt student rollout.
- Verifier-Based Polarity Validation: A verifier validates each teacher's polarity. This involves identifying whether a teacher's stance supports a successful outcome, in which case it generates positive supervision, or suppresses a failure, which also yields positive supervision. Conversely, an opposite stance leads to reversed supervision.
- Gated Objective: A robust gated objective is then applied. This objective is designed to distill informative disagreements between teachers and the student while simultaneously suppressing signals that are uncertain or extreme. This mechanism allows SGSD to leverage skills that may not always be perfectly relevant.
Findings
Experiments conducted on multiple mathematical reasoning benchmarks demonstrated that SGSD consistently improved performance compared to existing methods. Specifically:
- SGSD consistently outperformed GRPO.
- SGSD remained competitive with answer-conditioned OPSD, even under a weaker assumption regarding the nature of privileged information.
- For example, when evaluated on Qwen3-1.7B across AIME24, AIME25, and HMMT25 benchmarks, SGSD achieved an average improvement of 6.2% over GRPO and 1.7% over OPSD.
Why This Matters
The development of SGSD offers a method for enhancing LLM reasoning by utilizing an experience-derived skill bank as a form of privileged information. This approach addresses the scenario where such skills, though compact and reusable, might not always be perfectly relevant, potentially extending the applicability of self-distillation techniques to scenarios with less pristine supervisory signals.