Overview
Research introduces a data selection methodology, termed MASS (Manifold Coverage and Sparse Feature Coverage), for optimizing supervised fine-tuning in large language models (LLMs). This approach addresses challenges associated with scaling fine-tuning data by formulating data selection as a coarse-to-fine hierarchical coverage problem. The objective is to identify high-value subsets from extensive candidate pools, aiming to reduce training costs while simultaneously enhancing model performance.
Research Context
Supervised fine-tuning data for LLMs continues to grow in scale, presenting a need for efficient data selection strategies. Current methodologies often evaluate data diversity within the original embedding space. This direct measurement can lead to entanglement of several factors, including dominant semantic directions, fine-grained differences in supervision signals, and local noise, which can complicate the identification of truly high-value data. The presented work aims to mitigate these limitations by introducing a novel hierarchical framework for data selection.
Approach
The proposed MASS method employs a two-stage, hierarchical coverage strategy for data selection:
- Coarse Semantic Grouping: The initial stage involves learning low-dimensional principal manifold coordinates. This is achieved through the use of a dense autoencoder. The purpose of this step is to establish broad semantic categories or groups within the data.
- Quality-Aware Sparse Feature Coverage: Following coarse grouping, the second stage performs quality-aware sparse feature coverage. This process is conducted within each of the previously established semantic groups. A TopK sparse autoencoder is utilized for this purpose, enabling the identification of key features within these groups.
This hierarchical formulation transforms data selection into a problem of covering different aspects of data at varying levels of granularity, moving from a general semantic overview to specific feature representation.
Findings
Empirical evaluation of the MASS method was conducted using two specific datasets: Vision Flan and LLaVA-CoT. The experiments yielded several key findings:
- MASS consistently outperformed established data selection baselines. This performance advantage was observed across multiple budget constraints, indicating its robustness in resource-limited scenarios.
- In several experimental settings, the performance of models trained using data subsets selected by MASS either matched or surpassed the performance achieved with full data training. This outcome suggests that MASS can enable significant data reduction without compromising, and in some cases enhancing, model efficacy.
Why This Matters
The ability to select high-value data subsets from large candidate pools is crucial for mitigating the increasing training costs associated with scaling supervised fine-tuning data for LLMs. By improving data selection efficiency, methods like MASS can contribute to reducing computational resource consumption and accelerating the development and deployment of advanced language models.
Key Limitations Mentioned by Researchers
The source abstract does not explicitly mention any limitations of the research or methodology.