Overview
This research investigates the impact of training data, adaptation methods, and model scale on the tool-calling performance of language model agents. The study employed a controlled experimental design to compare different training paradigms and their effectiveness in both in-distribution scenarios and across varied datasets.
Research Context
The existing body of controlled evidence regarding the joint effects of training data, adaptation methods, and model scale on language model agents' tool-calling capabilities is limited. Understanding these interactions is crucial for optimizing agent development.
Approach
The study evaluated three primary training methodologies:
- Supervised Fine-Tuning (SFT) utilizing LoRA (Low-Rank Adaptation).
- Reinforcement Learning (RL) implemented through Group Relative Policy Optimization (GRPO).
- A sequential approach combining SFT followed by GRPO (SFT-GRPO).
These methods were applied to six distinct Qwen3 models, ranging in scale from 0.6 billion to 32 billion parameters. The evaluation encompassed two main performance metrics: in-distribution performance and cross-dataset transfer capabilities. Additionally, the study included analysis of dataset mixing strategies and a comparison between LoRA and full-parameter fine-tuning.
Findings
In-Distribution Performance
- SFT with LoRA emerged as the most effective in-distribution method across the entire evaluated model scale range (0.6B to 32B parameters).
- SFT with LoRA was identified as the strongest method in 15 out of 18 experimental settings when evaluated in-distribution.
Cross-Dataset Transfer
- For cross-dataset transfer, where training and test datasets differed, the performance of the methods was closer.
- GRPO achieved superior performance in 29 out of 54 settings during cross-dataset transfer.
- However, GRPO's average margin of superiority over SFT in these transfer scenarios was less than one point.
- The SFT-GRPO combined approach was rarely the strongest method in either in-distribution or cross-dataset transfer comparisons.
- Dataset mixing consistently provided robust transfer performance. This approach maintained performance close to specialized in-distribution training, irrespective of the adaptation method employed.
LoRA vs. Full-Parameter Fine-Tuning
- Additional analysis confirmed that LoRA exhibited superior performance compared to full-parameter fine-tuning.
- This superiority was attributed to LoRA's ability to better preserve pretrained agentic behavior.
Why This Matters
The findings offer insights into effective training strategies for language model agents designed for tool-calling. Identifying SFT with LoRA as a strong in-distribution method and understanding the implications of dataset mixing for robust transfer performance can inform practical development of these agents. The observation that LoRA better preserves pretrained agentic behavior compared to full-parameter fine-tuning suggests its utility in maintaining foundational model capabilities during adaptation.