Overview
FitText is introduced as a training-free framework designed to address challenges in tool-using agents, specifically concerning the retrieval of appropriate tools from large API ecosystems. The framework operates by making the tool interface revisable during execution through the generation, refinement, and evolution of natural-language pseudo-tool descriptions, which function as retrieval probes. This approach aims to overcome the semantic gap between user requests and tool documentation that can hinder initial static retrieval efforts.
Research Context
Efficient reasoning for tool-using agents is not solely dependent on shortening an answer trace; it also relies on agents reasoning over the correct action space. The scaling of API ecosystems to potentially tens of thousands of endpoints creates a concrete problem where static retrieval based on an initial query can fail before the planning phase begins. In such scenarios, improved planning alone is insufficient to compensate for a missing tool. The research frames this challenge as a problem of budgeted test-time retrieval, acknowledging the need for dynamic adjustment in tool discovery during the execution process rather than relying solely on initial static retrieval.
Approach
FitText supports multiple operational modes: serial refinement, parallel exploration, and Memetic Retrieval. The core mechanism involves generating natural-language pseudo-tool descriptions. These descriptions are then refined and evolved to serve as more effective retrieval probes. Memetic Retrieval specifically incorporates evolutionary selection, local refinement, and tool memory. The inclusion of tool memory aims to mitigate redundant search efforts by storing and recalling previously identified tools or descriptions. The framework's design allows for the dynamic alteration of the tool interface during the execution phase, departing from static retrieval methods.
Findings
- On the StableToolBench dataset, which comprises 16,464 APIs, Memetic FitText achieved a pooled pass rate of 84.3%.
- This performance represents a +26.7 percentage point improvement over static retrieval.
- Memetic FitText also showed improvements of +22.2 percentage points over Single-Pass, +23.2 percentage points over Re-Invoke, and +27.5 percentage points over Xu-style root refinement.
- The framework demonstrated superior performance across all evaluated current models.
- Gains in performance increased in correlation with model capability.
- The most significant improvements observed were in ambiguous multi-tool tasks. In these scenarios, dynamic re-retrieval, facilitated by FitText, was able to restore correct candidate tools after initial identification errors.
- When operating with 40-way concurrency, the parallel population execution within FitText maintained batched wall-clock time at approximately 1.01 times that of Single-Pass, despite the increased search workload.
Why This Matters
The ability of FitText to dynamically adapt tool interfaces during agent execution addresses a fundamental limitation in current tool-using agents, particularly within extensive API environments. By enabling agents to evolve their understanding of available tools in real-time, the framework enhances their capacity for accurate and robust reasoning, especially in complex and ambiguous task settings. This dynamic approach ensures that agents can more effectively navigate and utilize vast tool ecosystems, improving overall task completion rates and efficiency.