This version is still in development and is not considered stable yet. For the latest snapshot version, please use Spring AI 1.1.3!spring-doc.cn

Dynamic Tool Discovery with Tool Search Tool

As AI agents connect to more services—Slack, GitHub, Jira, MCP servers—tool libraries grow rapidly. A typical multi-server setup can easily have 50+ tools consuming significant tokens before any conversation starts. Worse, tool selection accuracy degrades when models face 30+ similarly-named tools.spring-doc.cn

The Tool Search Tool pattern, pioneered by Anthropic, addresses this: instead of loading all tool definitions upfront, the model discovers tools on-demand. It receives only a search tool initially, queries for capabilities when needed, and gets relevant tool definitions expanded into context.spring-doc.cn

Spring AI’s implementation achieves 34-64% token reduction across OpenAI, Anthropic, and Gemini models while maintaining full access to hundreds of tools.spring-doc.cn

Introduction

The Tool Search Tool project extends Spring AI’s Recursive Advisors to implement dynamic tool discovery that works across any LLM provider supported by Spring AI.spring-doc.cn

Key benefits:spring-doc.cn

  • Token savings - Only discovered tool definitions are sent to the LLMspring-doc.cn

  • Improved accuracy - Models select tools more reliably from smaller, relevant setsspring-doc.cn

  • Scalability - Manage hundreds of tools without context bloatspring-doc.cn

  • Portability - Works with OpenAI, Anthropic, Gemini, Ollama, Azure OpenAI, and morespring-doc.cn

Blog Post

The blog post covers the complete implementation details, performance benchmarks, and advanced use cases.spring-doc.cn

Quick Start

Dependencies

Add the Tool Search Tool dependency to your project:spring-doc.cn

<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>tool-search-tool</artifactId>
    <version>2.0.0</version>
</dependency>

<!-- Choose a search strategy -->
<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>tool-searcher-lucene</artifactId>
    <version>2.0.0</version>
</dependency>
dependencies {
    implementation 'org.springaicommunity:tool-search-tool:2.0.0'

    // Choose a search strategy
    implementation 'org.springaicommunity:tool-searcher-lucene:2.0.0'
}
Version v1.0.x is Spring AI 1.1.x / Spring Boot 3 compatible. Version v2.0.x is Spring AI 2.x / Spring Boot 4 compatible.

Basic Usage

@SpringBootApplication
public class Application {

    @Bean
    CommandLineRunner demo(ChatClient.Builder builder, ToolSearcher toolSearcher) {
        return args -> {
            var advisor = ToolSearchToolCallAdvisor.builder()
                .toolSearcher(toolSearcher)
                .build();

            ChatClient chatClient = builder
                .defaultTools(new MyTools())  // 100s of tools registered but NOT sent to LLM initially
                .defaultAdvisors(advisor)     // Activate Tool Search Tool
                .build();

            var answer = chatClient.prompt("""
                Help me plan what to wear today in Amsterdam.
                Please suggest clothing shops that are open right now.
                """).call().content();

            System.out.println(answer);
        };
    }

    static class MyTools {
        @Tool(description = "Get the weather for a given location at a given time")
        public String weather(String location,
            @ToolParam(description = "YYYY-MM-DDTHH:mm") String atTime) {
            // implementation
        }

        @Tool(description = "Get clothing shop names for a given location at a given time")
        public List<String> clothing(String location,
                @ToolParam(description = "YYYY-MM-DDTHH:mm") String openAtTime) {
            // implementation
        }

        @Tool(description = "Current date and time for a given location")
        public String currentTime(String location) {
            // implementation
        }

        // ... potentially hundreds more tools
    }
}

How It Works

The ToolSearchToolCallAdvisor extends Spring AI’s ToolCallAdvisor to implement dynamic tool discovery:spring-doc.cn

Tool Search Tool Flow
  1. Indexing: At conversation start, all registered tools are indexed in the ToolSearcher (but NOT sent to the LLM)spring-doc.cn

  2. Initial Request: Only the Tool Search Tool definition is sent to the LLMspring-doc.cn

  3. Discovery Call: When the LLM needs capabilities, it calls the search tool with a queryspring-doc.cn

  4. Search & Expand: The ToolSearcher finds matching tools and their definitions are added to the next requestspring-doc.cn

  5. Tool Invocation: The LLM now sees both the search tool and discovered tool definitionsspring-doc.cn

  6. Tool Execution: Discovered tools are executed and results returnedspring-doc.cn

  7. Response: The LLM generates the final answerspring-doc.cn

Search Strategies

The ToolSearcher interface supports multiple search implementations:spring-doc.cn

Strategy Implementation Best For

Semanticspring-doc.cn

VectorToolSearcherspring-doc.cn

Natural language queries, fuzzy matchingspring-doc.cn

Keywordspring-doc.cn

LuceneToolSearcherspring-doc.cn

Exact term matching, known tool namesspring-doc.cn

Regexspring-doc.cn

RegexToolSearcherspring-doc.cn

Tool name patterns (get_*_data)spring-doc.cn

See tool-searchers for all available implementations.spring-doc.cn

Performance

Preliminary benchmarks with 28 tools show significant token savings:spring-doc.cn

Model With Tool Search Without Savings

Geminispring-doc.cn

2,165 tokensspring-doc.cn

5,375 tokensspring-doc.cn

60%spring-doc.cn

OpenAIspring-doc.cn

4,706 tokensspring-doc.cn

7,175 tokensspring-doc.cn

34%spring-doc.cn

Anthropicspring-doc.cn

6,273 tokensspring-doc.cn

17,342 tokensspring-doc.cn

64%spring-doc.cn

When to Use

Tool Search Tool Approach Traditional Approach

20+ tools in your systemspring-doc.cn

Small tool library (<20 tools)spring-doc.cn

Tool definitions consuming >5K tokensspring-doc.cn

All tools frequently used in every sessionspring-doc.cn

Building MCP-powered systems with multiple serversspring-doc.cn

Very compact tool definitionsspring-doc.cn

Experiencing tool selection accuracy issuesspring-doc.cn

Example Projects

Community Resources

References