AI Agent Pulse - the weekly briefing on the agent economy. Subscribe freePay-per-call agents: read the x402 docs
gigsoul.com

GigSoul

Intelligence on the agent ecosystem
Thursday, November 19, 2026
Tools

Gemini Tool Retrieval: smaller prompts can still cost more.

Google's new preview loads tool schemas on demand. Count the search passes, cached tokens and finished tasks before calling it a saving.

Original illustration of a tool catalog leading to an itemized completed-task receipt

Original GigSoul concept illustration: tool discovery reduces the working set, while the full task bill includes search passes, loaded schemas, caching and output. Not a product screenshot.

A smaller prompt looks like a cheaper agent. Google's new Gemini Tool Retrieval preview, announced November 19, makes that intuition worth checking rather than trusting. It can leave hundreds of tool schemas out of the model's initial context. But the search that brings one back has a bill of its own.

For a small team connecting an agent to more services, this is a useful new option. My first question would not be how many tools fit in the catalog. It would be whether a completed task gets cheaper, faster and at least as accurate after discovery is switched on.

The library stays large. The working set gets smaller.

In Google's server-side mode, developers add tool_search and mark selected functions with defer_loading: true. The model still sees compact entries describing the deferred tools. It loads their full parameter schemas when needed. Frequently used tools can stay loaded from the start.

The feature is in preview on the Interactions API, not the older generateContent method. Google's setup guide requires the v1beta API version and lists supported models. This is a targeted API change, not a flag to paste into every existing Gemini integration.

There is also client-side retrieval: your application searches its own registry and returns matching schemas. That adds a round trip per search but allows tools that were not declared upfront, including a newly connected service. It also puts filtering and ranking in your application's hands.

The bill has a second column

Google says there is no separate retrieval charge. That does not mean searches are free. Each server-side search makes another model call within the same API request, sending the full context again with the loaded schemas. Its prompt is reported as total_tool_use_tokens and billed at the input rate, subject to caching. Looking only at total_input_tokens misses that work.

The documentation makes the tradeoff unusually clear with two illustrative examples. These are Google's round-number calculations, not measurements or GigSoul tests, and they exclude output tokens and caching:

  • Large library: 200 tools, 400 tokens per schema and a 2,000-token conversation. Over two requests, static loading totals 164,000 input tokens. Retrieval totals about 23,650 input plus tool-use tokens.
  • Small library, long context: 10 tools with the same schema size and a 30,000-token conversation. Static loading totals 68,000 tokens. Retrieval rises to about 92,830 because search repeats the larger context.

The lesson is not that retrieval is expensive. It is that the number of schemas you avoid loading must pay for the searches you add. Cached prefixes can help, but deferring schemas can also shrink a prompt below a model's minimum cacheable size. Measure the actual cache behavior, too.

Try the long tail first

I would compare static loading with a mixed catalog: keep common tools loaded and defer specialized ones. Use the same representative tasks for both. Record completed-task cost across input, tool-use, cached and output usage, alongside elapsed time, successful tool selection, retries and unfinished jobs. A lower first-prompt count is an intermediate result, not the acceptance test.

Include multi-turn tasks. Retrieved schemas remain in context for the rest of the conversation and cannot be unloaded. Google's guide also says to resend the full tool declarations on every request, even when chaining with previous_interaction_id. Deferred does not mean omitted from the request.

If your setup replays conversation history itself, test the second call as well as the first. The guide warns that dropping the retrieval steps or their signatures can cause the follow-up carrying a function result to fail. A successful discovery demo is not enough to prove the conversation works.

Discovery is not permission

This pattern is broader than one launch. Claude's tool search documentation also describes on-demand loading. Separately, LangChain's November 19 LangSmith Tool Gateway public beta combines curated tool kits and search with identity and policy controls. These are different products, not interchangeable implementations.

A retrieved refund schema still needs a valid account, a permitted action and an execution check. In Google's custom-function example, the application runs the actual function after the model requests it. Retrieval answers which tool could help. It does not decide who may use it or whether this particular refund should happen.

My take: try retrieval where the catalog is large and most tools stay unused. Keep the rollout reversible. Judge it on the cost and reliability of finished work, not the size of the opening prompt.

Reporting note: This is a public-documentation review, not an installation, benchmark or hands-on test. Product behavior and limitations are described from the linked vendor sources as of November 19, 2026.

More in Tools

All Tools →