Tool Instructions Are Like Device Manuals: Read When You Need Them

Maximizing Tool Use Performance in Agentic AI Systems

TL;DR: Move tool usage instructions from system prompts into tool definitions themselves. This improves reliability, reduces cognitive load, and simplifies agent architecture.

Building reliable LLM agents means getting tool usage right. As we developed our agent system, we discovered a simple but powerful principle: Tool usage instructions should live with the tool implementation, not scattered in global configuration.

This approach solves a common problem: poor tool organization leads to unreliable agents and debugging nightmares. When instructions are scattered across system prompts, agents struggle with tool selection, fail to follow procedures consistently, and become harder to debug as complexity grows.

The Problem: Instructions Far from Implementation

When building our agent, we put all tool guidance in the system prompt. One place for all the rules seemed logical. But as our toolbox grew, three problems emerged:

Unreliable tool selection: The agent struggled with deciding which tool to use and when, especially when chaining multiple tools in complex workflows.

Inconsistent instruction following: We needed specific behavior after tool execution, but getting the agent to follow procedural steps reliably was hit-or-miss.

System prompt overload: Our prompt became a mess mixing high-level agent guidance, tool usage details, and behavioral rules. Agent reasoning got harder and debugging became a nightmare.

The Problem in Practice

Our plan generation workflow showed this issue clearly. We had a generate_plan() tool that created plans for users, followed by a human-in-the-loop step for feedback. If the user confirmed the plan, we needed the agent to trigger the save_plan() tool.

Our system prompt approach:

After generating a plan, ask the user if they want to save it.
If they confirm, use the save_plan tool once.
Do not call generate_plan or save_plan multiple times.

What actually happened:

• Confirmation messages were sent twice, confusing users

• Sometimes the user wasn't asked to save the plan at all

• The agent would call generate_plan or save_plan multiple times in a single conversation

• Duplicate tool calls created inconsistent state and confused workflows

These issues happened frequently enough that we built explicit routing logic and conditional nodes to prevent duplicate tool calls. Our graph structure became unnecessarily complex.

The Solution: Colocate Instructions with Implementation

Instead of burying tool guidance in shared system prompts, we moved instructions directly to the tools.

Most agent frameworks support embedding usage instructions within tool definitions through docstrings, descriptions, or metadata fields.

Tool definitions with embedded instructions include usage guidance directly in the tool schema through descriptions and documentation, which gets packaged into the tool's metadata. Use the documentation to define the tool's purpose, what it should do, and what it shouldn't do:

def generate_plan(requirements: str) -> str:
    """Generates a plan based on user requirements.

    IMPORTANT: Only call this tool once per conversation unless the user
    explicitly requests changes. After generating, ask the user once:
    'Would you like me to save this plan?'
    """
    # plan generation logic
    return plan

def save_plan(plan_data: str) -> str:
    """Saves a plan to the database.

    IMPORTANT: Only call this once after user confirmation.
    Do not call if already executed in this conversation.
    """
    # save logic
    return "Plan saved successfully"

Tool registration connects these tools to your model, making the embedded instructions available during tool selection:

# Register tools with their embedded instructions
tools = [generate_plan, save_plan]
agent = Agent(model, tools)

# The agent now has access to usage instructions at decision time
workflow.add_node("agent", agent)

This approach enforces usage boundaries when the agent selects which tool to use, rather than hoping the LLM remembers scattered instructions from prompt history.

The Results

Moving instructions to the tools themselves delivered immediate improvements:

Higher reliability: The duplicate tool calls and confirmation issues became rare occurrences instead of regular problems. Tool usage became significantly more predictable overall.

Reduced cognitive load: The agent reasoned more effectively without reconciling conflicting instructions from a bloated system prompt.

Simpler workflow structure: We eliminated the explicit routing nodes and conditional logic needed to prevent duplicate tool calls.

The workflow constraints moved into the tools themselves, making our agent architecture cleaner and easier to maintain.

This approach follows software engineering best practices of keeping related code and documentation together, which in our experience led to more predictable agent behavior.

Think of it like reading a device manual: you could read the entire manual from cover to cover before using any device, but it's much more effective to consult the relevant section when you actually need to use a specific feature. Similarly, agents perform better when they receive tool-specific instructions at the moment of tool selection, rather than trying to remember scattered guidance from earlier in the conversation.

The Bigger Picture: Encapsulation for Agent Systems

This approach mirrors the software engineering principle of encapsulation—bundling data with the methods that operate on it. In our case, we're bundling tools with their usage instructions.

Just as good object-oriented design keeps related functionality together, effective agent design keeps tool behavior and tool guidance in the same place. This creates natural boundaries that make the system more predictable and maintainable.

System prompts work best for stateless instructions that apply globally:

• Agent persona and tone

• High-level behavioral guidelines

• Universal rules that apply everywhere

Tool-specific instructions belong encapsulated with the tools themselves:

• Usage constraints and procedural steps

• Context-dependent guidance

• Implementation-specific behavior rules

This encapsulation provides the same benefits in agent systems as it does in traditional software: better modularity, easier debugging, and clearer separation of concerns. When you need to modify how a tool behaves, you change it in one place rather than hunting through system prompts and hoping you found all the references.

The Takeaway

Tool instructions belong with the tools, not in your system prompt. Use tool documentation and schemas for implementation-level guidance and ensure your agent framework makes those instructions available at decision time.

It's a simple change that makes your agent more reliable, your code more maintainable, and your system prompts cleaner and more focused.