All categories/Agents and automation
Debugging agents · Level Advanced · Debugging Agent Tools
Wrong tool: why that one
Five tools available and it keeps picking the wrong one.
Prompt
You never tune an agent by feel. You build a reproducible failure case, classify it as misunderstanding, wrong tool or bad format, then change one variable at a time.
Task: Five tools available and it keeps picking the wrong one.
[Before you start, do three things]
1. Restate the goal in one sentence: who reads the output, and what decision it serves.
2. If I have not given you the following, ask me. Do not assume:
- names and descriptions of the five tools
- the full sequence of one wrong choice
- typical user phrasing
3. Say how you plan to do it (at most 5 steps) and wait for my go-ahead.
[Go through each of these]
- check for semantic overlap between descriptions
- check whether a name suggests the wrong use
- three confusable questions, the right tool, and the criterion
- whether to fix descriptions or add a routing step
[Process]
1. Freeze the failing case: full input and output
2. Classify: misunderstood, wrong tool, bad format, should have refused
3. Change one thing at a time
4. Regress on the same set and check for damage
5. Record before and after, not impressions
[Output]
Case, diagnosis, single change, regression, recorded comparison.
[Do not]
- Never change three things at once
- Do not judge on one or two examples
- 'Reads better' is not a result
- No conclusions without casesWhat to state before it starts
Leave these unsaid and the model fills them with an assumption. Most 'wrong answers' start here.
- names and descriptions of the five tools
- the full sequence of one wrong choice
- typical user phrasing
Go through each of these
Where this kind of work most often goes wrong. Already in the prompt, so you do not have to remember it.
- check for semantic overlap between descriptions
- check whether a name suggests the wrong use
- three confusable questions, the right tool, and the criterion
- whether to fix descriptions or add a routing step