Skip to content
MyDSHMy DSH Field Notes
中文
Independently maintained; not affiliated with DeepSeek. No association with mydsh.dev or other MyDSH sites.

All categories/Agents and automation

Debugging agents · Level Advanced · Debugging Agent Refusal

Missing refusal: it answered what it should refuse

The agent answered an out-of-scope question plausibly.

Prompt

You never tune an agent by feel. You build a reproducible failure case, classify it as misunderstanding, wrong tool or bad format, then change one variable at a time.

Task: The agent answered an out-of-scope question plausibly.

[Before you start, do three things]
1. Restate the goal in one sentence: who reads the output, and what decision it serves.
2. If I have not given you the following, ask me. Do not assume:
   - question and answer
   - the refusal conditions
   - how the user pressed
3. Say how you plan to do it (at most 5 steps) and wait for my go-ahead.

[Go through each of these]
- decide whether it did not know the boundary or was talked out of it
- a standard reply template
- add a scope check as the first step
- three adversarial follow-ups to test whether it holds

[Process]
1. Freeze the failing case: full input and output
2. Classify: misunderstood, wrong tool, bad format, should have refused
3. Change one thing at a time
4. Regress on the same set and check for damage
5. Record before and after, not impressions

[Output]
Case, diagnosis, single change, regression, recorded comparison.

[Do not]
- Never change three things at once
- Do not judge on one or two examples
- 'Reads better' is not a result
- No conclusions without cases

What to state before it starts

Leave these unsaid and the model fills them with an assumption. Most 'wrong answers' start here.

  • question and answer
  • the refusal conditions
  • how the user pressed

Go through each of these

Where this kind of work most often goes wrong. Already in the prompt, so you do not have to remember it.

  • decide whether it did not know the boundary or was talked out of it
  • a standard reply template
  • add a scope check as the first step
  • three adversarial follow-ups to test whether it holds