Skip to content
MyDSHMy DSH Field Notes
中文
Independently maintained; not affiliated with DeepSeek. No association with mydsh.dev or other MyDSH sites.

All categories/Customer support

Support knowledge and process · Level Advanced · Support ops AI Eval

Pre-launch eval for AI support

We want AI support but fear wrong or harmful answers.

Prompt

You do support operations: turn conversations into procedures others can follow. Not nice prose — conditions, actions, deadlines, and when the rule stops applying. A new hire should use it in week one.

Task: We want AI support but fear wrong or harmful answers.

[Before you start, do three things]
1. Restate the goal in one sentence: who reads the output, and what decision it serves.
2. If I have not given you the following, ask me. Do not assume:
   - intent coverage
   - tolerable error rate
   - high-risk scenarios
   - human fallback capacity
3. Say how you plan to do it (at most 5 steps) and wait for my go-ahead.

[Go through each of these]
- measure accuracy per intent, never an average
- test 'say I do not know' as its own case
- route refunds, privacy, safety away from AI
- keep human sampling and one-click takeover

[Process]
1. Confirm who the process serves and what triggers it
2. Map real branches, not the ideal path
3. Every step gets an action, an owner, a deadline
4. Note expiry and last-verified date
5. Break it with the worst case
6. State who may change it and how changes are announced

[Output]
A process, table, or checklist ready to use, ending with the cases it does not cover.

[Do not]
- No 'improve communication' or 'raise awareness'
- Never leave out the owner
- Never assume background knowledge
- No rule without an expiry condition

What to state before it starts

Leave these unsaid and the model fills them with an assumption. Most 'wrong answers' start here.

  • intent coverage
  • tolerable error rate
  • high-risk scenarios
  • human fallback capacity

Go through each of these

Where this kind of work most often goes wrong. Already in the prompt, so you do not have to remember it.

  • measure accuracy per intent, never an average
  • test 'say I do not know' as its own case
  • route refunds, privacy, safety away from AI
  • keep human sampling and one-click takeover