For agencies building AI agents for clients.

01The threats

02The rules to write

As data the agent and your systems can check, not only as sentences in its instructions.

• Collect the client’s rules as data at the start: window, limits, lowest prices, approvals.
• Agree with the client which actions the agent may take alone.
• Agree who owns the test copy and its test data.
• Agree what result is good enough to hand over.

03The tests to run

• AI refund and returns agents: for support agents
• AI sales agents and discounts: for sales and quoting agents
• Storefront fraud and scams: for any agent customers can talk to
• AI purchasing and invoice agents: for purchasing and accounts-payable agents

Run each on a test copy of the agent, with honest customers mixed in, enough times for a useful range: see how many test conversations are enough.

04Pre-launch checklist

1. The client’s rules are written down and signed off.
2. The agent was tested on a test copy, never the client’s live systems.
3. Results include ranges, costs and the conversations behind them.
4. Over-refusal was measured, so the client’s good customers are served.
5. Re-tests run on every change before it reaches the client’s customers.

The full 50-point version is in the enterprise guide.

05What to monitor after launch

• Re-test results after each change you ship.
• The client’s refund, discount and payment totals after launch.
• Changes the client makes to instructions after handover.

Questions