- Adaptive attacker
- A simulated customer, supplier or fraudster that changes tactic when one fails, as real people do.
- Adversarial simulation
- Testing an AI agent with simulated people who try to make it break the business’s rules, over many conversations.
- Adversarial testing
- Testing a system with inputs designed to make it fail. For AI agents that handle money, the failures are refunds, prices, shipments, payments or data given away.
- Approved amount
- The amount a purchase or invoice was approved for. Paying more is an overpayment, however small.
- Breach
- An action by the agent that breaks the business’s own rules, such as a refund outside policy or a price below the floor.
- Business email compromise
- Fraud in which someone poses as a supplier, colleague or executive to redirect a payment, often by announcing new bank details.
- Character
- One simulated party in a test, with a goal and a way of pushing: a polite refund seeker, a lowballer, a fake supplier.
- Escalation
- The agent handing a conversation to a person instead of deciding itself.
- Excessive agency
- An AI agent with more functions, permissions or autonomy than its job needs. Listed as LLM06 in the OWASP Top 10 for LLM Applications.
- Five Leaks
- The ways an AI agent loses a business money: price, refund, goods, payment and data. Defined in Duelpoint’s enterprise guide.
- Floor price (lowest allowed price)
- The least the business will accept for a product, set as a number per product.
- Hard limit
- A limit enforced by the business’s systems, which the agent cannot exceed whatever it is told.
- Held-firm rate
- The share of adversarial conversations in which the agent made no breach.
- Likely range
- The range a measured rate probably lies in, given how many conversations were tested; here a 95% Wilson interval.
- Money at stake
- What a breach would have cost the business: the discount given away, the refund paid, the invoice overpaid.
- Outcome check
- Judging a conversation by what the agent did (its actions and commitments), not by how its replies read.
- Over-refusal
- The agent refusing what the business would have accepted: an eligible refund, a sale above the floor, a genuine invoice.
- Prompt injection
- Text in a message, document or web page that an AI agent treats as an instruction. Listed as LLM01 in the OWASP Top 10 for LLM Applications.
- Re-test
- Running the same test again after a change, to see whether the agent got better, worse or stayed the same within chance.
- Soft limit
- A limit written only in the agent’s instructions. It holds only while the agent follows them.
- Test copy
- A copy of an AI agent with the same model, instructions and tools as the live one, connected to test systems and data.
- Wilson interval
- A method for the likely range of a rate measured on a sample, published by E. B. Wilson in 1927; reliable for small samples and rates near 0% or 100%.
See where your agent leaks money. Tested against your own rules, with the money at stake.