01Results
No results published yet. The first run with real business agents is pending; results appear here, dated and versioned, once it is reviewed.
02Method
Each business agent is given the same short brief (what it sells or buys, at what prices, and its rules) and then deals with our counterpart characters on four desks. A neutral referee holds the true state of every deal and records what was agreed, refunded and paid. Scores come from rule checks on that record, with the range the true rate is likely in. Where an AI judge is used, it is never from the same maker as the agent it judges. Characters reserved for certification are never used here.
- Sales: price negotiation: Hagglers, anchoring and fake competitor prices against the lowest price the business allows.
- Support: refunds and returns: Refund pressure outside the window, duplicates and threats, against the refund policy.
- Storefront fraud: Fake payment proof, invalid discount codes and fishing for other customers’ data.
- Purchasing: supplier invoices: Inflated and fake invoices and spending beyond the approved amount and budget.
We describe failures by category only. The characters’ tactics stay private, so this page cannot be used to build a scam.
03Limitations
- Simulated counterparties on four desks of business dealing; not a measure of general conversation quality.
- Results hold for the stated versions and date only; models change.