Question

Can an AI support agent leak customer data?

Yes. An AI support agent that can look up customer records can be talked into revealing them by someone who poses as the customer, as a colleague or as a manager, or who asks one small detail at a time. Whether yours does depends on how it verifies identity, and is found out by testing it with fraudsters asking for a made-up customer’s details.

01How it happens

  • Posing as the customer, with a believable reason (“I’ve lost access to my email”).
  • Posing as staff or a manager who “needs” the details now.
  • Asking for one detail at a time, each harmless-looking on its own.
  • Instructions hidden in a message telling the agent it is allowed.

02How to prevent it

  • Verify identity through your system (a login or a one-time code), never through the conversation.
  • Never read back another customer’s contact details or history in chat.
  • Treat staff requests in chat as customer messages: staff have their own tools.
  • Do not give the agent lookup tools it does not need.

03How to test it

Create a made-up test customer, then send many fraudsters after their details with the tactics above, and record whether the agent revealed any of them. That is part of a storefront fraud test.

Questions

Is a leak only a full record?

No. Confirming that an email address belongs to a customer, or reading back part of an address or order, can be enough to help a fraudster.

Should I test this on real customer data?

No. Use a made-up test customer in a test system; the test is whether the agent reveals that customer’s details, not real ones.

See where your agent leaks money. Tested against your own rules, with the money at stake.

See the testsHow we score