Scrape, Fake, Deliver: how AI phishing actually works

Every AI-powered scam I walk through in my security class follows the same routine: scrape, fake, deliver. Once you see the pattern, you can't unsee it, and that's the point.

Scrape

They don't steal the raw material. You volunteered it.

A birthday video on Facebook gives them your voice; three seconds of audio is all a clone needs. Instagram gives them your face from a hundred angles. Your website bio names your accountant and your bank. None of it hacked. All of it public.

Fake

For twenty years, we all trained on one smoke detector: bad grammar. Weird phrasing, "kindly do the needful," obvious typos. Delete, done.

That smoke detector is dead. AI writes better emails than most humans. In Oxford research, AI-written phishing got clicked 54% of the time, against 12% for the old human-written stuff. And KnowBe4 found that 82.6% of phishing emails now contain AI-generated content.

It goes further than email. In the Arup case, a real finance employee joined a video call with what looked like the company's CFO and several colleagues. Every other participant on that call, the CFO included, was AI-generated. The employee was the only human in the meeting, and by the end of it the company had wired out $25.6 million. And before you think you'd have caught it: in a meta-analysis of 56 studies, people correctly spotted high-quality deepfakes about 24.5% of the time. That's worse than a coin flip. Your eyes are not evidence anymore.

Deliver

The last step is the one that matters most: it always comes through a door you trust. A call from "your bank." A text from "your boss." An email from "the sheriff's office."

That last one isn't hypothetical. A man in Gillette wired $11,000 after a spoofed call from the "sheriff." That's not a big-city story. Gillette is about three hours and forty-five minutes up the road from me. He wasn't stupid. He was human, and the scam was built to exploit exactly how humans respond to authority and urgency.

What still works

You can't out-stare a deepfake, so stop reading the message and read the room. Three habits:

1. Pause. Every one of these scams runs on urgency. Urgency is the tell. Real institutions almost never need money moved in the next ten minutes.
2. Verify out-of-band. Hang up and call back on a number you look up yourself. New channel, every time, no exceptions.
3. Set a safe word. A shared secret with your family and your team that no scraper can find, because it has never been written down anywhere.

Notice that none of those are technology. The defense against AI-generated fakes is a human routine that doesn't depend on trusting your own eyes and ears.

Next Thursday: the risk nobody talks about because it's boring, which is exactly what makes it dangerous.

-Thayne