How has generative AI affected security? Fake invoices, cloned voices and spoofed email

How has generative AI affected security for a company that pays hundreds of suppliers a month? It took away the warning signs. A fake invoice or bank-change request now reads perfectly, a cloned voice can sound like your finance chief, and a spoofed email can carry your exact sender address. Looking right is no longer proof.

The answer is not a sharper eye. It is controls that don't care how a request looks or sounds: a call-back for bank changes, a second approver, approvals inside the payables system, sender authentication and training for the people who release payments. For the email records in depth, see our guide to DMARC policy not enabled.

What changed when generative AI arrived?

Mostly the quality, not the scam. Business email compromise, fake supplier invoices and urgent wire requests in an executive's name long predate these tools. What changed is how good, fast and cheap the fakes became.

A December 2024 FBI public service announcement put it plainly: "Generative AI reduces the time and effort criminals must expend to deceive their targets." The tools "can correct for human errors that might otherwise serve as warning signs of fraud." FinCEN's 2024 alert on deepfake media adds that generative AI can often produce content that is hard to tell from the real thing, and that criminals have reportedly used it to impersonate executives and order large transfers.

In its 2025 Internet Crime Report, the FBI's Internet Crime Complaint Center counted more than 22,000 complaints reporting AI-related information, with adjusted losses over 893 million dollars. Business email compromise drew about 3.05 billion dollars in reported losses, second only to investment fraud. Not every case is AI-enabled, the FBI notes; businesses reported more than 30 million dollars lost to the ones that were.

How do fake invoices and bank-change requests work now?

They fit into a process you already run. FinCEN's advisory on email fraud aimed at business processes describes criminals who study a company's vendors, contracts and payment routines, then "insert themselves into communications by impersonating a critical player in a business relationship or transaction." It tied fake vendor invoices to larger amounts than even fake requests from a chief executive, likely because supplier payments are large and recurring.

A made-up example, not a client: an email arrives from your packaging supplier's billing lead. It quotes a real purchase order, matches the supplier's tone and attaches an invoice in its usual layout, plus a letter announcing a new bank account. A writing tool polished it in seconds. Nothing is wrong except the account number.

In a group with several entities, a shared payables mailbox and vendor records in more than one system, one email can be enough to change a bank account.

Can a cloned voice really approve a payment?

Yes, if your process accepts a voice as approval. The FBI's 2024 notice says criminals use AI-generated audio to impersonate people and get into bank accounts, and create video "for real time video chats with alleged company executives." A May 2025 FBI warning on impersonation campaigns says a real voice and an AI clone "can sound nearly identical," and the 2025 report notes that voice cloning "can also be used to request wire payment."

Caller ID doesn't settle it. CISA's phishing guidance notes that attackers use internet phone services to spoof caller ID. So two familiar checks have quietly stopped working: "the finance chief called to confirm it" and "I called the number in the email." Both let the attacker choose who answers.

Why does a spoofed sender address still get through?

Because email on its own lets anyone type any address into the From line. Three public DNS records decide whether receiving servers believe it:

• SPF lists the servers allowed to send mail for your domain.
• DKIM signs each message, proving it came from your domain unchanged.
• DMARC checks that the visible From address matches what SPF or DKIM proved, and tells receivers what to do when it doesn't: nothing (p=none), the spam folder (p=quarantine) or refuse it (p=reject).

CISA says to set DMARC to reject for the mail you send, which "provides robust protection against other users receiving emails that impersonate a domain," and to check DMARC on incoming mail too.

Two limits. DMARC covers exact domains only, so a look-alike domain still needs the call-back rule. And a group with several entities, old brand names and parked domains needs records on every one. Our free email security check reads all three records for any domain in seconds.

Which controls work at company scale?

The ones that don't depend on spotting a fake. FinCEN frames the job as hardening how you authenticate the people in a conversation, authorize transactions and communicate changes. That means five controls.

Control	What it stops	Owner	Where it lives
Call-back on a known number for any bank change	Fake bank changes, cloned voices	Payables	Vendor master file
Two-person approval above a set limit	One person fooled by one message	Controller	Approval matrix
Approval paths in the payables system	Approvals by email, chat or phone	Finance, with IT	Payables or accounting system
SPF, DKIM and DMARC at reject on every domain	Email faking your exact domain	IT	Public DNS
Training for payment approvers	Urgency and authority tricks	Finance and IT	Onboarding, yearly refreshers

1. Call-back on a known number. The FBI's business email compromise guidance is short: "Use secondary channels and/or two-factor authentication to verify requests for changes in account information." Make it a rule with no exceptions: the call goes out, to a number already in the vendor master file or company directory, never one in the request. Log the call, and hold the first payment to new bank details until it is logged.

2. Two-person approval above a limit. NIST describes separation of duty as the principle that no user should have enough privileges to misuse the system on their own, and names the two-person rule as an example. Above your limit, a second approver who did not handle the request signs off. Bank changes get two people at any amount.

3. Approval paths in the payables system. If an approval can arrive by email, a fake email can approve. Move approvals, bank changes and payment releases into the payables or accounting system, routed by amount, entity and vendor, with every step logged. Our guide to accounts payable automation covers the approval matrix.

4. Sender authentication. IT publishes SPF and DKIM for every service that sends as the company and moves DMARC to reject in stages.

5. Training for approvers. CISA recommends regular training on social engineering and phishing, and notes that staff in hybrid setups, with fewer face-to-face exchanges, are more likely to be deceived. FinCEN pairs training with a "multi-faceted transaction verification process." Train the people who can change a vendor or release money, plus the executives whose names get used. Teach one idea: a perfect email, voice or video is not proof.

What is a key security concern when using generative AI inside the company?

Attackers are only half of it. NIST's Generative AI Profile names two primary information security risks: the tools lower the barrier for attacks such as phishing, and they widen the attack surface, because they are themselves "vulnerable to attacks like prompt injection or data poisoning." It also lists data privacy: leaks and unauthorized disclosure of sensitive data.

That means two working rules. First, staff don't paste bank details, payroll, contracts or customer records into public tools; our guide to AI training for employees sets out simple data rules. Second, treat anything an AI tool reads, such as incoming invoices or email, as untrusted. NIST describes indirect prompt injection, where instructions hidden in retrieved content make a tool behave in unintended ways. So the tool can read, sort and draft, but never change bank details or release a payment.

What should you do if a payment already went out?

Move fast. The FBI says "time is of the essence": call your bank, ask for a recall of the funds and file a complaint at ic3.gov with full transaction details. FinCEN has reported more success recovering funds when fraudulent wires are reported to law enforcement within 24 hours. In 2025, the FBI's Recovery Asset Team froze about 679 million of roughly 1.16 billion dollars in attempted theft, a 58 percent success rate. Put these steps and your bank's fraud contact in your procedures now.

How can Raileon help?

Start with the free email security check: it reads a domain's SPF, DKIM and DMARC records from public DNS, with no sign-up. If the records need work, we set up SPF, DKIM and DMARC, then check your real email still gets delivered.

On Thu, Nov 5, from 12:00 to 12:45 PM ET, we run a free live class on fake invoices and email spoofing, on raileon.com. You check your own domain privately and leave with a one-page sheet for your IT team with the 4 steps that fix it, and a card with 3 payment checks.

Our Business Audit, reviewed by a person, includes a security and email section. When the gap is in the process, Raileon designs, builds and runs custom systems with approvals inside the tools a business already uses. In payables work, every action is recorded, and payments always wait for a person's approval. Monthly care starts from $1,999 a month. Cancel any time. Your plan runs to the end of the month you've paid for. No exit fee.

FAQ

How has generative AI affected security?

It has made fake invoices, emails and voices convincing enough that looking right is no longer proof. The controls that still work verify a request through a separate channel and a second person, not by how it looks or sounds.

Does DMARC stop cloned voice calls?

No. DMARC stops email that fakes your exact domain, so phone and video requests still need a call-back to a number you already have on file.

Can staff reliably spot an AI-generated fake?

Not reliably. The FBI says AI-generated content is often difficult to identify, so train approvers to verify every bank change and urgent payment request the same way, however real it seems.

What should a finance team do first?

Write down the call-back rule for bank changes and check every domain the company owns. Then move approvals out of email and into the payables system, with a second approver above a set limit.

Check your domains before the next payment run

Run the free check on each domain, then bring the results to a free 20-minute call. We'll go through your payment approvals and what to fix first.

Book a free 20-minute call