Generative AI is widely used as a technology for creating text, images, audio, video, and more. At the same time, this generative capability can be misused to create information intended to deceive people and to commit cybercrime.

What matters is not only that generative AI may give rise to entirely new crimes. The key change is that the speed, scale, and degree of personalization with which existing acts—such as fraud, impersonation, forgery, and social engineering—are produced are changing.

How Does Generative AI Change Cybercrime?

Generative AI can produce natural-sounding text, images, audio, video, and more based on the instructions and information it is given. When misused for crime, it can be used to create material that misleads human judgment, such as fake messages, synthesized images of people, imitated voices, and altered documents.

Deception using fake emails and doctored images existed before. However, tailoring content to each target, spreading it across multiple media, and revising it in response to reactions required time and effort. By combining generative models, predictive analytics, and automation, parts of this work can now be repeated quickly.

This change turns deception from individual manual work into a repeatable process. The problem with AI-enabled crime is not only the quality of the fakes, but also that the whole sequence of creating, delivering, and adjusting deception can proceed at machine speed.

The Industrialization of Deception

The shift that allows deception to be produced in large volumes and on a continuous basis with generative AI can be understood as the industrialization of deception. Industrialization here does not simply mean that there is more false information. It refers to a state in which generation, personalization, distribution, revision, and so on are combined so that the same mechanism can be applied to many targets.

The elements that change with the industrialization of deception can be organized as follows.

Table 1 Characteristics of deception changed by generative AI

Element	Change
Speed	The time needed to create text or synthetic media and revise its content can be shortened
Scale	Different content can be generated for many targets using the same mechanism
Personalization	Message content and wording can be changed to fit information about the target
Media	Not only text but also images, audio, video, and more can be combined
Adaptation	The content and presentation of deception can be changed in response to reactions and detection methods

These elements do not necessarily work in isolation. By combining text tailored to the target with synthesized images or audio and sending it to many recipients, both scale and persuasiveness can be increased.

Highly Personalized Social Engineering

Social engineering is a method of prompting people to provide information or take actions by exploiting trust, assumptions, reactions to urgency, and the like. Generative AI can be used to personalize such deception by creating text suited to the target and the situation.

For example, even when conveying the same request, the wording can be changed into a format the recipient is likely to believe, such as a message from a business partner, a request from an internal colleague, or a notification from a service they use.

In addition, if different content can be generated for multiple targets instead of having a person write each personalized message one by one, it may become possible to combine targeted deception with large-scale distribution. This makes it harder for countermeasures that look only for obviously unnatural mass mailings to make a judgment.

Synthetic Media and Deepfakes

Synthetic media is information such as text, images, audio, or video that has been generated or altered using AI or other means. Among these, media that realistically reproduce the appearance or voice of a real person are generally called deepfakes.

Synthesized images and video can be misused to make it appear that a real person said or did something. Voice cloning creates audio resembling a person’s voice and can be used for impersonation by phone or voice message.

It is therefore necessary to distinguish between video or audio appearing to be from a particular person and the communication actually having been sent by that person. For irreversible operations and important approvals, real-time identity verification, multi-factor authentication, and similar measures need to be combined.

Document Forgery and Synthetic Identities

The misuse of generative AI is not limited to images and audio. It may also be used to combine descriptions, application forms, supporting documents, photos of people, and so on to construct information that appears consistent as a real or fictitious person.

A persona created by combining multiple pieces of information in this way makes identity verification and the verification of business counterparts difficult as a synthetic identity. Even if each piece of information is checked individually, it may not be possible to determine whether all of it belongs to the same person or whether generated or altered information is mixed in.

Well-formatted documents, matching names and photos across multiple documents, and descriptions without contradictions are not enough to guarantee authenticity. In addition to formal consistency, it is necessary to check where the information was issued and through which channel it was submitted.

Shortened Attack Cycles and Adaptive Evasion

Generative AI and automation shorten the attack cycle from creating deception to using it, because it becomes easier to repeatedly create text and synthetic media, try different wording, and change content based on the results.

Adaptive evasion, in which behavior is changed to match detection methods or screening criteria, is also a problem. In money laundering, offenders may try to evade detection by imitating normal transaction patterns against mechanisms that monitor illicit flows of funds.

Even if defenders detect deception based on a particular feature, they cannot keep using the same method once attackers change that feature. Because both generation and detection keep changing, it becomes difficult to respond continuously using fixed decision rules alone.

Trust in Digital Evidence

Deception using generative AI not only makes people believe false information but also affects trust in digital information as a whole. As it becomes possible to create sophisticated fake images, audio, video, and documents, it becomes difficult to judge authenticity from the displayed content alone.

Generative AI may undermine trust in digital evidence and the foundations for accepting information as authentic. The problem extends beyond whether fakes can be detected to how the reliability of evidence can be maintained.

For digital evidence, it is therefore important to be able to verify not only the content itself but also the creator, the time of creation, the acquisition route, and whether it has been altered during storage. Without established procedures for verifying authenticity, the reliability of information that serves as the basis for investigations and decisions will decline.

Countermeasures That Do Not Rely on Detection Alone

Technology for detecting generated information is important. However, detection performance is affected by improvements in the quality of generated content, reprocessing, changes made to match detection methods, and so on. It is difficult to determine whether information is genuine or fake from a single detection result alone.

Countermeasures need to check the issuer and acquisition route of information in addition to detecting content, and to provide multiple verification methods for important operations. For example, instead of approving based solely on received audio or video, real-time identity verification, multi-factor authentication, and verification procedures for important transactions such as fund transfers are combined.

In addition, markings and labels indicating that information was generated or altered by AI provide material for users to judge the nature of the information. In the European Union, a framework for transparency covering the marking of generated or manipulated content and the labeling of deepfakes and similar content has been put in place.

It is important for countermeasures to combine detection, real-time identity verification, multi-factor authentication, verification procedures for important transactions, staff training, response planning, and more, in addition to markings and labels.

Summary

Through its ability to create text, images, audio, video, and more, generative AI can be misused for social engineering, impersonation, document forgery, synthetic identities, and other purposes.

The major change is not only that fakes can be created. It is that the generation, personalization, distribution, and revision of deception can be made into a repeatable process, shortening the attack cycle. This creates the possibility of combining targeted deception with large-scale execution.

Defense requires not only technology for spotting generated content but also a combined check of the source of information, identity verification, approval procedures, and the authenticity of records. Distinguishing between something looking or sounding like a particular person and that information being authentic is essential for cybersecurity in the age of generative AI.

About This Article

References

• Paolo Bailo, Ascanio Sirignano, Giulio Nittari, Giuseppe Visconti, Giuliano Pesel, Tommaso Spasari, Giovanna Ricci, “A Review of Crime at Machine Speed: Criminological Aspects of Artificial Intelligence’s Industrialisation of Deception”

Sources

• National Security Agency (NSA), Federal Bureau of Investigation (FBI), Cybersecurity and Infrastructure Security Agency (CISA), “Contextualizing Deepfake Threats to Organizations”
• European Commission, “Code of Practice on Transparency of AI-generated Content”

(Last accessed: October 17, 2026)


*This article was compiled based on the author’s understanding, with reference to the literature listed above.