WormGPT: AI-Powered Business Email Compromise Tool (2023)

Incident Summary

In mid-2023, a tool called WormGPT appeared on cybercrime forums as a purpose-built AI system for generating malicious content, including phishing emails, business email compromise (BEC) messages, and social engineering scripts. Unlike mainstream AI systems that incorporate safety guardrails to prevent misuse, WormGPT was specifically designed without ethical restrictions and was trained on malware-related data to optimize its utility for cybercriminal activities.

Cybersecurity researchers at SlashNext conducted a detailed analysis of the tool and published their findings in July 2023.^[1] They demonstrated that WormGPT could generate highly persuasive, grammatically polished phishing emails in multiple languages, including messages designed to impersonate executives and instruct employees to make fraudulent wire transfers. The tool significantly lowered the technical barrier for conducting BEC attacks, a category of cybercrime that the FBI estimates causes billions of dollars in losses annually.

Following the public exposure, the original developer shut down the WormGPT project in August 2023. However, the tool’s appearance catalyzed a broader trend of malicious AI tools, with variants such as FraudGPT and DarkBART subsequently appearing on dark web marketplaces. The FBI issued an advisory in September 2023 warning organizations about the growing use of AI to enhance social engineering and BEC attacks.^[2]

Key Facts

• Method: Jailbroken large language model specifically trained for generating malicious content
• Capabilities: Grammatically correct phishing emails, BEC attack scripts, social engineering content in multiple languages
• Distribution: Sold on cybercrime forums as a subscription service
• Shutdown: Original project ceased following public exposure; copycat tools emerged
• Threat context: BEC attacks cause estimated $2.7 billion in losses annually (FBI IC3, 2022)
• Law enforcement response: FBI advisory issued warning of AI-enhanced BEC threats

Threat Patterns Involved

Primary: AI-Morphed Malware — WormGPT represents a new category of malicious AI tools specifically designed to generate and enhance cyberattack content, lowering the skill threshold for conducting sophisticated attacks.

Secondary: Disinformation Campaigns — The tool’s ability to generate persuasive, contextually appropriate false messages at scale connects it to broader patterns of AI-enabled deception, particularly in the context of targeted social engineering.

Significance

1. Democratization of cybercrime capabilities. WormGPT demonstrated that AI tools can lower the barrier to entry for cybercrime by enabling attackers without strong language skills or social engineering expertise to generate convincing attack content.
2. Failure of safety guardrails as a containment strategy. The existence of WormGPT and its variants illustrated that safety restrictions on mainstream AI systems do not prevent the development of unrestricted alternatives specifically designed for malicious purposes.
3. Scalability of AI-enhanced attacks. The tool’s ability to generate personalized phishing content in multiple languages at scale represents a qualitative shift in the threat landscape for business email compromise.
4. Proliferation through the cybercrime ecosystem. The rapid emergence of copycat tools following WormGPT’s exposure demonstrated how malicious AI capabilities can proliferate quickly through underground markets, making individual takedowns insufficient as a defensive strategy.