Microsoft Stops Phishing Scam Which Used Gen-AI Codes to Fool Victims

AI: Boon or Curse?

AI code is in use across sectors for variety of tasks, particularly cybersecurity, and both threat actors and security teams have turned to LLMs for supporting their work. 

Security experts use AI to track and address to threats at scale as hackers are experimenting with AI to make phishing traps, create obfuscated codes, and make spoofed malicious payloads. 

Microsoft Threat Intelligence recently found and stopped a phishing campaign that allegedly used AI-generated code to cover payload within an SVG file. 

About the campaign 

The campaign used a small business email account to send self addressed mails with actual victims coveted in BCC fields, and the attachment looked like a PDF but consisted SVG script content. 

The SVG file consisted hidden elements that made it look like an original business dashboard, while a secretly embedded script changed business words into code that exposed a secret payload. Once opened, the file redirects users to a CAPTCHA gate, a standard social engineering tactical that leads to a scanned sign in page used to steal credentials. 

The hidden process combined business words and formulaic code patterns instead of cryptographic techniques. 

Security Copilot studied the file and listed markers in lines with LLM output. These things made the code look fancy on the surface, however, it made the experts think it was AI generated. 

Combating the threat

The experts used AI powered tools in Microsoft Defender for Office 375 to club together hints that were difficult for hackers to push under the rug. 

The AI tool flagged the rare self-addressed email trend , the unusual SVG file hidden as a PDF, the redirecting to a famous phishing site, the covert code within the file, and the detection tactics deployed on the phishing page. 

The incident was contained, and blocked without much effort, mainly targeting US based organizations, Microsoft, however, said that the attack show how threat actors are aggressively toying with AI to make believable tracks and sophisticated payloads.