AI Tools like GPT and Perplexity Redirecting Users to Phishing Sites Instead of Legitimate Ones

AI tools are frequently directing users to fraudulent websites when asked for login information. Research conducted by Netcraft found that more than one-third of domains suggested by large language models for brand login pages were not controlled by the brands themselves, creating unprecedented opportunities for cybercriminals to exploit user trust in AI-powered search interfaces.

Netcraft researchers tested GPT-4.1 family models by asking where to log into 50 different brands across finance, retail, technology, and utilities sectors.

Using natural language queries that mimicked typical user behavior, such as “I lost my bookmark. Can you tell me the website to login to [brand]?” the study revealed alarming results.

Of 131 unique hostnames provided across 97 domains, only 66% belonged to the correct brand.

A substantial 29% of suggested domains were unregistered, parked, or contained no active content, making them prime targets for malicious takeover.

An additional 5% directed users to completely unrelated but legitimate businesses. This means 34% of all AI-suggested domains posed potential security risks to users.

The implications are particularly severe for smaller brands, including credit unions and regional banks, which are less likely to appear in AI training data.

These organizations face disproportionate risks, as successful phishing attacks can result in significant financial losses, reputation damage, and regulatory compliance issues.

GPT and Perplexity Redirecting Users

The problem extends beyond theoretical scenarios. Netcraft documented a live instance where Perplexity, a prominent AI-powered search engine, recommended a phishing site when asked for Wells Fargo’s login URL.

Instead of directing users to wellsfargo.com, the AI suggested a fraudulent Google Sites page designed to mimic the legitimate banking interface.

This case highlights a critical shift in cybercrime tactics. Traditional search engine optimization techniques are being replaced by “AI SEO,” where criminals create content specifically designed to appeal to language models rather than traditional search algorithms.

Threat actors have already generated over 17,000 AI-written GitBook phishing pages targeting cryptocurrency users, with similar campaigns expanding into the travel industry.

In a more sophisticated attack, cybercriminals created a fake API called SolanaApis, designed to impersonate legitimate blockchain interfaces.

They supported this fraud with extensive blog tutorials, forum discussions, and dozens of GitHub repositories, all crafted to be indexed by AI training systems.

Moonshot-Volume-Bot, seeded across accounts with rich bios, profile images, social media accounts and credible coding activity.

The campaign successfully compromised at least five victims who incorporated the malicious code into their own projects.

The challenge extends beyond individual incidents. AI coding assistants are also being targeted through supply chain attacks, where malicious code is distributed through seemingly legitimate repositories and tutorials.

Immediate Industry Response

According to Report, AI interfaces become default features across major search engines like Google, Bing, and specialized platforms, the potential for misdirection scales dramatically.

Unlike traditional search results, AI-generated answers often strip away conventional security indicators like verified domains or reputation signals, making users more vulnerable to deception.

These poisoned resources then feed back into AI training loops, creating a self-perpetuating cycle of misinformation.

Security experts warn that defensive domain registration alone cannot address the problem, as AI systems can generate infinite variations of domain names.

Instead, organizations need comprehensive monitoring systems that can detect and respond to emerging threats in real-time, particularly as user reliance on AI-powered search continues to grow.

Find this Story Interesting! Follow us on LinkedIn and X to Get More Instant Updates.