What is hyper‑personalised, AI‑driven phishing and how to prevent it?

AI makes individualised, context-bound and trust-exploiting phishing faster, cheaper and easier to repeat. The approach is not based on new human psychology, but on the more precise exploitation of old human weaknesses for criminal purposes.

Phishing messages are no longer merely mass messages sent to everyone. With the help of AI, attackers can build deliberate scam messages that are tailored to the recipient and their situation.

AI is used to identify potential targets and combine available information about the people selected as targets, such as their leisure and work roles, professional connections, organisations and current events on which they may have commented on social media.

After searching, analysing and structuring this information, an attacker can use AI to produce a message that appears to relate to the recipient’s real everyday context, such as work, invoicing, HR, access rights or an ongoing work or personal project.

The most important new change in phishing messages is the ability to build a credible context. A person does not necessarily fall for a phishing message simply because it is written in fluent, error-free language, but because it arrives at the right moment or matches their work role. If the message also appears to come from the right party, relates to a familiar process and contains details that are meaningful to the recipient, it can more easily bypass the normal suspicion that it might be a phishing or scam message. AI makes it easier to personalise phishing messages because it can quickly adapt the same scam idea into different forms for different people.

The threat of AI in phishing messages is often reduced to the absence of language errors, but linguistic accuracy should be seen here only as a secondary factor. People have fallen for scams even when the messages have been linguistically clumsy, because phishing exploits a target’s sense of urgency, trust in authority, sense of duty and routine ways of working.

AI does not invent these psychological mechanisms anew; it helps attackers target them more precisely using information collected about the person being deceived.

Another essential change is scale. In the past, a precisely targeted phishing message, or spear phishing, required considerable human effort, research and message tailoring. It was therefore used mainly against high-value targets. AI lowers this threshold: an attacker can quickly and cheaply produce large numbers of messages tailored to different people. This makes personalised phishing messages more common.

In addition, the issue is no longer necessarily limited to email or text messages. The same scam narrative can be carried across several channels: as a message, a phone call, a meeting invitation or an appearance using voice and image. This makes the attack more credible because it resembles an ordinary chain of workplace communication. Communication arriving from several directions can also create a sense of urgency and pressure. In addition to text, AI can be used to falsify voice and image, further increasing the credibility of a multi-channel scam.

Since scams still exploit the old human weaknesses, prevention must combine technology and people. The role of technology is to support the detection of anomalies, the assessment of message origin and the escalation of atypical requests for review. At the same time, employees need clear operating models:

• a suspicious request is verified through another channel,

• urgency is not acted on automatically,

• unusual payments, access-rights requests or information requests are paused for checking

The best protection emerges when technology helps identify anomalies and people are permitted to slow down when a message feels too urgent, personal or unusual.

In a Microsoft environment, this is supported in particular by Microsoft Defender for Office 365’s detection of phishing, sender spoofing and impersonation attacks, together with Microsoft Defender XDR’s investigation views and advanced hunting queries.

At Context&, we specialise in applying Microsoft technologies in customer environments, and we can also support a human-centred approach through training and the development of operating models.

This blog is the second part of a blog series on AI assisted frauds and how organizations can protect themselves. You can read the first part here!