AI Social Engineering
AI has fundamentally transformed social engineering from a craft into a scalable weapon. Deepfake video, real-time voice cloning, and LLM-generated phishing can now bypass human intuition at scale. Understanding these techniques is essential for red teamers and defenders alike.
Legal & Ethical Boundaries
The AI Social Engineering Threat Landscape
AI Social Engineering Attack Surface (2026)
Generation
Text / Email
LLM-crafted phishing
Voice Clone
3s sample → full voice
Deepfake Video
Real-time face swap + lip sync
Delivery
Objectives
Real-World Incidents
- 2024 — $25M deepfake heist: A Hong Kong finance worker was tricked into transferring funds after a video call with AI-generated deepfakes of senior executives.
- 2024 — CEO voice clone: Criminals used AI voice cloning to impersonate a CEO, authorising a fraudulent €220K wire transfer via phone call.
- 2025 — Election deepfakes: AI-generated robocalls mimicking political candidates used to suppress voter turnout in multiple countries.
Why Attackers Love AI
- Scale: Generate thousands of unique, personalised phishing emails in minutes
- Quality: Perfect grammar, cultural context, and writing style mimicry
- Speed: Real-time voice cloning needs only a 3-second sample
- Cost: Open-source models make deepfakes free to produce
- Evasion: Each output is unique — defeats signature-based email filters
1. LLM-Powered Phishing
Traditional phishing relies on templates that security-aware users learn to spot. AI-generated phishing is contextually unique, grammatically perfect, and can be personalised using OSINT data scraped from LinkedIn, social media, and company websites.
Red Team Simulation Framework
# Phishing simulation framework for authorised red team engagements
# REQUIRES: Written authorisation with social engineering in scope
import openai
import json
def generate_phishing_pretext(target_info: dict, scenario: str) -> str:
"""Generate a context-appropriate phishing pretext.
Args:
target_info: OSINT data about the target (name, role, company, interests)
scenario: Attack scenario (credential_harvest, malware_delivery, wire_fraud)
"""
prompt = f"""You are simulating a phishing email for an authorised red team engagement.
Target profile:
- Name: {target_info['name']}
- Role: {target_info['role']}
- Company: {target_info['company']}
- Recent activity: {target_info.get('recent_activity', 'N/A')}
Scenario: {scenario}
Generate a realistic phishing email that would be contextually appropriate for this
target. Include subject line, sender name, and email body. The email should leverage
the target's role and recent activity for credibility.
Format as JSON: {{"subject": "", "from_name": "", "from_address": "", "body": ""}}"""
response = openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0.7
)
return json.loads(response.choices[0].message.content)
# Example usage in authorised engagement
target = {
"name": "Jane Smith",
"role": "VP of Engineering",
"company": "Acme Corp",
"recent_activity": "Spoke at CloudConf 2026 about Kubernetes migration"
}
email = generate_phishing_pretext(target, "credential_harvest")
print(f"Subject: {email['subject']}")
print(f"From: {email['from_name']} <{email['from_address']}>")
print(f"\n{email['body']}")# Phishing simulation framework for authorised red team engagements
# REQUIRES: Written authorisation with social engineering in scope
import openai
import json
def generate_phishing_pretext(target_info: dict, scenario: str) -> str:
"""Generate a context-appropriate phishing pretext.
Args:
target_info: OSINT data about the target (name, role, company, interests)
scenario: Attack scenario (credential_harvest, malware_delivery, wire_fraud)
"""
prompt = f"""You are simulating a phishing email for an authorised red team engagement.
Target profile:
- Name: {target_info['name']}
- Role: {target_info['role']}
- Company: {target_info['company']}
- Recent activity: {target_info.get('recent_activity', 'N/A')}
Scenario: {scenario}
Generate a realistic phishing email that would be contextually appropriate for this
target. Include subject line, sender name, and email body. The email should leverage
the target's role and recent activity for credibility.
Format as JSON: {{"subject": "", "from_name": "", "from_address": "", "body": ""}}"""
response = openai.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0.7
)
return json.loads(response.choices[0].message.content)
# Example usage in authorised engagement
target = {
"name": "Jane Smith",
"role": "VP of Engineering",
"company": "Acme Corp",
"recent_activity": "Spoke at CloudConf 2026 about Kubernetes migration"
}
email = generate_phishing_pretext(target, "credential_harvest")
print(f"Subject: {email['subject']}")
print(f"From: {email['from_name']} <{email['from_address']}>")
print(f"\n{email['body']}")GoPhish + LLM Integration
2. Voice Cloning & Vishing
Modern voice cloning models need as little as 3 seconds of audio to produce a convincing clone. Combined with real-time speech-to-speech models, attackers can conduct live phone calls in someone else's voice.
Voice Cloning Tools (Research/Red Team)
| Tool | Type | Sample Needed | Real-time? |
|---|---|---|---|
| ElevenLabs | Cloud API | ~1 min audio | Yes (streaming) |
| OpenVoice | Open-source | ~10 seconds | Near real-time |
| XTTS v2 (Coqui) | Open-source | ~6 seconds | Near real-time |
| RVC (Retrieval Voice) | Open-source | ~10 min (training) | Yes |
| Vall-E / Vall-E X | Research (Microsoft) | 3 seconds | No (batch) |
Red Team Vishing Workflow
# Vishing attack simulation workflow (authorised engagement only)
# Step 1: Collect voice sample from public sources
# LinkedIn videos, YouTube talks, podcast appearances, earnings calls
yt-dlp -x --audio-format wav "https://youtube.com/watch?v=TARGET_TALK"
# Step 2: Clone voice with OpenVoice (local, no data leakage)
git clone https://github.com/myshell-ai/OpenVoice.git
cd OpenVoice
pip install -e .
python openvoice_cli.py \
--reference_audio target_sample.wav \
--text "Hi, this is [Name] from IT. We detected unusual activity on your account.
I need you to verify your identity by logging into our security portal." \
--output vishing_sample.wav
# Step 3: Real-time voice conversion for live calls
# Use RVC or SoVITS for live voice conversion during a phone call
# Pipe microphone → voice model → VOIP output
# Step 4: Combine with AI-generated pretext
# Feed OSINT about the target into an LLM to generate a contextual script
# The pretext should reference real projects, people, or events# Vishing attack simulation workflow (authorised engagement only)
# Step 1: Collect voice sample from public sources
# LinkedIn videos, YouTube talks, podcast appearances, earnings calls
yt-dlp -x --audio-format wav "https://youtube.com/watch?v=TARGET_TALK"
# Step 2: Clone voice with OpenVoice (local, no data leakage)
git clone https://github.com/myshell-ai/OpenVoice.git
cd OpenVoice
pip install -e .
python openvoice_cli.py \
--reference_audio target_sample.wav \
--text "Hi, this is [Name] from IT. We detected unusual activity on your account.
I need you to verify your identity by logging into our security portal." \
--output vishing_sample.wav
# Step 3: Real-time voice conversion for live calls
# Use RVC or SoVITS for live voice conversion during a phone call
# Pipe microphone → voice model → VOIP output
# Step 4: Combine with AI-generated pretext
# Feed OSINT about the target into an LLM to generate a contextual script
# The pretext should reference real projects, people, or events3. Deepfake Video
Real-time deepfake technology allows attackers to impersonate anyone on a video call. This has already been used in the wild for fraud, and the barrier to entry is dropping rapidly.
Deepfake Tools
- DeepFaceLive: Real-time face swap for video calls (open-source)
- SimSwap: High-fidelity face swapping with single image
- Wav2Lip: Accurate lip sync for any face with any audio
- FaceFusion: Next-gen face swapping and enhancement
- Roop: One-click face swap (simple but effective)
Attack Scenarios
- Executive impersonation: Deepfake CEO on Zoom authorising wire transfers
- IT help desk: Fake IT admin on Teams requesting credentials
- Vendor impersonation: Fake supplier representative changing payment details
- KYC bypass: Deepfake video verification to open fraudulent accounts
4. Detection & Defence
Defending against AI social engineering requires both technical controls and human awareness training. Traditional email filters are insufficient against LLM-generated, contextually unique content.
Technical Defences
- Voice verification protocols: Callback procedures with pre-shared code words
- Deepfake detection models: Microsoft Video Authenticator, Intel FakeCatcher
- AI email analysis: Analyse writing style deviation from known sender patterns
- DMARC / SPF / DKIM: Still essential — blocks impersonation at the email protocol level
- Out-of-band verification: Verify high-value requests via a separate communication channel
Human Defences
- AI-aware training: Teach staff that voice and video can be faked
- Challenge phrases: Pre-agreed words for verifying identity in calls
- Dual authorisation: Wire transfers require 2 people to approve
- Red team exercises: Regular simulated attacks with AI-generated content
- Slow down urgency: Train staff to pause when pressured for immediate action
Red Team Reporting
5. Building Your AI SE Toolkit
# Recommended AI social engineering toolkit for red teamers
# All tools should be run in an isolated VM
# Text generation (phishing pretexts)
pip install openai # GPT-4o API for phishing generation
# OR use local models:
ollama pull dolphin-mixtral # Uncensored local model
# Voice cloning
git clone https://github.com/myshell-ai/OpenVoice.git
pip install -e OpenVoice/
# Alternative: XTTS v2
pip install TTS
# Deepfake video
git clone https://github.com/iperov/DeepFaceLive.git
# OR lightweight:
pip install roop
# OSINT for target profiling
pip install theHarvester
pip install social-analyzer
# Campaign management
# GoPhish for email campaigns: https://getgophish.com
# Track: open rate, click rate, credential harvest rate
# Audio sample collection
pip install yt-dlp # Download public talks/interviews# Recommended AI social engineering toolkit for red teamers
# All tools should be run in an isolated VM
# Text generation (phishing pretexts)
pip install openai # GPT-4o API for phishing generation
# OR use local models:
ollama pull dolphin-mixtral # Uncensored local model
# Voice cloning
git clone https://github.com/myshell-ai/OpenVoice.git
pip install -e OpenVoice/
# Alternative: XTTS v2
pip install TTS
# Deepfake video
git clone https://github.com/iperov/DeepFaceLive.git
# OR lightweight:
pip install roop
# OSINT for target profiling
pip install theHarvester
pip install social-analyzer
# Campaign management
# GoPhish for email campaigns: https://getgophish.com
# Track: open rate, click rate, credential harvest rate
# Audio sample collection
pip install yt-dlp # Download public talks/interviewsGetting Started