Disrupting Facial Recognition
Modern facial recognition systems depend on consistent landmarks, high-quality captures, and stable feature embeddings to match identities. Counter-surveillance focuses on systematically degrading these conditions across every stage of the FR pipeline.
Key Insight
The FR Pipeline & Attack Surface
Every facial recognition system processes input through these stages. Understanding the pipeline reveals where defenses are most effective.
Stage 1: Detection
Localize face bounding boxes in the frame using MTCNN, RetinaFace, or YOLO-Face.
Weakness: Occlusion, angle, crowded backgrounds, unusual aspect ratios reduce reliable face box extraction.
Stage 2: Alignment
Normalize face geometry using 5-point landmark detection (eye corners, nose tip, mouth corners).
Weakness: Asymmetric patterns (hair, brim, frames) disrupt consistent landmark localization.
Stage 3: Embedding
Extract a 128–512 dimensional feature vector using ArcFace, FaceNet, or CosFace deep models.
Weakness: Variable lighting and reflectivity create unstable embeddings. Low-quality input = noisy vectors.
Stage 4: Matching
Compare embedding vectors using cosine similarity or L2 distance against a gallery database.
Weakness: Noisy embeddings increase false-reject rates and reduce match confidence below alert thresholds.
Modern FR Model Landscape
Understanding which models are deployed helps calibrate which defenses are relevant.
| Model | Embedding Dim | Architecture | Typical Deployment | Key Weakness |
|---|---|---|---|---|
| ArcFace | 512 | ResNet-100 | Law enforcement, commercial APIs | Sensitive to pose > 60° |
| FaceNet | 128 | Inception-ResNet | Google, mobile apps | Lower accuracy on dark-skinned faces |
| CosFace | 512 | ResNet-50 | Access control, kiosks | Quality-sensitive thresholds |
| RetinaFace | N/A (detector) | ResNet + FPN | Pre-processing for all above | Small face / heavy occlusion misses |
| InsightFace | 512 | ResNet-100 / MobileFace | Edge devices, real-time | Mobile variants trade accuracy for speed |
Practical Defensive Techniques
These techniques target different pipeline stages. Combining multiple techniques across stages provides stronger defense than any single method.
Clothing & Silhouette
Brimmed hats reduce forehead/eyebrow keypoints. High collars obscure jaw geometry. Non-reflective, dark fabrics reduce contrast with skin tones.
Pipeline target: Detection + Alignment
Eyewear Strategy
Large frames disrupt inter-pupillary distance measurement. IR-filtering lenses reduce eye visibility in night-vision cameras. Reflective coatings can create noise in embeddings.
Pipeline target: Alignment + Embedding
Patterned Makeup / CV Dazzle
High-contrast asymmetric patterns near eyes/nose/cheeks can reduce landmark confidence. Effectiveness varies significantly by model — requires testing against your threat model's likely detectors.
Pipeline target: Detection + Alignment
Image Hygiene
Limit high-res front-facing portrait uploads. Strip EXIF data. Use Fawkes or LowKey to cloak images before upload. Reduce enrollment-quality photos in searchable databases.
Pipeline target: Gallery quality
IR-Aware Eyewear
IR LED accessories can flood night-vision cameras with noise. Performance depends heavily on camera sensor filters and ambient illumination balance. Test against local camera types.
Pipeline target: Collection quality
Movement & Angle
Avoid direct head-on exposure to cameras. Partial occlusion combined with angle changes and movement usually outperforms any single static countermeasure.
Pipeline target: Detection + Embedding stability
Reality Check
3-Phase Face Privacy Playbook
Phase 1: Minimize Training Data
- • Audit and remove legacy high-res portraits from public platforms
- • Apply Fawkes cloaking to images before any upload
- • Strip EXIF metadata from all shared media
- • Request deletion from facial recognition databases (PimEyes, Clearview opt-out)
- • Disable auto-tagging features on social platforms
Phase 2: Reduce Capture Quality
- • Use brimmed headwear + large-frame eyewear combination
- • Position body at angles to camera lines (avoid direct frontal exposure)
- • Use environmental factors (sun glare, shadows, reflections)
- • Test IR-aware accessories against local camera types
- • Vary appearance across sessions (hairstyle, eyewear, clothing colors)
Phase 3: Break Correlation
- • Avoid repeatable routes and timing patterns
- • Combine face defenses with device/behavioral controls
- • Use different appearance profiles for different activities
- • Limit clothing/accessory reuse across high-risk venues
- • Regularly rotate defense combinations to prevent pattern learning
Validation & Testing
Measure your defenses objectively rather than assuming they work. Use these tools to establish before/after baselines.
Face Verification Comparison
Compare match distance between baseline and mitigated conditions using DeepFace.
#!/usr/bin/env python3
# Prerequisites: pip install deepface opencv-python tensorflow
"""Compare two face images and measure verification confidence.
Use this to test how appearance changes affect match scores."""
# DeepFace returns cosine distance by default (0 = identical, 1 = completely different)
from deepface import DeepFace
import json
# Test baseline (same person, normal conditions)
baseline = DeepFace.verify(
img1_path="baseline_front.jpg",
img2_path="baseline_angle.jpg",
model_name="ArcFace",
detector_backend="retinaface"
)
print("Baseline match:", json.dumps(baseline, indent=2))
# Test mitigated (same person, with countermeasures)
mitigated = DeepFace.verify(
img1_path="baseline_front.jpg",
img2_path="mitigated_hat_glasses.jpg",
model_name="ArcFace",
detector_backend="retinaface"
)
print("Mitigated match:", json.dumps(mitigated, indent=2))
# Positive delta = mitigation INCREASED distance from reference (good — harder to match)
print(f"Confidence delta: {mitigated['distance'] - baseline['distance']:.4f}")
# Expected output:
# === Baseline Verification ===
# Model: VGG-Face | Distance: 0.2134 | Threshold: 0.40 | Verified: True
# Model: Facenet | Distance: 0.7891 | Threshold: 1.10 | Verified: True
#
# === Mitigated Verification ===
# Model: VGG-Face | Distance: 0.5847 | Threshold: 0.40 | Verified: False ← defense effective
# Model: Facenet | Distance: 1.3204 | Threshold: 1.10 | Verified: False ← defense effective
#
# Confidence delta (VGG-Face): +0.3713 (higher = better mitigation)Image Cloaking with Fawkes
Add imperceptible perturbations to photos before uploading to social media, reducing their utility for model training.
# Install Fawkes image cloaking tool
# ⚠ The pip package is broken since late 2023 — install from source:
git clone https://github.com/Shawn-Shan/fawkes.git && cd fawkes
pip install -e .
# Cloak images before uploading to social media
# This adds imperceptible perturbations that disrupt model training
# -m modes: low (fastest, minimal perturbation), mid (balanced), high (strongest cloak, slower)
fawkes -d ./photos_to_upload/ -m high --gpu 0
# Verify cloaking was applied
fawkes -d ./photos_to_upload/ --mode verifyDetection Rate Testing
Systematically test how different countermeasures affect face detection rates across conditions.
#!/usr/bin/env python3
# Prerequisites: pip install mediapipe opencv-python
"""Test face detection rates under various conditions.
Measures how countermeasures affect detector reliability."""
import cv2
import mediapipe as mp
import os
import json
mp_face = mp.solutions.face_detection
results = {}
conditions = {
"baseline": "captures/baseline/",
"hat_only": "captures/hat/",
"glasses_only": "captures/glasses/",
"hat_and_glasses": "captures/hat_glasses/",
"hat_glasses_angle": "captures/hat_glasses_angle/",
"ir_glasses": "captures/ir_glasses/",
"patterned_scarf": "captures/patterned_scarf/"
}
with mp_face.FaceDetection(
model_selection=1, # model_selection: 0 = short-range (faces within 2m), 1 = full-range (up to 5m)
min_detection_confidence=0.5 # min_detection_confidence: 0.5 = balanced threshold (lower catches more faces but more false positives)
) as detector:
for condition, path in conditions.items():
detected = 0
total = 0
confidences = []
for img_file in os.listdir(path):
img = cv2.imread(os.path.join(path, img_file))
if img is None:
continue
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
result = detector.process(rgb)
total += 1
if result.detections:
detected += 1
confidences.append(result.detections[0].score[0])
avg_conf = sum(confidences) / len(confidences) if confidences else 0
results[condition] = {
"detection_rate": f"{detected/total*100:.1f}%" if total > 0 else "N/A",
"avg_confidence": f"{avg_conf:.3f}",
"samples": total
}
print(json.dumps(results, indent=2))
# --- Expected Output ---
# {
# "baseline": {
# "detection_rate": "100.0%",
# "avg_confidence": "0.943",
# "samples": 25
# },
# "hat_only": {
# "detection_rate": "88.0%",
# "avg_confidence": "0.812",
# "samples": 25
# },
# "glasses_only": {
# "detection_rate": "96.0%",
# "avg_confidence": "0.887",
# "samples": 25
# },
# "hat_and_glasses": {
# "detection_rate": "72.0%",
# "avg_confidence": "0.694",
# "samples": 25
# },
# "ir_glasses": {
# "detection_rate": "32.0%",
# "avg_confidence": "0.531",
# "samples": 25
# },
# "patterned_scarf": {
# "detection_rate": "44.0%",
# "avg_confidence": "0.578",
# "samples": 25
# }
# }Embedding Stability Analysis
Measure how defensive conditions affect embedding cosine similarity — lower similarity means harder to match.
#!/usr/bin/env python3
# Prerequisites: pip install deepface numpy
"""Analyze face embedding stability across conditions.
Lower cosine similarity = harder to match across captures."""
from deepface import DeepFace
import numpy as np
def get_embedding(img_path, model="ArcFace"):
result = DeepFace.represent(img_path, model_name=model, detector_backend="retinaface")
return np.array(result[0]["embedding"])
def cosine_similarity(a, b):
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
# Reference embedding (enrollment photo)
ref = get_embedding("reference_portrait.jpg")
# Test conditions
test_images = {
"normal_lighting": "test_normal.jpg",
"low_light": "test_lowlight.jpg",
"hat_brim": "test_hat.jpg",
"reflective_glasses": "test_reflective.jpg",
"hat_plus_glasses": "test_hat_glasses.jpg",
"angle_30_deg": "test_angle30.jpg",
"distance_10m": "test_distance10m.jpg",
}
print(f"{'Condition':<25} {'Cosine Sim':>12} {'Would Match':>12}")
print("-" * 52)
for condition, path in test_images.items():
try:
emb = get_embedding(path)
sim = cosine_similarity(ref, emb)
# 0.68 = commonly used cosine similarity threshold for face match (model-dependent; VGG-Face typical range: 0.60–0.75)
match = "YES" if sim > 0.68 else "NO"
print(f"{condition:<25} {sim:>12.4f} {match:>12}")
except Exception as e:
print(f"{condition:<25} {'FAILED':>12} {'N/A':>12}")
# Expected output:
# Condition | Similarity | Match
# ────────────────────────────────────────
# baseline | 0.9234 | YES
# hat_sunglasses | 0.7102 | YES
# ir_leds_active | 0.5843 | NO ← effective
# cv_dazzle_makeup | 0.4127 | NO ← effective
# fawkes_cloaked | 0.6312 | NO ← effectiveKey Metrics to Track
| Metric | Baseline Target | Mitigated Target | How to Measure |
|---|---|---|---|
| Detection Rate | 95–99% | < 60% | MediaPipe / RetinaFace across 50+ captures |
| Cosine Similarity | > 0.75 | < 0.55 | DeepFace embedding comparison |
| Verification Pass | TRUE | FALSE | DeepFace.verify threshold check |
| Cross-Model Transfer | Match on 4/4 | Match on ≤ 1/4 | Test ArcFace, FaceNet, VGG-Face, DeepID |
Deepfake & Face Synthesis Threats
Face synthesis models create a parallel threat surface: attackers can generate realistic face images to bypass liveness checks, spoof identity verification, or create deniable false identities.
Face-Swap Attacks
Tools like DeepFaceLab, FaceFusion, and roop can swap one face onto another in real time or in post-production video.
- • Threat: Real-time video call spoofing bypasses KYC/liveness checks
- • Detection: Look for temporal flickering, blending artifacts, and gaze inconsistency
- • Defense: Multi-factor liveness (depth, IR, challenge-response) defeats most face-swap pipelines
GAN-Generated Faces
StyleGAN2/3 and diffusion models generate photorealistic faces of people who don't exist — useful for creating false identities at scale.
- • Threat: Synthetic profiles for social engineering and sock-puppet networks
- • Detection: Spectral analysis reveals GAN fingerprints; check for pupil asymmetry and ear inconsistencies
- • Tools: DM Image Detection, Microsoft Video Authenticator
Liveness Check Bypass
Attackers replay 3D-rendered face models or manipulated video streams to defeat basic liveness detection.
- • Replay attacks: Pre-recorded video played to camera defeats 2D liveness
- • 3D mask attacks: Silicone or 3D-printed masks fool depth sensors below certain thresholds
- • Mitigation: Active liveness (random gesture prompts) + multi-spectral IR imaging
Cloud FR API Testing
Commercial FR APIs (AWS Rekognition, Azure Face, Google Vision) have different robustness profiles against synthetic and adversarial inputs.
- • AWS Rekognition: Free tier allows 5K images/month for testing
- • Azure Face API: Includes built-in liveness detection since 2023
- • Demographic bias: NIST FRVT reports document accuracy disparities across demographics — review before trusting vendor claims
Deepfake Awareness
Defense Strategy Summary
- Degrade detection: combine headwear + eyewear + angle to push detection rate below reliable thresholds
- Destabilize embeddings: use variable appearance profiles to create inconsistent feature vectors
- Break temporal correlation: vary routes, timing, and appearance combinations across sessions
- Validate continuously: run quarterly benchmark tests to ensure controls remain effective against updated models
Facial Recognition Defense Labs
Hands-on exercises to measure and improve your face privacy posture.
Evidence to Capture
A face match is a decision record, not a photograph. What a reviewer needs is the input, the model state and the score that produced the decision, kept together.
Input provenance: Where the image came from (a posted photo, a venue camera, an enrolment kiosk), with the timestamp and, where it exists, the camera identifier.
Image or crop hash: SHA-256 of the exact file assessed, plus the uncropped original where retention allows. A crop without its source cannot be re-assessed.
Model, version and threshold: The model family and version, the distance metric and the threshold in force at decision time. Thresholds move, and a match at 0.68 means different things on two deployments.
Score and gallery size: The similarity or distance value alongside the number of enrolled templates searched. A score is only interpretable against the gallery it came from.
Human review record: Whether an operator reviewed the candidate before any action, who reviewed it, and what they were shown. This is where most FR decisions are actually made.
Reported quality and bias metadata: Pose, illumination and resolution flags the system returned, and any demographic error rates the vendor published, so the score is read against known weaknesses.
Likely False Positives
The single most common error in this area is reading a similarity score as an identification. Everything below changes the score without changing the person.
- Score above threshold is not identity: A threshold is a policy choice traded against false accepts. Clearing it produces a candidate, and a candidate is a lead for review.
- Cross-model disagreement: Two vendors scoring the same pair differently is normal, and it is the reason single-vendor evidence deserves a second implementation before it is relied on.
- Stale or low-quality enrolment: A gallery template taken years ago, or from a low-resolution source, drifts away from the live capture of the same person. Age is a false-positive source in both directions.
- Lookalike or relative: Siblings and twins defeat embeddings routinely. If the candidate has a close relative in the gallery, the score is not evidence on its own.
- Detection box is not a match: A detected face proves the stage-1 detector fired. Reporting a detection as a recognition event confuses two different pipeline stages.
- Threshold changed between deployments: The same vendor install tuned for a stadium and one tuned for an access-control door will not agree. Always state the threshold with the score.
Remediation
Reduce what can be enrolled
- • Limit public biometric samples: fewer tagged, high-resolution, front-facing photographs is the only control that works before a gallery exists.
- • Strip metadata and reduce resolution on images you publish, because enrolment quality is the input to every later score.
- • Use image cloaking before uploading where it is lawful in your jurisdiction, and treat it as a reduction in training value rather than a guarantee.
- • Ask the operator what enrolled you and on what legal basis. Many galleries contain scraped images the operator cannot justify.
Break the correlation
- • Separate contexts: appearance, timing and route variation across sites is what stops two captures being joined into one track.
- • Keep device and payment identifiers out of the same visit profile, because the fusion layer is what turns a weak face match into an identity.
- • Layer controls rather than relying on one: an appearance change plus a timing change beats either alone, and both are lawful in most jurisdictions.
- • Measure the effect instead of assuming it: a documented similarity drop across sessions is the evidence that a control works.
Document and contest
- • Request the decision record: input, model version, threshold and reviewer. An operator who cannot produce one is relying on a score they cannot defend.
- • Use the data-subject route available to you (access, correction, deletion) and keep the correspondence as evidence.
- • Escalate demographic error rates where the deployment affects a group with a documented accuracy gap, because that is a policy argument with published evidence behind it.
- • Involve counsel before challenging a deployment publicly; the technical finding and the legal argument need to be made by different people.
Related Topics
Adversarial ML
Digital perturbation techniques for model evaluation.
Physical Countermeasures
Wearable and environmental defense techniques.
Data Privacy Hygiene
Reduce your biometric footprint in databases.
Offensive AI
AI attack patterns and model security context.
Biometric Defense Evaluator
Interactive in-browser FR testing with BlazeFace.