FaceMatch
An identity-verification engine we built in house, matching a person's live capture against the photo on their official document, designed for accuracy, speed, and resilience against spoofing.
- Domain
- Identity Verification
- Discipline
- Computer Vision
- Type
- In-house R&D build
Overview
FaceMatch is an identity-verification system that confirms a person is who they claim to be by matching a live selfie against the portrait printed on an official document: a passport, national ID, or driver's license.
We built it as a self-directed R&D project to work through a problem we find genuinely hard: establishing trust in seconds, without human review, and without ever mishandling the biometric data involved.
The Problem
Manual identity checks are slow, inconsistent, and expensive. The conditions that break automated ones are mundane: poor lighting, glare on laminated documents, varied capture angles, and deliberate spoofing attempts using printed photos or screens.
So we set ourselves three constraints: fast enough for a live onboarding flow, accurate enough to be trusted without a human in the loop, and robust enough to reject fraudulent attempts. Every design decision below follows from those three.
The Solution
We built a multi-stage vision pipeline where each stage isolates a single, well-defined responsibility.
Document Detection
A YOLO-based detector locates and rectifies the document, then isolates the portrait region regardless of angle or background.
Face Detection & Alignment
Faces from both the document and the live capture are detected, cropped, and geometrically aligned to a canonical pose.
Embedding & Matching
A deep metric-learning model maps each face to a 512-dimensional embedding; cosine similarity yields a calibrated match score.
Liveness & Anti-Spoofing
A dedicated classifier flags presentation attacks such as printed photos, replayed screens, and masks before matching is trusted.
Architecture
A stateless API fronts an asynchronous vision pipeline. Each stage is independently scalable, and results are cached to keep repeat verifications instant.
Challenges
Real-world image quality was the hardest constraint: glare, motion blur, and low light degrade embeddings. We addressed it with aggressive augmentation during training and a quality gate that requests recapture rather than guessing.
Spoof resistance required a separate liveness model and a calibrated decision threshold. There is no universally correct setting here: the balance between false rejections and fraudulent acceptances is a risk decision, so we built it as a tunable parameter rather than a hardcoded constant.
Latency under load meant moving preprocessing to the GPU, batching embedding inference, and caching document crops in Redis for retries.
What It Demonstrates
The interesting result was architectural, not numerical. Splitting detection, alignment, embedding, and liveness into four independently testable stages meant we could reason about each failure mode in isolation, rather than staring at one opaque accuracy figure and guessing which part broke.
It also proved out a pattern we now reach for by default: when a model is uncertain, ask for a better input instead of guessing. The quality gate that requests a recapture turned out to matter more than any tuning we did to the matching model itself.
Future Improvements
- 1
On-device inference
Push face detection and quality checks to the client device to reduce bandwidth and improve privacy.
- 2
Expanded document coverage
Add support for additional regional ID formats and machine-readable-zone parsing.
- 3
Continuous evaluation
A shadow-evaluation pipeline that surfaces drift and edge cases as the evaluation set grows.
Technologies Used
Have a problem like this to solve?
We build these systems end to end. Let's talk about yours.
Book a Consultation