Introduction
The Vision Engine is the assistant’s perceptual foundation. Its purpose is not surveillance, tracking, or behavioral analysis. It answers a single, constrained question: is a known person present, and if so, who? Everything in its design follows from this limitation. Correctly implemented, the Vision Engine enables personalization without compromising privacy or system trust.
Detection vs Recognition
Face detection and face recognition are often confused but serve different roles. Detection answers “is there a face in the frame?” Recognition answers “whose face is this?” Detection is lightweight and continuous; recognition is heavier and event-driven. Separating the two is critical for performance and privacy. Detection runs frequently, recognition only when necessary.
Design Principles
The Vision Engine follows four non-negotiable principles:
– local processing only
– no continuous recording
– no raw image storage
– explicit owner-controlled enrollment
Frames are processed in memory and discarded immediately. Only numerical embeddings are stored.
Camera Strategy
Use a single fixed camera covering an entry zone. Wide-angle lenses reduce blind spots but increase distortion; moderate field-of-view lenses simplify embeddings. Native CSI cameras are preferred for lower latency and CPU overhead. USB cameras are acceptable but less deterministic under load.
Face Detection Pipeline
Face detection should be fast, robust, and tolerant to lighting changes. The detector’s only responsibility is to locate faces and provide bounding boxes. False positives are acceptable; missed detections should be rare. Detection runs continuously at a reduced frame rate to conserve resources.
Recognition Pipeline
Recognition is triggered only when:
– a face remains in view for a minimum time
– the bounding box is stable
– detection confidence exceeds a threshold
The cropped face is converted into an embedding vector using a lightweight neural model. This vector represents facial features numerically and contains no reconstructable image data.
Embedding Database
Each known person is represented by multiple embeddings generated from different reference photos. These are stored locally in a simple database (file-based, SQLite, or vector index). Matching uses cosine similarity or Euclidean distance. Thresholds must be conservative to avoid misidentification.
Enrollment Process
Only the owner can enroll new identities. Enrollment consists of uploading several reference images under controlled lighting and angles. Embeddings are generated once and stored. No further learning occurs automatically. This prevents silent drift and identity corruption.
Decision Logic
Recognition results are never binary. Each match includes a confidence score. The system reacts only above a strict acceptance threshold. Below threshold, the face is treated as unknown. Unknown faces trigger no scenarios and no logging beyond transient system metrics.
Privacy Safeguards
The Vision Engine does not:
– stream video externally
– archive frames
– perform emotion or behavior analysis
– identify unknown individuals
Physical camera indicators are recommended. The system must be auditable and predictable.
Performance Considerations
Raspberry Pi 5 can sustain real-time detection and event-based recognition concurrently if frame rates are controlled. Detection should be decoupled from recognition threads. CPU affinity and memory locality improve latency consistency. Thermal stability directly affects recognition reliability.
Failure Modes
Common failure modes include poor lighting, occlusions, and extreme angles. These are acceptable limitations. The system must fail safely by treating uncertain inputs as unknown rather than guessing.
Integration with Identity Engine
The Vision Engine outputs only one thing: a candidate identity with confidence. All decisions beyond that point belong to the Identity and Scenario Engines. This strict separation prevents accidental coupling between perception and behavior.
What Comes Next
With visual perception in place, the next article introduces the Identity Engine in detail: ownership, trust boundaries, scenario mapping, and how recognized faces become meaningful interactions rather than raw labels.
