How it works

This page explains the conceptual model behind AI Insights & People: what happens between a camera seeing a face and that face appearing on your People timeline as an identified member, an "Unsure" visitor, or not at all.

You don't need any of this to use the product — but it makes the tuning settings and the Self Learning review queue much easier to reason about. The same pipeline is drawn live, with your facility's actual thresholds, in the Logic Visualisation panels on the Member Matching and Shadow Profiles settings pages.

The cast of characters

The identification pipeline

Every detected face flows through the same ordered decision chain. Order matters — an earlier stage's outcome prevents later stages from running.

1. Blacklist check (always first)

The face is compared against your facility's blacklist collection at a fixed 95% threshold (not configurable). On a match, processing stops immediately: the detection is dropped and never appears on the timeline. This is what makes the blacklist effective against static false positives like posters and mannequins — see Blacklist.

2. Face extraction

The system verifies there is actually a usable face in the image and extracts its features, pose, and a confidence score. No face → dropped.

3. Search identified members

The face is searched against the identified collection — every member with validated photos or previously confirmed detections. The search runs from the Identified Collection Search Floor upward (default 40% similarity on the Balanced preset) so borderline candidates are at least considered.

4. Visit-assisted matching

Before the final decision, the system checks recent facility check-ins. If a candidate member checked in within the Check-in Lookback Window (default 30 mins), their similarity score receives the Visiting Member Bonus (default +5). The reasoning: if someone who looks 82% like Alex Nguyen swiped Alex Nguyen's card three minutes ago, it's almost certainly Alex.

5. The match decision

• Score ≥ Identified Match Threshold (default 85%, Balanced) → the detection is confirmed as that member.
• Borderline scores — within the Borderline Confidence Zone (default +7 points above threshold) — get an extra pose check: the face must be reasonably front-facing (default identified limits: yaw 55° / pitch 30° / roll 30°). Steep-angle faces at borderline confidence are dropped rather than risk a wrong identification.
• Score below the Drop Threshold (default 45%) → the similarity is "not high enough to confirm the correct person with certainty" and the detection is dropped from matching.
• Anything in between → the person is treated as unsure and continues to step 6.

6. Shadow profile matching

Unmatched faces are searched against the unidentified collection — the shadow profiles of previously seen unsure people:

• Score ≥ Profile Link Threshold (default 80%) → the detection is linked to that existing shadow profile. The same visitor keeps the same "Unsure - …" identity across days and cameras.
• No match → the detection becomes a candidate for a new shadow profile, but only after quality gates (next step).

7. Quality gates for new shadow profiles

Creating a shadow profile means committing to a reference photo the system will compare against for weeks — so the photo has to be good. Two tiers run in sequence:

Photos that pass get attribute analysis (estimated age, gender, emotion) and then a final race-condition re-check: because detections process in parallel, the system searches the shadow collection once more before creating the profile, so two frames of the same new visitor don't create two profiles. The outcome is either New Shadow Profile Created or Matched Existing Shadow Profile.

All of these thresholds are tunable via Lenient / Balanced / Strict presets — see Face recognition tuning.

Self learning: closing the identification gap

Face matching alone can't identify a member who has no photo — or a photo too poor to pass validation. Self Learning solves this with behaviour instead of biometrics:

1. Every unidentified person's detection times are correlated against member check-ins over a lookback period (default 8 correlated visits required, looking at recent months of history, with a timing window of about 10 minutes either side of each check-in).
2. If exactly one member consistently checked in whenever this person was detected, validation checks run (gender alignment, optional age tolerance).
3. Checks pass → the profile is automatically linked to that member. Checks fail → it lands in the Pending Review queue for a human decision. Multiple candidate members → no match; the system keeps watching.

Separately, once a person is identified, the system quietly keeps its face reference fresh: near-perfect matches (99%+ similarity) can be auto-added as shadow photos, capped at a small rotating set (6) per person, so recognition keeps working as appearance changes. These auto-added photos are labelled "Auto-added by self-learning" in the profile.

The full pipeline, its statuses, and the review workflow are documented in Self Learning.

From detections to the timeline

Raw detections would be unreadable, so the timeline shows visits: detections merged into groups using the Detection Grouping settings — by default a new visit starts after a 45 minute gap, and unidentified people only appear at all with at least 3 detections spanning 5 minutes (so someone briefly walking past a door camera doesn't clutter the view). A toolbar toggle on the timeline can disable this threshold filter when you want to see every brief appearance.

What runs where

Each facility has its own isolated face collections (identified, unidentified, and blacklist) — recognition never crosses facilities. Face images and AI metadata are stored in your chosen storage region and age out with AI Metadata retention (default 24 months). See Security & privacy.

Glossary