diff --git a/docs/Behavision-Architecture.html b/docs/Behavision-Architecture.html new file mode 100644 index 0000000..4357aa1 --- /dev/null +++ b/docs/Behavision-Architecture.html @@ -0,0 +1,577 @@ +Behavision Architecture + + + + + + + + + +
+
+
Behavision · Technical Overview
+

Behavision Architecture

+

Face recognition for retail. An engine that sees, an agent that delivers, a platform that understands — in nine diagrams.

+ +
+ process + store + the path in focus + data + control / pull + trust boundary +
+
+
+ +
+ + +
+
01

The whole system

Video stays inside the shop. Only visit records cross the boundary — and every connection across it is made from the inside, outward.

+
+ + SHOP NETWORKLOYALY CLOUDPEOPLE + + + outbound only + + + + + RTSP + + + + Shop PC + + + Recognition engine + Python · ONNX Runtime · FAISS + detect → track → identify + SQLite + + + AgentGo · supervisor · camera sync + + + durable spool — one file per event + + + Shop appWails · window + system tray + runs with no internet; + the spool drains when it returns + + video + detections + + + + + MosquittoMQTT · TLS · per-tenant ACL + + + + Behavision serverGo · one binary + ingestdedupe · reinforce + API + web48 routes · SSE + + + + PostgreSQL + master database + + + + visits · QoS 1 + + + + + pull: cameras, checks + + + Head-office consoleowner · manager + Mobile appsales staff + Platform admincreates merchants + + + + https · session + +
+
+
Three tiers, one direction of trust: the shop initiates every connection it has.
+
One server binary carries ingest, the API and the head-office web app.
+
One API for the console, the mobile app and the shop app alike.
+
Offline is a delay, not a loss: visits queue on disk until the broker confirms them.
+
+
+ + +
+
02

Inside the shop PC

Three processes on one machine, each in the language its job is best done in, sharing one state root.

+
+ + + + Recognition enginePython 3.10+ · private venv + capture threadper camera · latest frame + worker threadper camera · one track = one person + + models · ONNX RuntimeYuNet · ArcFace r50 · genderage · CoreML / DirectML + SQLite galleryidentities · embeddings · sightings + FAISS index + rebuilt at boot + FastAPI on 127.0.0.1:8010 · Basic auth, credential generated on first start + + + + Agent — Go libraryagent/pkg · shared by app and headless agent + supervisorstart · restart · backoff + bridgeloopback webhook + spoolbounded · acked per event + pumpMQTT QoS 1 · TLS + camera reconcilerpulls desired state · runs placement checks + + + + Shop app — WailsGo + React in the system webview · 12 MB + windowLive · Customers · Cameras + system traygreen / amber / red · start · stop · quit + camera relayloopback · no credential in the page + + + events + health · stats + embeds + + + + One state root — ProgramData\Behavision + data\behavision.db · data\cameras.json (DPAPI) · data\api_credentials.txt · models\ · runtime\ (the engine's Python) · agent.json · spool\ + + BEHAVISION_DATA_DIR + +
+
+
Python where the recognition ecosystem is — ONNX, OpenCV, FAISS are first-class.
+
Go for lifecycle and delivery — static binaries, cross-compiled to Windows from anywhere.
+
Wails for the UI — window, tray and supervisor in one process; a service cannot draw a tray icon.
+
Exact search: 100,000 identities in 21.9 ms. Identity is decided once per track, so this is queries per minute, not per frame.
+
+
+ + +
+
03

Recognition: one decision per visit

Frames become tracks; tracks accumulate evidence; a track is identified once. A "not sure" outcome is what stops one person becoming three, and a stranger becoming a regular.

+
+ + + Frames + Detect + Track + Quality gate + Align + embed + Average ≥ 3 + + + 15 fps · latest frame + YuNet · 5 landmarksscore ≥ 0.82 + greedy IoU 0.3one track per person + sharp · size · light · frontalper-camera threshold + Umeyama → 112×112ArcFace r50 · 512-d + normalised mean≥ 4 hits + + + + + + + cosine vsgallery + + + + known≥ 0.42 · person.seen + + not sure0.32 – 0.42 · retry ≤ 8× + + + new< 0.32 · enrol + + + + GallerySQLite + FAISS · ≤ 5 views per person + reinforce: good quality, not a near-duplicate + enrol as "Visitor N" + index searched once per track + + + wait 0.5 s for a better frame + +
+
+
Never per frame. Single-frame decisions turned one walk-past into three or four "people"; averaging fixed it.
+
Model-tagged embeddings. Only same-model vectors share an index; swapping encoders can never mix spaces.
+
Quality is per camera, match is shared. Every camera writes into one gallery.
+
Measured: 103 tracks → 7 people, 44 correct re-recognitions, in five minutes on the office camera.
+
+
+ + +
+
04

Delivery: durable before published

Nothing is removed from the shop's disk until the broker has confirmed it, and the server drops what it has already seen. That pair is what makes an outage a delay and not a hole.

+
+ + SHOP PCBROKERCLOUD + + + + + engine + bridge + spool + waker + pump + Mosquitto + ingest + API · SSE + + + webhook POST + event_id = site·cam·id·sec + append → fsync + rung AFTER append + QoS 1 · in order + TLS 8883 · ACL by tenant + INSERT … ON CONFLICT + arrivals feed + + PostgreSQL + head office + + + + + + + + + + + + wake + publish + deliver + write + doorbell + push + + + + PUBACK → delete the spool file. Never before. + + + + + offline?backoff 1 → 30 s · spool grows + + + +
+
+
QoS 1, clean session. QoS 0 could delete an event the wire dropped; QoS 2 buys nothing the derived id doesn't already give.
+
Ordered. A failed publish stops the batch — a customer's visits are a timeline.
+
Bounded and honest. The spool has a cap and reports what it dropped; a corrupt entry is quarantined, never retried forever.
+
Measured: 120 simultaneous visits published, 120 delivered; end to end in ~3 s.
+
+
+ + +
+
05

Local gallery, master database

Two stores with two jobs. The shop PC's gallery recognises people in that shop, offline if need be. The platform's database knows the business: customers across shops, history, reports, tenancy.

+
+ + SHOP PCPLATFORM + + + + SQLite gallery + the only persistent state on the PC · WAL + identities Visitor N, labelembeddings 512-d · tagged by modelsightings identity × camera × time + FAISS index — rebuilt from SQLite at boot + exact inner product · numpy fallback identical + + + cameras.jsonpasswords DPAPI-encrypted · machine-bound + agent.jsonbroker login · agent token · sealed at rest + + + PostgreSQL + every table carries client_id · self-migrating schema · 13 migrations + + clients slug = MQTT topic prefix + sites · agents slug · tz · heartbeat · fraction_below_gate + app_users owner · manager · staff · bcrypt + visitors number → V-42 · per tenant + visits seq (feed cursor) · source_event_id (dedupe) + visitor_embeddings ≤ 5 · reinforced server-side + site_cameras password sealed AES-GCM, aad = site + sessions SHA-256 of tokens · revocable + visit_faces · camera_snapshots · audit_log + + Object storage (optional)presigned PUT from the shop PC · presigned GET for staff · private ACL in the signature + + + + + visit + template ↑ + cameras · checks ↓ + heartbeat · health ↑ + +
+
+
Video, frames and raw images never cross. A template and a timestamp do.
+
Tenancy is a column and a rule, and the two agree: the tenant comes from the session, never from the request.
+
References are immutable — slugs, camera ids, customer numbers — because other systems store them. Display names are free to change.
+
Feed by seq, never by the camera's clock: lossless under bursts and backlogs; cursors are opaque.
+
+
+ + +
+
06

Clients and the API

Three kinds of people and one kind of machine, all through one API. Sessions are opaque tokens in a table, so "log that device out, now" actually works.

+
+ + + Head-office consoleReact · embedded in server + Mobile apparrivals · customers · sales + Shop appengine on loopback · cloud for the rest + Shop PC agenttoken issued once at enrolment + + + + Session + 256-bit opaque tokenstored as SHA-256 onlyrefresh rotates in placerevocable per device, instantly + + tenant ← session.client_id + staff arrivals · customers · sales + manager + cameras · team · erasure + owner + mint owners + + + /api + + /auth/login · refresh · sessions · register + /visits · /visits/stream ·································· SSE, cursor + /visitors · /history · /profile · /image · /purchases + /sites · /sites/{s}/check · /enrolment-code + /cameras · /check · /snapshot.jpg · /live ······· SSE relay + /reports/footfall · /reports/conversion + /team · /team/members · /team/{id}/password · /invitations + /assistant ··································· tools, never SQL + /admin/clients ····························· platform admin only + + + /agent/* — enrol · cameras · checks · faces · upload-url · live + agent token only · a user session is refused + + + + email + password + Ids accept names: /api/visitors/V-42 · ?site=chennai · /api/cameras/cam1 — a uuid still works everywhere. + +
+
+
Login is boring on purpose. Unknown address and wrong password are byte-identical and cost the same time.
+
Another tenant's data is 404, never 403 — nothing to enumerate.
+
Live video at head office: the agent pushes ~13 fps only while someone watches. 259 KB/s measured.
+
The assistant answers from the same report tools, as the signed-in user; it has no tenant parameter to misuse.
+
+
+ + +
+
07

Onboarding: each tier creates the next

No credential ships inside an installer, and nobody creates their own account from nothing.

+
+ + + Platform admin + Merchant owner + Sales staff + Shop PC + Cameras + + + + + + + + + + POST /api/admin/clientscompany + owner, one transactionowner password, shown once + POST /api/team/membersor /team/invitations → a code they redeemstaff password, shown once + POST /api/sites/{shop}/enrolment-codesingle use · 7 days · redeemed by the PC:broker login + agent token + CA to pin + head office pushes camerasthe PC pulls · adopts local ones uppasswords travel only to that site's agent + + Single use is enforced by the UPDATE itself, so two PCs racing on one code cannot both win. A wrong, spent or expired code all read the same. + +
+
+
Direct or by invitation. A manager can hand over a generated password, or let the salesperson choose their own via a code.
+
Reset signs the lost phone out in the same transaction as the new password.
+
Standalone is a first-class answer on the setup screen: a single-till shop with no head office runs the full product locally.
+
Demo build: cameras ship sealed (AES-256-GCM); the unlock code travels separately from the zip.
+
+
+ + +
+
08

Where every secret lives

Biometric data is treated as biometric data. Each credential has one home and one protection, and none of them is ever returned by an API.

+
+ + SHOP PCIN TRANSITDATABASEPOLICY + + + + camera passwordsDPAPI · machine-bound · has_password only + engine API credentialgenerated on first start · 0600 · read by the agent + agent token · broker passwordsealed in agent.json · earned by enrolment, never shipped + face templatesSQLite · treated as personal data · erasure deletes outright + + MQTTTLS 8883 · pinned issuer · plaintext to any non-loopback host is refused + APIHTTPS behind Traefik · bearer sessions + imagespresigned URLs, minutes-long · private ACL inside the signature + shop PC ↔ engineloopback only · relay token per run, no password in the page + + broker + camera passwordsAES-256-GCM · aad = owning site · row copies don't decrypt + user passwordsbcrypt cost 12 · 10 failures / 15 min per account + session tokensSHA-256 only · a dump holds no usable session + audit_logevery face-image hand-out, every code minted, every merchant created + + video never leavesrecognition runs in the shop + photos are opt-instore_faces defaults to off + tenancy is structuralsession decides · 404, never 403 + erasure erasesobject first · 502 changes nothing + + +
+
+ + +
+
09

The stack, and the numbers behind it

Each layer, the choice, and the one reason that decided it.

+
+ + + ClientsReact console (embedded) · mobile app · Wails shop appone API; UI can never lag its server + API + webGo · one binary · opaque sessions in a tableinstant per-device revocation; JWTs cannot + TransportMosquitto · MQTT QoS 1 · TLS · per-tenant ACLbuilt for many outbound clients; ~10 MB + Master dataPostgreSQL · self-applying migrations · advisory lock · checksumstransactions across tenant + owner; keyset feeds + Shop agentGo library · spool · pump · supervisor · reconcilerstatic binary, cross-compiled; one tested implementation + RecognitionPython · YuNet · ArcFace r50 (ONNX Runtime) · FAISS IndexFlatIP · SQLite WAL97.25 IJB-C · exact search · no second process + Camerasany RTSP camera · make picker fills the stream path · placement proved by a 25-second walk-past + + +
+
+
103 → 7tracks to people, 5 min, office camera — 44 re-recognitions
+
120 / 120simultaneous visits delivered, real broker and database
+
21.9 msexact search over 100,000 identities
+
~3 scamera to head-office feed
+
14 fpslive picture on the shop PC vs a 15 fps camera
+
10 / 10install steps on a clean machine, both cameras connected
+
+
+ +
+ +