Loyaly/Behavision
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 22 Wiki Activity

Behavision: face recognition for retail, edge to head office

Browse Source 
Five components that ship as one product:

- behavision/  the recognition engine. RTSP ingest, YuNet detection, IoU
               tracking, ArcFace embeddings, a FAISS/SQLite gallery, and a
               FastAPI dashboard. Identity is decided once per TRACK from an
               average of at least three embeddings, never per frame.
- agent/       the Go edge agent: supervises the engine, holds a durable
               spool, and drains it to MQTT. Nothing is acked before the
               broker confirms.
- desktop/     the shop PC application (Wails + React + tray).
- server/      the cloud API, MQTT consumer, reports and assistant.
- web/         platform.loyaly.ai, the head-office app, embedded in the
               server binary.

The gallery stores 512-float embeddings and timestamps - no images unless
`app.store_faces` is switched on. Those embeddings are biometric personal
data under GDPR and India's DPDP: template inversion reconstructs a
recognisable face from an ArcFace vector, so data/behavision.db is treated
as a biometric database and DELETE /api/visitors/{id} is a real erasure.

CLAUDE.md carries the reasoning behind every non-obvious decision here,
including the ones that were measured and the ones that were wrong first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HViLj9gYNRtSr7YVZmW5sn
This commit is contained in:
Suriyakumarvijayanayagam
2026-09-04 11:14:18 +05:30
commit dad04e8cda
216 changed files with 40473 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

217
RUN.md Normal file
View File

@@ -0,0 +1,217 @@
# Running the platform locally
Three processes and two containers. Nothing here touches production.
## The short way
```bash
./run-local.sh # Postgres + Mosquitto + build + accounts + run
```
Idempotent — run it again after a reset and it rebuilds the same state. It
prints the two logins and serves <http://127.0.0.1:8088>.
**Port 8088, not 8080.** Docker Desktop listens on 127.0.0.1:8080 itself, and
an earlier run of this ended up talking to Docker's own listener and reading
its 401 as a Behavision one.
Everything it writes lives in `.local/` — the binary, the encryption key, the
broker's password file. Keep `.local/env.sh`: without that key every sealed
camera and broker password is unrecoverable.
The rest of this file is the same thing done by hand.
## 1. Database
```bash
docker run -d --name bv-pg -p 55432:5432 \
-e POSTGRES_PASSWORD=test -e POSTGRES_DB=behavision \
pgvector/pgvector:pg16 # pgvector, NOT plain postgres — 001 needs it
for f in server/migrations/*.sql; do
docker exec -i bv-pg psql -U postgres -d behavision -v ON_ERROR_STOP=1 -q < "$f"
done
```
## 2. Build
The web app builds INTO the Go module, so the server must be built after it.
```bash
cd web && npm install && npm run build # -> server/internal/web/dist
cd ../server && go build -o bv-server ./cmd/behavision-server
```
## 3. First accounts
The first platform admin comes from the CLI, because creating it cannot require
being signed in as one.
```bash
export DATABASE_URL='postgres://postgres:test@127.0.0.1:55432/behavision'
# -raw prints the key alone. Without it the command prints JSON, for pasting
# into an env file - and `$(... | tail -1)` then captures the whole object and
# hands the server a key it cannot parse.
export BEHAVISION_SECRET_KEY="$(./bv-server provision key -raw)"
./bv-server provision user -email root@loyaly.ai -role admin \
-name "Loyaly Platform" -password 'loyaly-root-2026'
```
Every company after that is created from the web UI: sign in as the admin,
**Companies → New company**. That is the only place tenants are made — there is
no public registration.
## 4. Run
```bash
LISTEN_ADDR=127.0.0.1:8080 ./bv-server
```
Open <http://127.0.0.1:8080>. The API and the web app are the same binary on the
same port; in production Traefik puts `platform.loyaly.ai` in front of it.
## Accounts in the local demo
| who | sign in | sees |
|---|---|---|
| Platform admin | `admin@loyaly.ai` / `loyaly-platform-2026` | Companies only |
| TeNext owner | `suriya@tenext.in` / `tenext-2026` | Shops, Live, Cameras, Customers, Reports |
One address is one account across the whole platform, so the same email cannot
be both a platform admin and a tenant user. See CLAUDE.md.
## Optional: the broker, for live arrivals
Only needed to watch visits arrive in real time. Reports and Customers work
without it.
```bash
docker run -d --name bv-mqtt -p 51883:1883 \
-v "$PWD/mosquitto:/mosquitto/config" eclipse-mosquitto:2
```
The config needs a `passwd` file containing the server's own broker user and one
user per site, and an `acl` granting the server `read bv/#` and each site
`write bv/<client>.<site>/#`. `provision site` prints the site's password once.
Then run the server with `MQTT_URL`, `MQTT_USERNAME` and `MQTT_PASSWORD` set.
## Onboarding a shop, the way a customer is onboarded
1. **Company** — sign in as the platform admin, **Companies → New company**.
The owner's password is shown once.
2. **Shop** — `provision site -client <slug> -site <slug> ...`, then add the
broker user it prints to Mosquitto. Still a command; see the note in
CLAUDE.md about what would have to change for this to be self-service.
3. **Installation code** — sign in as the owner, **Shops → the shop →
Set up a shop PC → Create an installation code**. Shown once, works once.
4. **The shop PC** — open Behavision on it and type the code into **Set up this
PC**. It comes back with the broker credentials, its own API token and its
topic prefix, and starts publishing immediately.
5. **Cameras** — **Cameras → Set up a camera** at head office. The shop PC picks
it up within about two minutes.
6. **Prove it** — **Shops → the shop → Run the check**.
## Running a real shop PC on this machine
The engine and the agent both honour `BEHAVISION_DATA_DIR`, so a checkout can
act as a shop PC:
```bash
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
cd agent && CGO_ENABLED=0 go build -o ../.local/behavision-agent . && cd ..
# agent.json: written by the desktop app's "Set up this PC" screen in real life.
# Locally, redeem a code by hand and write client_slug / site_slug / broker
# credentials / agent_token into $BEHAVISION_DATA_DIR/agent.json.
BEHAVISION_DATA_DIR=$PWD ./.local/behavision-agent run
```
The agent starts the engine itself — do not also run `python -m behavision run`,
or two processes fight over one camera and one SQLite WAL. It passes the
bridge's URL to the engine through `BEHAVISION_WEBHOOK_URL`, and reads the
engine's generated Basic credential from `data/api_credentials.txt`.
`./.local/behavision-agent status` prints what the agent can see of the engine —
the fastest way to tell "the engine is down" from "the agent cannot talk to it".
## Cameras
Onboard one from **Cameras → Set up a camera** on the platform. The shop PC picks it
up within about two minutes and it turns from *Waiting for the shop PC* to
*Connected* once the agent has actually reached it.
Cameras already configured on a shop PC appear here on their own — the agent
offers them up on its first sync, so switching this on does not disturb a site
that is already running.
The picture on each card is the engine's **latest frame**, uploaded about once a
minute, not live video: the camera stream lives on the shop PC's loopback behind
a router, and putting live video in a browser at head office needs a relay this
deployment does not have.
Camera passwords are encrypted with `BEHAVISION_SECRET_KEY`. Without that key
set, a camera can be saved but not with a password, and the API says so rather
than storing it blank.
## Proving a camera works
**Cameras → Set up a camera** walks it in four steps: name and make, connection
details, a connection test, then a walk-past check. Picking the make fills in
the RTSP path, which is the field nobody can find.
A camera is only "verified" once someone has walked past it. Until then the card
says *Not checked yet* even when the stream is connected — those are different
claims, and a camera can stream perfectly while producing views nothing can
recognise.
**Cameras → Is this shop working?** runs the end-to-end check: PC online,
recognition running, cameras connected, faces recognisable, visits arriving. It
stops at the first failure rather than guessing past it.
Checks are jobs the shop PC picks up on its next sync, so allow a couple of
minutes — or restart the agent to make it sync now.
## The assistant
Set `ANTHROPIC_API_KEY` before starting the server and the **Ask** button
appears in the top bar. Without it the server logs that the assistant is off,
the panel says so, and nothing else changes.
It answers from the same business questions the screens ask — shops, cameras,
footfall, sales, customers — and can request a camera check. Every tool runs as
the signed-in user, so it can only see what that person could already see, and
staff are refused camera checks the same way the UI refuses them.
Uses `claude-sonnet-5`; set `BEHAVISION_ASSISTANT_MODEL` to change it without a
rebuild. A conversation is capped at 8 tool round trips.
If your key is **identity-linked** (issued against a user rather than an org),
it also needs `ANTHROPIC_WORKSPACE_ID` — a `wrkspc_...` id from
console.anthropic.com → Settings → Workspaces. Without it every Anthropic
endpoint returns 400, and the server says so by name rather than reporting a
generic fault. A classic API key needs nothing extra.
## Tests
```bash
cd server && go test ./... # no database needed
cd server && TEST_DATABASE_URL="$DATABASE_URL" go test ./... # adds the SQL tests
cd agent && go test ./...
```
The live database tests skip themselves when `TEST_DATABASE_URL` is unset, so
the suite stays runnable with no services — the same rule the bucket tests and
the Python tests follow.
## Windows binaries, from a Mac
```bash
cd desktop/frontend && npm run build # go:embed needs frontend/dist
cd .. && GOOS=windows CGO_ENABLED=0 go build -o behavision-desktop.exe .
cd ../agent && GOOS=windows CGO_ENABLED=0 go build -o behavision-agent.exe .
```
These compile. They have never been RUN on Windows — no `wails build`, no
installer, no code signing.