Loyaly/Behavision
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 22 Wiki Activity

Behavision: face recognition for retail, edge to head office

Browse Source 
Five components that ship as one product:

- behavision/  the recognition engine. RTSP ingest, YuNet detection, IoU
               tracking, ArcFace embeddings, a FAISS/SQLite gallery, and a
               FastAPI dashboard. Identity is decided once per TRACK from an
               average of at least three embeddings, never per frame.
- agent/       the Go edge agent: supervises the engine, holds a durable
               spool, and drains it to MQTT. Nothing is acked before the
               broker confirms.
- desktop/     the shop PC application (Wails + React + tray).
- server/      the cloud API, MQTT consumer, reports and assistant.
- web/         platform.loyaly.ai, the head-office app, embedded in the
               server binary.

The gallery stores 512-float embeddings and timestamps - no images unless
`app.store_faces` is switched on. Those embeddings are biometric personal
data under GDPR and India's DPDP: template inversion reconstructs a
recognisable face from an ArcFace vector, so data/behavision.db is treated
as a biometric database and DELETE /api/visitors/{id} is a real erasure.

CLAUDE.md carries the reasoning behind every non-obvious decision here,
including the ones that were measured and the ones that were wrong first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HViLj9gYNRtSr7YVZmW5sn
This commit is contained in:
Suriyakumarvijayanayagam
2026-09-04 11:14:18 +05:30
commit dad04e8cda
216 changed files with 40473 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

291
agent/pkg/cameras/cameras.go Normal file
View File

@@ -0,0 +1,291 @@
// Package cameras keeps a shop PC's cameras in step with head office.
//
// The split is forced by the network, not by taste: only this PC is on the
// camera's LAN, so only this PC can connect to it — but the person onboarding a
// camera is often in an office somewhere else. So head office holds the DESIRED
// configuration and the agent pulls it.
//
// Pull, never push. A shop PC sits behind a router with no inbound route, so it
// has to ask; and asking makes the whole thing idempotent — a sync that fails
// halfway is fixed by the next one rather than leaving two systems disagreeing.
package cameras
import (
"context"
"log"
"time"
)
// Engine is the local recognition engine's camera API.
type Engine interface {
List(ctx context.Context) ([]Local, error)
Add(ctx context.Context, cam Local) error
Update(ctx context.Context, id string, cam Local) error
Remove(ctx context.Context, id string) error
Snapshot(ctx context.Context, id string) ([]byte, error)
}
// Cloud is head office.
type Cloud interface {
Desired(ctx context.Context) ([]Desired, error)
Report(ctx context.Context, rep Report) error
UploadSnapshot(ctx context.Context, jpeg []byte) (key string, err error)
}
// Local is a camera as the engine holds it.
type Local struct {
ID string `json:"id"`
Label string `json:"label,omitempty"`
Host string `json:"host,omitempty"`
Port int `json:"port,omitempty"`
Path string `json:"path,omitempty"`
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
MaxWidth int `json:"max_width,omitempty"`
Tuning map[string]any `json:"tuning,omitempty"`
Connected bool `json:"connected"`
}
// Desired is a camera as head office holds it.
type Desired struct {
CameraID string `json:"camera_id"`
Label string `json:"label"`
Host string `json:"host"`
Port int `json:"port"`
Path string `json:"path"`
Username string `json:"username"`
Password string `json:"password"`
MaxWidth int `json:"max_width"`
Tuning map[string]any `json:"tuning"`
Enabled bool `json:"enabled"`
Revision int64 `json:"revision"`
Deleted bool `json:"deleted"`
}
type State struct {
CameraID string `json:"camera_id"`
Connected bool `json:"connected"`
SnapshotKey string `json:"snapshot_key,omitempty"`
}
type Report struct {
State []State `json:"state,omitempty"`
Adopt []Desired `json:"adopt,omitempty"`
}
// Syncer reconciles the two, on a timer.
type Syncer struct {
Engine Engine
Cloud Cloud
Log *log.Logger
// Every how often to reconcile configuration. Cameras change rarely, and
// each sync is a database read on the server for every site in the estate,
// so this is minutes rather than seconds.
Interval time.Duration
// How often to send a fresh picture of each camera. A shop floor does not
// change much, and each frame is a few tens of kilobytes uploaded over the
// same connection the visits have to travel on.
SnapshotEvery time.Duration
// Checks and Prober are the "prove this camera works" half. Both nil on a
// PC that has never been claimed, and the syncer simply skips that work
// rather than treating it as a failure.
Checks Checks
Prober Prober
// applied remembers the revision last pushed into the engine, so an
// unchanged site costs one request and no engine calls at all.
applied map[string]int64
}
const (
DefaultInterval = 2 * time.Minute
DefaultSnapshotEvery = 60 * time.Second
)
// New builds a fully wired Syncer from the two clients every caller already
// has.
//
// It exists because the four fields were assembled by hand at each call site
// and both of them - the headless agent and the desktop app - set Engine and
// Cloud and forgot Checks and Prober. runChecks returns silently when either
// is nil (correct: an unclaimed PC has neither), so pressing "Test connection"
// at head office left the camera saying "checking..." until the five-minute
// stale release, and then said nothing at all. No error, on either side.
//
// The same two objects satisfy all four interfaces, so there was never a
// reason for a caller to choose.
func New(eng *EngineClient, cloud *CloudClient, log *log.Logger) *Syncer {
return &Syncer{Engine: eng, Cloud: cloud, Checks: cloud, Prober: eng, Log: log}
}
// Run reconciles until ctx is cancelled.
func (s *Syncer) Run(ctx context.Context) {
interval, snapEvery := s.Interval, s.SnapshotEvery
if interval <= 0 {
interval = DefaultInterval
}
if snapEvery <= 0 {
snapEvery = DefaultSnapshotEvery
}
// Immediately on start, so a PC that has just been claimed picks up its
// cameras now rather than in two minutes.
s.Once(ctx)
config := time.NewTicker(interval)
defer config.Stop()
snaps := time.NewTicker(snapEvery)
defer snaps.Stop()
for {
select {
case <-ctx.Done():
return
case <-config.C:
s.Once(ctx)
case <-snaps.C:
s.report(ctx)
}
}
}
// Once performs one full reconcile: pull desired, apply, then report back.
func (s *Syncer) Once(ctx context.Context) {
if s.applied == nil {
s.applied = map[string]int64{}
}
desired, err := s.Cloud.Desired(ctx)
if err != nil {
// Not fatal and not even unusual: an unclaimed PC has no credentials
// and a disconnected one has no network. The engine keeps running the
// cameras it already has, which is the whole point of the local store.
s.logf("camera sync: %v", err)
return
}
local, err := s.Engine.List(ctx)
if err != nil {
s.logf("camera sync: engine unavailable: %v", err)
return
}
have := map[string]Local{}
for _, c := range local {
have[c.ID] = c
}
known := map[string]bool{}
for _, d := range desired {
known[d.CameraID] = true
_, exists := have[d.CameraID]
switch {
case d.Deleted || !d.Enabled:
if exists {
if err := s.Engine.Remove(ctx, d.CameraID); err != nil {
s.logf("camera %s: remove failed: %v", d.CameraID, err)
continue
}
s.logf("camera %s removed (head office)", d.CameraID)
}
delete(s.applied, d.CameraID)
case !exists:
if err := s.Engine.Add(ctx, toLocal(d)); err != nil {
s.logf("camera %s: add failed: %v", d.CameraID, err)
continue
}
s.applied[d.CameraID] = d.Revision
s.logf("camera %s added from head office", d.CameraID)
case s.applied[d.CameraID] != d.Revision:
// The revision is what keeps this cheap. Without it every sync
// would PATCH every camera, and a PATCH restarts the connection —
// so a healthy site would drop its own video every two minutes.
if err := s.Engine.Update(ctx, d.CameraID, toLocal(d)); err != nil {
s.logf("camera %s: update failed: %v", d.CameraID, err)
continue
}
s.applied[d.CameraID] = d.Revision
s.logf("camera %s updated to revision %d", d.CameraID, d.Revision)
}
}
// Anything running here that head office has never heard of gets offered
// up. Without this, switching the feature on would delete every camera an
// existing site is already running — including the one it was commissioned
// with. The server refuses to overwrite its own config with these, and
// keeps tombstones, so a deleted camera is not resurrected.
var adopt []Desired
for id, c := range have {
if known[id] {
continue
}
adopt = append(adopt, Desired{
CameraID: id, Label: orElse(c.Label, id), Host: c.Host,
Port: c.Port, Path: c.Path, Username: c.Username,
Password: c.Password, MaxWidth: c.MaxWidth, Tuning: c.Tuning,
Enabled: true,
})
}
s.reportWith(ctx, adopt)
// Last, so a check requested against a camera added in the same breath
// finds it already applied. Inside Once() rather than beside it in the
// loop, so it also runs on startup and cannot be called twice a tick.
s.runChecks(ctx, desired)
}
func (s *Syncer) report(ctx context.Context) { s.reportWith(ctx, nil) }
// reportWith sends observed state, and a fresh picture from each camera.
func (s *Syncer) reportWith(ctx context.Context, adopt []Desired) {
local, err := s.Engine.List(ctx)
if err != nil {
s.logf("camera report: engine unavailable: %v", err)
return
}
rep := Report{Adopt: adopt}
for _, c := range local {
st := State{CameraID: c.ID, Connected: c.Connected}
if c.Connected {
// A snapshot failure never blocks the state report. Knowing a
// camera is down matters far more than having a picture of it,
// and the picture is the part most likely to fail.
if jpeg, err := s.Engine.Snapshot(ctx, c.ID); err == nil && len(jpeg) > 0 {
if key, err := s.Cloud.UploadSnapshot(ctx, jpeg); err == nil {
st.SnapshotKey = key
} else {
s.logf("camera %s: snapshot upload failed: %v", c.ID, err)
}
}
}
rep.State = append(rep.State, st)
}
if len(rep.State) == 0 && len(rep.Adopt) == 0 {
return
}
if err := s.Cloud.Report(ctx, rep); err != nil {
s.logf("camera report: %v", err)
}
}
func toLocal(d Desired) Local {
return Local{
ID: d.CameraID, Label: d.Label, Host: d.Host, Port: d.Port,
Path: d.Path, Username: d.Username, Password: d.Password,
MaxWidth: d.MaxWidth, Tuning: d.Tuning,
}
}
func orElse(s, fallback string) string {
if s == "" {
return fallback
}
return s
}
func (s *Syncer) logf(format string, args ...any) {
if s.Log != nil {
s.Log.Printf(format, args...)
}
}

241
agent/pkg/cameras/cameras_test.go Normal file
View File

@@ -0,0 +1,241 @@
package cameras
import (
"context"
"errors"
"testing"
)
type fakeEngine struct {
cams map[string]Local
added []string
updated []string
removed []string
listErr error
snapshot []byte
}
func newEngine(cams ...Local) *fakeEngine {
m := map[string]Local{}
for _, c := range cams {
m[c.ID] = c
}
return &fakeEngine{cams: m, snapshot: []byte("\xff\xd8jpeg")}
}
func (f *fakeEngine) List(context.Context) ([]Local, error) {
if f.listErr != nil {
return nil, f.listErr
}
var out []Local
for _, c := range f.cams {
out = append(out, c)
}
return out, nil
}
func (f *fakeEngine) Add(_ context.Context, c Local) error {
f.cams[c.ID] = c
f.added = append(f.added, c.ID)
return nil
}
func (f *fakeEngine) Update(_ context.Context, id string, c Local) error {
c.ID = id
f.cams[id] = c
f.updated = append(f.updated, id)
return nil
}
func (f *fakeEngine) Remove(_ context.Context, id string) error {
delete(f.cams, id)
f.removed = append(f.removed, id)
return nil
}
func (f *fakeEngine) Snapshot(context.Context, string) ([]byte, error) {
return f.snapshot, nil
}
type fakeCloud struct {
desired []Desired
desiredErr error
reports []Report
uploads int
uploadErr error
}
func (f *fakeCloud) Desired(context.Context) ([]Desired, error) {
return f.desired, f.desiredErr
}
func (f *fakeCloud) Report(_ context.Context, r Report) error {
f.reports = append(f.reports, r)
return nil
}
func (f *fakeCloud) UploadSnapshot(context.Context, []byte) (string, error) {
if f.uploadErr != nil {
return "", f.uploadErr
}
f.uploads++
return "snap/key.jpg", nil
}
func syncer(e *fakeEngine, c *fakeCloud) *Syncer {
return &Syncer{Engine: e, Cloud: c}
}
func TestACameraAddedAtHeadOfficeAppearsOnTheShopPC(t *testing.T) {
e, c := newEngine(), &fakeCloud{desired: []Desired{{
CameraID: "entrance", Label: "Entrance", Host: "192.168.0.138",
Port: 554, Path: "/ch0_0.264", Username: "admin", Password: "s3cret",
MaxWidth: 1280, Enabled: true, Revision: 1,
}}}
syncer(e, c).Once(context.Background())
got, ok := e.cams["entrance"]
if !ok {
t.Fatal("the camera was never created on the PC")
}
if got.Host != "192.168.0.138" || got.Password != "s3cret" {
t.Fatalf("connection details did not travel: %+v", got)
}
}
// The whole reason adoption exists. Every existing site is already running
// cameras configured locally - including the office camera this was tested with
// - and a reconcile that only pushed downwards would delete all of them the
// first time it ran.
func TestACameraAlreadyRunningLocallyIsOfferedToHeadOfficeNotDeleted(t *testing.T) {
e := newEngine(Local{ID: "office", Label: "Office", Host: "192.168.0.138",
Port: 554, Username: "admin", Password: "s3cret", Connected: true})
c := &fakeCloud{}
syncer(e, c).Once(context.Background())
if _, ok := e.cams["office"]; !ok {
t.Fatal("an existing camera was deleted by the first sync")
}
if len(c.reports) == 0 || len(c.reports[0].Adopt) != 1 {
t.Fatalf("the camera was not offered for adoption: %+v", c.reports)
}
if got := c.reports[0].Adopt[0]; got.CameraID != "office" || got.Password != "s3cret" {
t.Fatalf("adoption dropped details the camera needs: %+v", got)
}
}
// A tombstone must win over adoption, or a deleted camera comes straight back
// on the next sync and the operator cannot work out why.
func TestADeletedCameraIsRemovedAndNotReadopted(t *testing.T) {
e := newEngine(Local{ID: "entrance", Host: "10.0.0.5", Connected: true})
c := &fakeCloud{desired: []Desired{
{CameraID: "entrance", Enabled: true, Revision: 3, Deleted: true},
}}
s := syncer(e, c)
s.Once(context.Background())
if _, ok := e.cams["entrance"]; ok {
t.Fatal("a camera deleted at head office is still running")
}
for _, r := range c.reports {
for _, a := range r.Adopt {
if a.CameraID == "entrance" {
t.Fatal("the deleted camera was offered back for adoption")
}
}
}
}
// A PATCH restarts the camera connection, so re-applying unchanged config every
// two minutes would make a healthy site drop its own video permanently.
func TestUnchangedConfigurationTouchesTheEngineOnce(t *testing.T) {
e := newEngine()
c := &fakeCloud{desired: []Desired{{CameraID: "entrance", Host: "10.0.0.5",
Enabled: true, Revision: 7}}}
s := syncer(e, c)
s.Once(context.Background())
s.Once(context.Background())
s.Once(context.Background())
if len(e.added) != 1 {
t.Fatalf("added %d times, want 1", len(e.added))
}
if len(e.updated) != 0 {
t.Fatalf("updated %d times with no change - every one restarts the stream", len(e.updated))
}
}
func TestANewRevisionIsApplied(t *testing.T) {
e := newEngine()
c := &fakeCloud{desired: []Desired{{CameraID: "entrance", Host: "10.0.0.5",
Enabled: true, Revision: 1}}}
s := syncer(e, c)
s.Once(context.Background())
c.desired[0].Host = "10.0.0.9"
c.desired[0].Revision = 2
s.Once(context.Background())
if len(e.updated) != 1 {
t.Fatalf("updated %d times, want 1", len(e.updated))
}
if e.cams["entrance"].Host != "10.0.0.9" {
t.Fatalf("the new address was not applied: %+v", e.cams["entrance"])
}
}
// An unclaimed PC, or one with no internet, must keep running the cameras it
// already has. Wiping local config because head office is unreachable would
// stop a shop recognising anybody for the duration of an outage.
func TestAnUnreachableHeadOfficeChangesNothingLocally(t *testing.T) {
e := newEngine(Local{ID: "entrance", Host: "10.0.0.5", Connected: true})
c := &fakeCloud{desiredErr: errors.New("this PC is not claimed by a company yet")}
syncer(e, c).Once(context.Background())
if _, ok := e.cams["entrance"]; !ok {
t.Fatal("local cameras were removed because the cloud was unreachable")
}
if len(e.removed) != 0 {
t.Fatalf("removed %v", e.removed)
}
}
// Knowing a camera is down matters far more than having a picture of it, and
// the picture is the part most likely to fail.
func TestAFailedSnapshotStillReportsWhetherTheCameraIsUp(t *testing.T) {
e := newEngine(Local{ID: "entrance", Connected: true})
c := &fakeCloud{uploadErr: errors.New("bucket unreachable")}
syncer(e, c).Once(context.Background())
if len(c.reports) == 0 || len(c.reports[0].State) != 1 {
t.Fatalf("no state was reported: %+v", c.reports)
}
st := c.reports[0].State[0]
if !st.Connected {
t.Error("connected state was lost with the snapshot")
}
if st.SnapshotKey != "" {
t.Error("a failed upload reported a key anyway")
}
}
// No point photographing a camera that is not producing frames, and the attempt
// costs a request per sync per dead camera.
func TestADisconnectedCameraIsNotPhotographed(t *testing.T) {
e := newEngine(Local{ID: "entrance", Connected: false})
c := &fakeCloud{}
syncer(e, c).Once(context.Background())
if c.uploads != 0 {
t.Fatalf("uploaded %d snapshots of a disconnected camera", c.uploads)
}
if c.reports[0].State[0].Connected {
t.Error("a disconnected camera was reported as up")
}
}
func TestAnEngineThatIsNotRunningIsNotAnError(t *testing.T) {
e := newEngine()
e.listErr = errors.New("connection refused")
c := &fakeCloud{desired: []Desired{{CameraID: "entrance", Enabled: true, Revision: 1}}}
syncer(e, c).Once(context.Background()) // must not panic
if len(c.reports) != 0 {
t.Fatal("reported state it could not have observed")
}
}

247
agent/pkg/cameras/checks.go Normal file
View File

@@ -0,0 +1,247 @@
package cameras
import (
"context"
"encoding/base64"
"fmt"
"strings"
"time"
)
// Running the checks head office asks for.
//
// Both answers come from the engine, which already knows how to give them and
// already phrases them for whoever is standing next to the camera. Nothing here
// re-words a verdict; it carries one.
// Job is one check the shop PC has been asked to run.
type Job struct {
CameraID string `json:"camera_id"`
Kind string `json:"kind"`
Seconds int `json:"seconds"`
}
// Result is what it found.
type Result struct {
CameraID string `json:"camera_id"`
OK bool `json:"ok"`
Verdict string `json:"verdict,omitempty"`
Headline string `json:"headline,omitempty"`
Advice []string `json:"advice,omitempty"`
Detail map[string]any `json:"detail,omitempty"`
ImageKey string `json:"image_key,omitempty"`
}
// Prober is the half of the engine that answers "does this camera work".
type Prober interface {
// Test opens the stream once and lets go, returning a frame. The engine's
// probe checks TCP reachability first, so a wrong address answers in
// milliseconds instead of the ~75 s an FFmpeg connect would take.
Test(ctx context.Context, cam Local) (TestResult, error)
// Placement watches for `seconds` and judges whether a person walking past
// produced a view worth enrolling.
Placement(ctx context.Context, cameraID string, seconds int) (map[string]any, error)
}
type TestResult struct {
OK bool `json:"ok"`
Error string `json:"error,omitempty"`
Width int `json:"width,omitempty"`
Height int `json:"height,omitempty"`
// Snapshot is base64 JPEG, the operator's proof that the camera is
// pointing where they think it is.
Snapshot string `json:"snapshot_jpeg_b64,omitempty"`
}
// Checks is the client for the job queue.
type Checks interface {
Pending(ctx context.Context) ([]Job, error)
Submit(ctx context.Context, res Result) error
}
// runChecks picks up whatever head office has asked for and answers it.
//
// Called from the same sync loop as configuration, so a check requested at head
// office is picked up on the next tick. Deliberately not its own faster poll: a
// placement check needs a human to walk about anyway, so shaving a minute off
// the request buys nothing an operator would notice.
func (s *Syncer) runChecks(ctx context.Context, desired []Desired) {
if s.Checks == nil || s.Prober == nil {
return
}
jobs, err := s.Checks.Pending(ctx)
if err != nil {
s.logf("camera checks: %v", err)
return
}
for _, job := range jobs {
res := s.runOne(ctx, job, desired)
if err := s.Checks.Submit(ctx, res); err != nil {
// Nothing to retry against: the server released the claim on a
// timeout, so the operator's next press starts a fresh one. Losing
// a result is better than a queue of stale verdicts.
s.logf("camera %s: could not report the check: %v", job.CameraID, err)
}
}
}
func (s *Syncer) runOne(ctx context.Context, job Job, desired []Desired) Result {
res := Result{CameraID: job.CameraID}
local, err := s.Engine.List(ctx)
if err != nil {
res.Headline = "the recognition software on this PC is not responding"
res.Advice = []string{"Open Behavision on the shop's PC and make sure it is started."}
return res
}
var cam Local
var found bool
for _, c := range local {
if c.ID == job.CameraID {
cam, found = c, true
break
}
}
if !found {
// The camera exists at head office but the PC has not applied it yet.
// Honest, and it tells the operator to wait rather than to go and look
// at the cabling.
res.Headline = "this PC has not set up that camera yet"
res.Advice = []string{"It is applied within a couple of minutes of being added. Try again shortly."}
return res
}
// The engine never returns a camera password - by design, it reports
// `has_password` and nothing else - so probing with what it hands back
// dials the camera with an empty credential. That failed, and reported
// "could not open stream - check the host, port, path and credentials"
// about a camera the same PC had been streaming for an hour, with advice
// sending the installer to check the very credential that was never sent.
//
// Head office has the real one, and this sync already fetched it.
if cam.Password == "" {
for _, d := range desired {
if d.CameraID == job.CameraID {
cam.Password = d.Password
break
}
}
}
switch job.Kind {
case "placement":
return s.runPlacement(ctx, job, cam)
default:
return s.runConnection(ctx, job, cam)
}
}
func (s *Syncer) runConnection(ctx context.Context, job Job, cam Local) Result {
res := Result{CameraID: job.CameraID}
out, err := s.Prober.Test(ctx, cam)
if err != nil {
res.Headline = "could not test the camera: " + err.Error()
return res
}
if !out.OK {
res.Verdict = "unreachable"
// The engine's own sentence. It distinguishes a refused connection from
// a wrong path from a stream that opens and never sends a frame, and
// those need three different things done about them.
res.Headline = out.Error
res.Advice = adviceFor(out.Error)
return res
}
res.OK = true
res.Verdict = "reachable"
res.Headline = fmt.Sprintf("connected — %d×%d", out.Width, out.Height)
res.Detail = map[string]any{"width": out.Width, "height": out.Height}
res.Advice = []string{
"Check the picture below is the view you expect.",
"Then run a walk-past check to prove faces here can actually be recognised.",
}
if out.Snapshot != "" {
if jpeg, err := base64.StdEncoding.DecodeString(out.Snapshot); err == nil {
if key, err := s.Cloud.UploadSnapshot(ctx, jpeg); err == nil {
res.ImageKey = key
} else {
// A missing picture does not invalidate the result: the camera
// still connected, which is what was asked.
s.logf("camera %s: check snapshot upload failed: %v", job.CameraID, err)
}
}
}
return res
}
func (s *Syncer) runPlacement(ctx context.Context, job Job, cam Local) Result {
res := Result{CameraID: job.CameraID}
seconds := job.Seconds
if seconds <= 0 {
seconds = 25
}
// Room for the watch itself plus the engine's own overhead. Without the
// margin the context dies at the exact moment the verdict is computed.
ctx, cancel := context.WithTimeout(ctx, time.Duration(seconds+30)*time.Second)
defer cancel()
report, err := s.Prober.Placement(ctx, job.CameraID, seconds)
if err != nil {
res.Headline = "the walk-past check could not be run: " + err.Error()
return res
}
res.Detail = report
res.Verdict, _ = report["verdict"].(string)
res.Headline, _ = report["headline"].(string)
if adv, ok := report["advice"].([]any); ok {
for _, a := range adv {
if str, ok := a.(string); ok {
res.Advice = append(res.Advice, str)
}
}
}
// Only `good` is a pass. `marginal` means half the visitors are silently
// discarded, which is not a working camera - calling it one is how a site
// gets signed off and discovered three weeks later from a footfall report
// that was always zero.
res.OK = res.Verdict == "good"
return res
}
// adviceFor turns the engine's diagnosis into the next thing to do.
//
// Matched on the engine's own wording rather than an error code, because the
// engine returns prose - and prose that is already correct. This adds the
// action, it does not restate the problem.
func adviceFor(engineError string) []string {
msg := strings.ToLower(engineError)
switch {
case strings.Contains(msg, "refused"):
return []string{
"Something answered at that address but refused the connection.",
"The port is usually 554 for an RTSP camera. Check the port first.",
}
case strings.Contains(msg, "unreachable"), strings.Contains(msg, "timed out"),
strings.Contains(msg, "no route"):
return []string{
"Nothing answered at that address from the shop's PC.",
"Check the camera is powered on and plugged into the same network as the PC.",
"Confirm the address in the camera's own app or on its label.",
}
case strings.Contains(msg, "could not open"):
return []string{
"The address is reachable but the stream would not open.",
"This is usually the stream path or the camera's username and password.",
"Pick your camera's make above to fill in the usual path for it.",
}
case strings.Contains(msg, "no frame"):
return []string{
"The camera accepted the connection but sent no picture.",
"Some cameras only allow one viewer at a time — close any app watching it.",
}
}
return []string{"Check the address, port, stream path, username and password."}
}

247
agent/pkg/cameras/checks_test.go Normal file
View File

@@ -0,0 +1,247 @@
package cameras
import (
"context"
"errors"
"strings"
"testing"
)
type fakeProber struct {
test TestResult
testErr error
placement map[string]any
placeErr error
placedFor int
// testedWith records the camera the probe was actually handed, which is
// where the credential either arrives or does not.
testedWith Local
}
func (f *fakeProber) Test(_ context.Context, cam Local) (TestResult, error) {
f.testedWith = cam
return f.test, f.testErr
}
func (f *fakeProber) Placement(_ context.Context, _ string, seconds int) (map[string]any, error) {
f.placedFor = seconds
return f.placement, f.placeErr
}
type fakeChecks struct {
jobs []Job
submitted []Result
pendErr error
}
func (f *fakeChecks) Pending(context.Context) ([]Job, error) { return f.jobs, f.pendErr }
func (f *fakeChecks) Submit(_ context.Context, r Result) error {
f.submitted = append(f.submitted, r)
return nil
}
func checker(e *fakeEngine, p *fakeProber, c *fakeChecks) *Syncer {
return &Syncer{Engine: e, Cloud: &fakeCloud{}, Prober: p, Checks: c}
}
func TestAReachableCameraReportsItsResolutionAndAPicture(t *testing.T) {
e := newEngine(Local{ID: "entrance", Host: "192.168.0.138", Connected: true})
p := &fakeProber{test: TestResult{OK: true, Width: 1920, Height: 1080,
Snapshot: "/9j/4AAQSkZJRg=="}}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "connection"}}}
checker(e, p, c).runChecks(context.Background(), nil)
if len(c.submitted) != 1 {
t.Fatalf("submitted %d results", len(c.submitted))
}
got := c.submitted[0]
if !got.OK {
t.Fatalf("a working camera reported as failing: %+v", got)
}
if !strings.Contains(got.Headline, "1920") {
t.Errorf("headline does not say what was found: %q", got.Headline)
}
if got.ImageKey == "" {
t.Error("no picture uploaded, so the operator cannot see what it is pointing at")
}
}
// The engine already tells three different failures apart, and each needs a
// different thing done about it. Carrying its sentence and adding the action is
// the whole design; re-wording it here would be a fourth description of the
// same fault.
func TestEachConnectionFailureGetsItsOwnAdvice(t *testing.T) {
cases := map[string]string{
"connection refused": "port",
"host unreachable": "powered on",
"could not open stream - check the host, port, path and credentials": "stream path",
"connected but no frame arrived within 12s": "one viewer at a time",
}
for engineErr, want := range cases {
e := newEngine(Local{ID: "entrance", Connected: true})
p := &fakeProber{test: TestResult{OK: false, Error: engineErr}}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "connection"}}}
checker(e, p, c).runChecks(context.Background(), nil)
got := c.submitted[0]
if got.OK {
t.Errorf("%q reported as a pass", engineErr)
}
// The engine's own sentence must survive intact.
if got.Headline != engineErr {
t.Errorf("headline %q, want the engine's own words %q", got.Headline, engineErr)
}
joined := strings.ToLower(strings.Join(got.Advice, " "))
if !strings.Contains(joined, want) {
t.Errorf("%q -> advice %q, expected it to mention %q", engineErr, joined, want)
}
}
}
// Only `good` is a pass. `marginal` means half the visitors are silently
// discarded, and signing that off as working is exactly how a site runs for
// weeks recognising almost nobody.
func TestOnlyAGoodPlacementCounts(t *testing.T) {
for verdict, wantOK := range map[string]bool{
"good": true, "marginal": false, "poor": false,
"no_faces": false, "artifact": false, "inconclusive": false,
"no_completed_passes": false,
} {
e := newEngine(Local{ID: "entrance", Connected: true})
p := &fakeProber{placement: map[string]any{
"verdict": verdict, "headline": "h",
"advice": []any{"do the thing"},
}}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "placement", Seconds: 25}}}
checker(e, p, c).runChecks(context.Background(), nil)
if got := c.submitted[0].OK; got != wantOK {
t.Errorf("verdict %q -> ok=%v, want %v", verdict, got, wantOK)
}
}
}
// The engine's advice is written for the person standing next to the camera.
// It must reach them.
func TestThePlacementAdviceIsCarriedThroughVerbatim(t *testing.T) {
e := newEngine(Local{ID: "entrance", Connected: true})
p := &fakeProber{placement: map[string]any{
"verdict": "poor",
"headline": "most visitors here cannot be recognised",
"advice": []any{
"Face the camera the way people walk in, at about head height.",
"Re-run this check after moving it.",
},
"faces": float64(11),
}}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "placement", Seconds: 25}}}
checker(e, p, c).runChecks(context.Background(), nil)
got := c.submitted[0]
if got.Headline != "most visitors here cannot be recognised" {
t.Errorf("headline changed: %q", got.Headline)
}
if len(got.Advice) != 2 || !strings.Contains(got.Advice[0], "head height") {
t.Errorf("advice did not survive: %+v", got.Advice)
}
// Everything else the engine said travels too, so a new field reaches the
// UI without a schema change on the way.
if got.Detail["faces"] != float64(11) {
t.Errorf("detail was dropped: %+v", got.Detail)
}
if p.placedFor != 25 {
t.Errorf("watched for %ds, want 25", p.placedFor)
}
}
// A camera added at head office 30 seconds ago has not reached the PC yet.
// Telling the operator to check the cabling would send them to the wrong place.
func TestACameraTheShopPCHasNotAppliedYetSaysSo(t *testing.T) {
e := newEngine() // engine knows nothing about it
p := &fakeProber{}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "connection"}}}
checker(e, p, c).runChecks(context.Background(), nil)
got := c.submitted[0]
if got.OK {
t.Fatal("reported a pass for a camera that does not exist here")
}
if !strings.Contains(got.Headline, "not set up that camera yet") {
t.Errorf("headline blames the wrong thing: %q", got.Headline)
}
if !strings.Contains(strings.Join(got.Advice, " "), "Try again shortly") {
t.Errorf("advice does not tell them to wait: %+v", got.Advice)
}
}
// "The engine is not running" and "the camera is broken" need opposite actions.
func TestAStoppedEngineIsNotReportedAsABrokenCamera(t *testing.T) {
e := newEngine()
e.listErr = errors.New("connection refused")
p := &fakeProber{}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "connection"}}}
checker(e, p, c).runChecks(context.Background(), nil)
got := c.submitted[0]
if !strings.Contains(got.Headline, "not responding") {
t.Fatalf("headline blames the camera: %q", got.Headline)
}
if !strings.Contains(strings.Join(got.Advice, " "), "make sure it is started") {
t.Errorf("advice: %+v", got.Advice)
}
}
// An unclaimed PC has neither, and must not treat that as a fault.
func TestASyncerWithNoCheckSupportSkipsQuietly(t *testing.T) {
s := &Syncer{Engine: newEngine(), Cloud: &fakeCloud{}}
s.runChecks(context.Background(), nil) // must not panic
}
func TestNoPendingChecksSubmitsNothing(t *testing.T) {
c := &fakeChecks{}
checker(newEngine(), &fakeProber{}, c).runChecks(context.Background(), nil)
if len(c.submitted) != 0 {
t.Fatalf("submitted %d results with no jobs", len(c.submitted))
}
}
// The engine deliberately never returns a camera password - it reports
// has_password and nothing else - so probing with what the engine hands back
// dials the camera with an empty credential. That reported "could not open
// stream - check the host, port, path and credentials" about a camera the very
// same PC had been streaming for an hour, and sent the installer to check the
// one thing that had never been sent.
func TestTheProbeIsGivenThePasswordHeadOfficeHolds(t *testing.T) {
e := newEngine(Local{ID: "entrance", Host: "192.168.0.138", Username: "admin",
Connected: true}) // no Password: the engine does not return one
p := &fakeProber{test: TestResult{OK: true, Width: 1920, Height: 1080}}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "connection"}}}
desired := []Desired{{CameraID: "entrance", Username: "admin", Password: "hunter2"}}
checker(e, p, c).runChecks(context.Background(), desired)
if p.testedWith.Password != "hunter2" {
t.Fatalf("the probe was handed password %q - a working camera would be "+
"reported unreachable", p.testedWith.Password)
}
}
// A password the engine DOES have is not overwritten by head office's copy:
// the local one is what the camera is actually being streamed with.
func TestALocalPasswordWins(t *testing.T) {
e := newEngine(Local{ID: "entrance", Password: "local", Connected: true})
p := &fakeProber{test: TestResult{OK: true}}
c := &fakeChecks{jobs: []Job{{CameraID: "entrance", Kind: "connection"}}}
checker(e, p, c).runChecks(context.Background(),
[]Desired{{CameraID: "entrance", Password: "remote"}})
if p.testedWith.Password != "local" {
t.Fatalf("probe used %q, want the engine's own", p.testedWith.Password)
}
}

263
agent/pkg/cameras/clients.go Normal file
View File

@@ -0,0 +1,263 @@
package cameras
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
)
// EngineClient talks to the recognition engine on this PC's loopback.
type EngineClient struct {
Base string
User string
Password string
Client *http.Client
}
func NewEngineClient(base, user, password string) *EngineClient {
return &EngineClient{
Base: strings.TrimRight(base, "/"), User: user, Password: password,
// Generous, because adding a camera makes the engine dial it, and a
// wrong address takes the full RTSP timeout to fail. Shorter than that
// and every genuinely-bad camera looks like an engine fault instead.
Client: &http.Client{Timeout: 30 * time.Second},
}
}
func (e *EngineClient) do(ctx context.Context, method, path string, body, out any) error {
var rdr io.Reader
if body != nil {
b, err := json.Marshal(body)
if err != nil {
return err
}
rdr = bytes.NewReader(b)
}
req, err := http.NewRequestWithContext(ctx, method, e.Base+path, rdr)
if err != nil {
return err
}
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
if e.User != "" {
req.SetBasicAuth(e.User, e.Password)
}
resp, err := e.Client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
// The engine's message, not just a status. "camera stored but failed to
// start: connection refused" is something an operator can act on;
// "500" is not, and this string ends up in the agent log a support
// engineer reads.
msg, _ := io.ReadAll(io.LimitReader(resp.Body, 2048))
return fmt.Errorf("engine %s: %s", resp.Status, strings.TrimSpace(string(msg)))
}
if out == nil {
return nil
}
return json.NewDecoder(io.LimitReader(resp.Body, 1<<20)).Decode(out)
}
func (e *EngineClient) List(ctx context.Context) ([]Local, error) {
var out []Local
return out, e.do(ctx, http.MethodGet, "/api/cameras", nil, &out)
}
func (e *EngineClient) Add(ctx context.Context, cam Local) error {
return e.do(ctx, http.MethodPost, "/api/cameras", cam, nil)
}
func (e *EngineClient) Update(ctx context.Context, id string, cam Local) error {
// The engine takes the id from the path on PATCH and refuses it in the
// body, so it is cleared here rather than at the call site.
cam.ID = ""
return e.do(ctx, http.MethodPatch, "/api/cameras/"+url.PathEscape(id), cam, nil)
}
func (e *EngineClient) Remove(ctx context.Context, id string) error {
return e.do(ctx, http.MethodDelete, "/api/cameras/"+url.PathEscape(id), nil, nil)
}
// Snapshot fetches the most recent frame the engine holds.
//
// Not a fresh capture: the engine already keeps the latest frame in memory for
// its own MJPEG stream, so this costs a memory copy rather than a camera round
// trip. A camera that has not produced a frame yet answers 503, which is a
// normal state on a just-added camera and not an error worth logging loudly.
func (e *EngineClient) Snapshot(ctx context.Context, id string) ([]byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet,
e.Base+"/api/cameras/"+url.PathEscape(id)+"/frame.jpg", nil)
if err != nil {
return nil, err
}
if e.User != "" {
req.SetBasicAuth(e.User, e.Password)
}
resp, err := e.Client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("engine %s", resp.Status)
}
// Bounded. A frame is tens of kilobytes; anything near this cap means
// something other than a JPEG is on the other end.
return io.ReadAll(io.LimitReader(resp.Body, 8<<20))
}
// CloudClient talks to head office with this agent's own token.
type CloudClient struct {
Base string
Token string
Client *http.Client
// Upload is the existing image path: the server mints a presigned URL and
// the agent PUTs to it. Reused rather than reimplemented, so a shop PC
// still never holds bucket credentials — the reason that path exists.
Upload func(ctx context.Context, jpeg []byte) (string, error)
}
func NewCloudClient(base, token string) *CloudClient {
return &CloudClient{
Base: strings.TrimRight(base, "/"), Token: token,
Client: &http.Client{Timeout: 20 * time.Second},
}
}
func (c *CloudClient) do(ctx context.Context, method, path string, body, out any) error {
if c.Token == "" {
// An unclaimed PC. Said plainly, because this is the normal state
// between installing the software and typing an enrolment code, and it
// must not read as a fault in the log.
return fmt.Errorf("this PC is not claimed by a company yet")
}
var rdr io.Reader
if body != nil {
b, err := json.Marshal(body)
if err != nil {
return err
}
rdr = bytes.NewReader(b)
}
req, err := http.NewRequestWithContext(ctx, method, c.Base+path, rdr)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+c.Token)
if body != nil {
req.Header.Set("Content-Type", "application/json")
}
resp, err := c.Client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("head office: %s", resp.Status)
}
if out == nil || resp.StatusCode == http.StatusNoContent {
return nil
}
return json.NewDecoder(io.LimitReader(resp.Body, 4<<20)).Decode(out)
}
func (c *CloudClient) Desired(ctx context.Context) ([]Desired, error) {
var body struct {
Cameras []Desired `json:"cameras"`
}
if err := c.do(ctx, http.MethodGet, "/api/agent/cameras", nil, &body); err != nil {
return nil, err
}
return body.Cameras, nil
}
func (c *CloudClient) Report(ctx context.Context, rep Report) error {
return c.do(ctx, http.MethodPost, "/api/agent/cameras", rep, nil)
}
func (c *CloudClient) UploadSnapshot(ctx context.Context, jpeg []byte) (string, error) {
if c.Upload == nil {
return "", fmt.Errorf("images are not enabled for this deployment")
}
return c.Upload(ctx, jpeg)
}
// ---------------------------------------------------------------- probing
// Test opens the candidate stream once, without saving it.
//
// The engine does this as a sync handler in its threadpool because
// cv2.VideoCapture blocks hard, and it checks TCP reachability first - so a
// wrong address, which is the single most likely thing anybody types, answers
// in milliseconds rather than the ~75 s an FFmpeg connect takes to give up.
func (e *EngineClient) Test(ctx context.Context, cam Local) (TestResult, error) {
var out TestResult
// A generous ceiling: the engine's own deadline is 12 s for the frame plus
// 3 s to connect, and cutting it off earlier would report a timeout of our
// own making as if it were the camera's.
ctx, cancel := context.WithTimeout(ctx, 45*time.Second)
defer cancel()
return out, e.do(ctx, http.MethodPost, "/api/cameras/test", cam, &out)
}
// Placement runs the engine's commissioning watch and returns its report whole.
//
// Polled rather than awaited: the engine starts the watch and answers
// immediately, so the run survives this request being retried, and the report
// arrives with `running: true` until it does not.
func (e *EngineClient) Placement(ctx context.Context, cameraID string, seconds int) (
map[string]any, error) {
path := "/api/cameras/" + url.PathEscape(cameraID) + "/commission"
var report map[string]any
if err := e.do(ctx, http.MethodPost, path,
map[string]any{"seconds": seconds}, &report); err != nil {
return nil, err
}
deadline := time.Now().Add(time.Duration(seconds+20) * time.Second)
for time.Now().Before(deadline) {
select {
case <-ctx.Done():
return report, ctx.Err()
case <-time.After(2 * time.Second):
}
var latest map[string]any
if err := e.do(ctx, http.MethodGet, path, nil, &latest); err != nil {
// Keep the last good report rather than losing the whole run to
// one failed poll - the engine may simply have been busy.
continue
}
report = latest
if running, _ := latest["running"].(bool); !running {
return report, nil
}
}
return report, nil
}
// ---------------------------------------------------------------- check jobs
func (c *CloudClient) Pending(ctx context.Context) ([]Job, error) {
var body struct {
Checks []Job `json:"checks"`
}
if err := c.do(ctx, http.MethodGet, "/api/agent/checks", nil, &body); err != nil {
return nil, err
}
return body.Checks, nil
}
func (c *CloudClient) Submit(ctx context.Context, res Result) error {
return c.do(ctx, http.MethodPost, "/api/agent/checks", res, nil)
}

29
agent/pkg/cameras/wiring_test.go Normal file
View File

@@ -0,0 +1,29 @@
package cameras
import (
"log"
"testing"
)
// The bug this guards: both callers built the Syncer as a struct literal, set
// Engine and Cloud, and left Checks and Prober nil. runChecks returns silently
// when either is nil - correct for an unclaimed PC - so a camera check
// requested at head office was claimed by nobody and sat at "checking..." until
// the server released it five minutes later. Nothing logged, on either side.
func TestNewWiresEveryHalfOfTheSyncer(t *testing.T) {
eng := NewEngineClient("http://127.0.0.1:8010", "u", "p")
cloud := NewCloudClient("https://example.invalid", "token")
s := New(eng, cloud, log.Default())
if s.Engine == nil || s.Cloud == nil {
t.Fatal("configuration half not wired")
}
// The half that proves a camera works. A Syncer without these is not a
// broken Syncer, which is exactly why the omission was invisible.
if s.Checks == nil {
t.Error("Checks is nil: head office's camera checks would never be claimed")
}
if s.Prober == nil {
t.Error("Prober is nil: a claimed check could never be answered")
}
}