Three uuids on one arrival, three different answers
Asked of the row the feed actually returns. site_id had a reference all along and the feed was not sending it. A client could read the shop's NAME off an arrival and still had no way to ask for that shop except by uuid - the exact gap the reference scheme exists to close. site_slug now travels with it. visit_id stays a uuid and needs no reference: no route takes it, it is a key a client de-duplicates on because delivery is at-least-once, and nobody says a visit id out loud. The uuid in a face URL must STAY random. visit_faces.id is gen_random_uuid() and a derived or sequential one would let somebody walk a shop's customers by date - the same reason bucket keys are random rather than derived from the event id. A readable identifier is right for a customer and wrong for the thing that points at their photograph. And seq is now json:"-". visits.seq is a plain bigserial, so it counts every visit on the PLATFORM, and shipping it put the total footfall of every customer we have on every row of every tenant's feed - the same German-tank estimate that decided visitors.number had to be per client. It was a convenience for "have I fallen behind", nothing ever read it, and the cursor answers that without disclosing a number. The SSE event id was never the raw value; it has always been the opaque cursor. The one test that broke was reading seq back off the wire to assert the cursor pointed at the last row of a burst. It asserts against the seeded position now: the property is unchanged, and the test can no longer see what a client cannot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HViLj9gYNRtSr7YVZmW5sn
This commit is contained in:
17
API.md
17
API.md
@@ -220,9 +220,10 @@ in. Reactivating restores the account but not their old sessions.
|
|||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"arrivals": [{
|
"arrivals": [{
|
||||||
"visit_id": "…", "seq": 412,
|
"visit_id": "…",
|
||||||
"occurred_at": "2026-09-05T06:01:45Z",
|
"occurred_at": "2026-09-05T06:01:45Z",
|
||||||
"site_id": "…", "site": "TeNext Chennai", "camera_id": "Office1",
|
"site_id": "…", "site": "TeNext Chennai", "site_slug": "chennai",
|
||||||
|
"camera_id": "Office1",
|
||||||
"visitor_id": "…", "visitor_ref": "V-42", "label": "Priya",
|
"visitor_id": "…", "visitor_ref": "V-42", "label": "Priya",
|
||||||
"is_new_visitor": false, "similarity": 0.71, "quality": 0.66,
|
"is_new_visitor": false, "similarity": 0.71, "quality": 0.66,
|
||||||
"attributes": { "gender": "Male", "age": 32, "emotion": "neutral" },
|
"attributes": { "gender": "Male", "age": 32, "emotion": "neutral" },
|
||||||
@@ -243,6 +244,18 @@ again without one.
|
|||||||
|
|
||||||
An empty poll returns your own cursor back, not an empty string.
|
An empty poll returns your own cursor back, not an empty string.
|
||||||
|
|
||||||
|
**There is no `seq` on the wire.** It existed as a convenience for "have I
|
||||||
|
fallen behind"; `visits.seq` is a plain bigserial, so it counted every visit on
|
||||||
|
the *platform* and put the total footfall of every customer we have on every row
|
||||||
|
of every tenant's feed. The cursor — opaque and version-prefixed — is the
|
||||||
|
supported way to know your position, and the only one you need.
|
||||||
|
|
||||||
|
Of the three ids on an arrival, only one of them is a reference you would type:
|
||||||
|
`site_slug`. `visit_id` addresses no route — it is a key for de-duplicating
|
||||||
|
rows, since delivery is at-least-once. And the uuid inside an image URL is
|
||||||
|
**deliberately random**: a derived or sequential one would let somebody
|
||||||
|
enumerate a shop's customers by date.
|
||||||
|
|
||||||
### `GET /api/visits/stream` — server-sent events
|
### `GET /api/visits/stream` — server-sent events
|
||||||
|
|
||||||
The same rows, pushed. Send `Authorization` (so `EventSource` will not do —
|
The same rows, pushed. Send `Authorization` (so `EventSource` will not do —
|
||||||
|
|||||||
30
CLAUDE.md
30
CLAUDE.md
@@ -2047,6 +2047,36 @@ the engine's own diagnostic dashboard.
|
|||||||
human name. The prop carrying it is `customerRef`, not `ref` — React reserves
|
human name. The prop carrying it is `customerRef`, not `ref` — React reserves
|
||||||
that name, so it would never have reached the component.
|
that name, so it would never have reached the component.
|
||||||
|
|
||||||
|
### Three uuids on one arrival, three different answers
|
||||||
|
|
||||||
|
Asked of the row the feed actually returns, and they do not get the same reply:
|
||||||
|
|
||||||
|
- **`site_id`** had a reference all along and the feed was not sending it. A
|
||||||
|
client could read the shop's *name* off an arrival and still had no way to ask
|
||||||
|
for that shop except by uuid, which is the exact gap the scheme exists to
|
||||||
|
close. `site_slug` now travels with it.
|
||||||
|
- **`visit_id` stays a uuid, and needs no reference.** No route takes it; it is
|
||||||
|
a key a client de-duplicates on, because delivery is at-least-once. Nobody
|
||||||
|
says a visit id out loud.
|
||||||
|
- **The uuid in a face URL must STAY random.** `visit_faces.id` is
|
||||||
|
`gen_random_uuid()` and a derived or sequential one would let somebody walk a
|
||||||
|
shop's customers by date — the same reason object keys in the bucket are
|
||||||
|
random rather than derived from the event id. A readable identifier is right
|
||||||
|
for a customer and wrong for the thing that points at their photograph.
|
||||||
|
|
||||||
|
And one field left with it: **`seq` is now `json:"-"`**. `visits.seq` is a plain
|
||||||
|
bigserial, so it counts every visit on the *platform*, and shipping it put the
|
||||||
|
total footfall of every customer we have on every row of every tenant's feed —
|
||||||
|
the same German-tank estimate that decided `visitors.number` had to be per
|
||||||
|
client. It was there as a convenience for *"have I fallen behind"*, nothing ever
|
||||||
|
read it, and the cursor already answers that question without disclosing a
|
||||||
|
number. The SSE event id was never the raw value; it has always been the opaque
|
||||||
|
cursor.
|
||||||
|
|
||||||
|
The one test that broke was reading `seq` back off the wire to assert the cursor
|
||||||
|
pointed at the last row of a burst. It asserts against the seeded position now —
|
||||||
|
the property is unchanged, and the test can no longer see what a client cannot.
|
||||||
|
|
||||||
Fixture note: `embedding(seed)` fills every dimension with one value, so after
|
Fixture note: `embedding(seed)` fills every dimension with one value, so after
|
||||||
L2 normalisation 0.31 and 0.62 are the **same direction** and the matcher
|
L2 normalisation 0.31 and 0.62 are the **same direction** and the matcher
|
||||||
correctly calls them one person. Tests that need several different people use
|
correctly calls them one person. Tests that need several different people use
|
||||||
|
|||||||
@@ -86,9 +86,12 @@ func TestFourPeopleArrivingTogetherComeBackInOneRequest(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("cursor from a burst is unreadable: %v", err)
|
t.Fatalf("cursor from a burst is unreadable: %v", err)
|
||||||
}
|
}
|
||||||
if seq != page.Arrivals[3].Seq {
|
// Against the SEEDED position, not one read back off the wire: `seq` is
|
||||||
|
// json:"-" because it counts every visit on the platform, so a client can
|
||||||
|
// no longer see it - and the cursor is the whole reason it does not need to.
|
||||||
|
if want := int64(4); seq != want {
|
||||||
t.Errorf("cursor should point at the LAST row of the burst, got %d want %d",
|
t.Errorf("cursor should point at the LAST row of the burst, got %d want %d",
|
||||||
seq, page.Arrivals[3].Seq)
|
seq, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package api
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -108,3 +109,34 @@ func TestBothSiteParameterNamesAreAccepted(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// An arrival names its shop three ways, and the one a client can filter by must
|
||||||
|
// be among them. Reading "TeNext Chennai" off a row and then having no way to
|
||||||
|
// ask for that shop except by uuid is the exact gap the reference scheme
|
||||||
|
// exists to close.
|
||||||
|
func TestAnArrivalCarriesTheShopReferenceItCanBeFilteredBy(t *testing.T) {
|
||||||
|
srv, fs := newServer(t)
|
||||||
|
seedUser(fs)
|
||||||
|
fs.arrivals = []Arrival{{
|
||||||
|
VisitID: "c64dc53f-c7f0-4e61-a3b4-9a230f52b3a3", Seq: 265,
|
||||||
|
SiteID: "7c9bb456-e0c6-436d-ba01-5d6cc4e3f466",
|
||||||
|
Site: "TeNext Chennai", SiteSlug: "chennai",
|
||||||
|
}}
|
||||||
|
tok := login(t, srv, "manager@acme.com", "correct horse battery").Token
|
||||||
|
|
||||||
|
rec := do(t, srv, http.MethodGet, "/api/visits", tok, nil)
|
||||||
|
if rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("got %d: %s", rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
body := rec.Body.String()
|
||||||
|
if !strings.Contains(body, `"site_slug":"chennai"`) {
|
||||||
|
t.Fatalf("an arrival must carry the shop's reference: %s", body)
|
||||||
|
}
|
||||||
|
|
||||||
|
// seq is a plain bigserial, so it counts every visit on the PLATFORM. It
|
||||||
|
// must not travel: that is the total footfall of every customer we have,
|
||||||
|
// on every row of every tenant's feed.
|
||||||
|
if strings.Contains(body, `"seq"`) {
|
||||||
|
t.Fatalf("the platform-wide visit counter leaked into the feed: %s", body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -238,13 +238,26 @@ type AgentPrincipal struct {
|
|||||||
type Arrival struct {
|
type Arrival struct {
|
||||||
VisitID string `json:"visit_id"`
|
VisitID string `json:"visit_id"`
|
||||||
// Seq is this visit's position in the feed - assigned by the server when it
|
// Seq is this visit's position in the feed - assigned by the server when it
|
||||||
// learned of the visit, not by the camera. Exposed because a client that
|
// learned of the visit, not by the camera. It drives the cursor and the
|
||||||
// wants to know whether it has fallen behind can compare two of them; the
|
// ordering, and it is `json:"-"` on purpose.
|
||||||
// cursor remains the supported way to page.
|
//
|
||||||
Seq int64 `json:"seq"`
|
// `visits.seq` is a plain bigserial, so it counts every visit on the
|
||||||
|
// PLATFORM, not this tenant's. Sending it put the total footfall of every
|
||||||
|
// customer we have on every row of every feed - the same German-tank
|
||||||
|
// estimate that decided `visitors.number` had to be per client, and a
|
||||||
|
// number no tenant should be able to read off another. It used to ship as
|
||||||
|
// a convenience for "have I fallen behind"; nothing ever read it, and the
|
||||||
|
// cursor - opaque and version-prefixed for exactly this reason - already
|
||||||
|
// answers that.
|
||||||
|
Seq int64 `json:"-"`
|
||||||
OccurredAt string `json:"occurred_at"`
|
OccurredAt string `json:"occurred_at"`
|
||||||
SiteID string `json:"site_id"`
|
SiteID string `json:"site_id"`
|
||||||
Site string `json:"site"`
|
Site string `json:"site"`
|
||||||
|
// SiteSlug is the shop's reference - "chennai" - and is what `?site=`
|
||||||
|
// takes. Without it a client could read the shop's NAME off an arrival and
|
||||||
|
// still had no way to filter by that shop except the uuid, which is the
|
||||||
|
// gap the whole reference scheme exists to close.
|
||||||
|
SiteSlug string `json:"site_slug,omitempty"`
|
||||||
CameraID string `json:"camera_id"`
|
CameraID string `json:"camera_id"`
|
||||||
IsNew bool `json:"is_new_visitor"`
|
IsNew bool `json:"is_new_visitor"`
|
||||||
Similarity float64 `json:"similarity,omitempty"`
|
Similarity float64 `json:"similarity,omitempty"`
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import (
|
|||||||
// mean the first poll of a feed and every poll after it returned different
|
// mean the first poll of a feed and every poll after it returned different
|
||||||
// shapes, which is the kind of bug that only shows up under load.
|
// shapes, which is the kind of bug that only shows up under load.
|
||||||
const arrivalColumns = `
|
const arrivalColumns = `
|
||||||
vi.id::text, vi.seq, vi.occurred_at, vi.site_id::text, si.name, vi.camera_id,
|
vi.id::text, vi.seq, vi.occurred_at, vi.site_id::text, si.name, si.slug, vi.camera_id,
|
||||||
vi.is_new_visitor, vi.similarity, vi.quality, vi.attributes, vi.image_key,
|
vi.is_new_visitor, vi.similarity, vi.quality, vi.attributes, vi.image_key,
|
||||||
COALESCE(vi.visitor_id::text, ''),
|
COALESCE(vi.visitor_id::text, ''),
|
||||||
COALESCE(vs.number, 0),
|
COALESCE(vs.number, 0),
|
||||||
@@ -113,7 +113,7 @@ func (s *Store) Arrivals(ctx context.Context, q api.ArrivalQuery) ([]api.Arrival
|
|||||||
var sim, qual *float64
|
var sim, qual *float64
|
||||||
var imageKey string
|
var imageKey string
|
||||||
var number int64
|
var number int64
|
||||||
if err := rows.Scan(&a.VisitID, &a.Seq, &at, &a.SiteID, &a.Site, &a.CameraID,
|
if err := rows.Scan(&a.VisitID, &a.Seq, &at, &a.SiteID, &a.Site, &a.SiteSlug, &a.CameraID,
|
||||||
&a.IsNew, &sim, &qual, &a.Attributes, &imageKey,
|
&a.IsNew, &sim, &qual, &a.Attributes, &imageKey,
|
||||||
&a.VisitorID, &number, &a.Label, &a.Name); err != nil {
|
&a.VisitorID, &number, &a.Label, &a.Name); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
|
|||||||
Reference in New Issue
Block a user