Compare commits

..

5 Commits

Author SHA1 Message Date
bb5f40926f Add the assistant tool registry and its first tool
Phase 1 of Nearle Buddy: an agent names a tool, and the registry decides
whether that is allowed, whether the arguments make sense, who is asking,
and what gets recorded — then runs a handler a person wrote and tested.

No agent gets raw table access. The usual argument for tools over
generated SQL is safety; here there is a harder one. The fields on this
backend do not mean what their names say, and it is measured:
orders.deliverystatus is an empty string on all 181 rows of tenant 1147,
orders.orderstatus never carries the six middle delivery stages,
deliveries.ridername holds statuses as often as names, deliverytype is
empty on every row in production. A model writing SQL gets each of those
wrong with no error — it reports a cancel rate from a column of empty
strings and nobody can tell. A model calling a tool cannot, because the
correction lives in the handler beside the measurement that justified it.

Call does five things in order: find the tool, check the agent's
allow-list, validate arguments, confirm the caller is scoped to
something, run the handler — writing exactly one audit row whatever
happens, refusals included. A trail of successes answers "did anything
try to read another tenant?" with silence, which reads the same as no.

The model has no say in whose data is read. stuck_orders has no tenantid
field on its schema — absent, not rejected — and the tenant comes from
the session claims added in the previous commit. Arguments the tool did
not declare are dropped rather than passed on, so a model sending a
`where` clause gets it discarded.

stuck_orders: deliveries a rider was given and has not accepted, ten
minutes for a look, twenty-five for somebody now. Derived from assigntime
and orderstatus, so it does not depend on anyone having been watching.
Carries the wait in minutes, what to do, where to check it, and what it
covered. A capped answer says so — an empty result and a truncated one
look identical to a model and it will call both "none".

The audit sink writes to the log for now; a database sink is phase 8.
Nothing calls the registry yet: the loop and the model gateway are phase 2.

37 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-23 11:22:57 +05:30
c516c224e5 Authenticate the console's /web surface
The /web endpoints have never had authentication. The console keeps its
login record in per-tab sessionStorage and sends no Authorization header,
so every endpoint under /v1/web read `tenantid` off the query string and
believed it — one number in a URL reached another merchant's orders,
stock, staff and takings. `createposuser` under /v1/web/tenants minted
till credentials on the strength of an unauthenticated request, which the
route file already flagged in as many words.

Closed the same way posauth.go closed it for the terminals, in the same
order: the caller holds a token this server signed, and the tenant they
name is the tenant inside that token.

- utils/webtoken.go   same HMAC construction as the POS token, 12h TTL,
                      a `w1.` prefix so the two kinds cannot verify as
                      each other
- middleware/webauth.go  verifies the token, pins the tenant, and checks
                      a named branch belongs to it; reads the tenant from
                      the query, the body, and inside a JSON array, since
                      createdeliveries posts one
- login now issues the token; the console sends it as Bearer

Platform access rides on issuperadmin and nothing else. Not the role —
app_roles calls roleid 1 "Super admin" and tenant onboarding wrote 1 for
every shop owner, so a role test would promote every merchant on the
platform. Not a zero tenant either, or a user row with the field unset
becomes the one session that reads everything. Both near-misses have
tests.

WEB_AUTH_REQUIRED defaults to off. The console in production does not
send a token yet, and enforcing before it does would lock every merchant
out of a working product. A token that IS sent is always verified, and
one naming the wrong tenant is always refused; the flag only decides what
happens to a request carrying none. This should be a short-lived state.

Still trusting the caller: partnerid, customerid and appuserid, which
some list endpoints also scope on. Noted in the middleware header.

25 tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-23 11:22:20 +05:30
771d6a51cf Merge scan-to-order: ambiguous-label candidates and match method 2026-09-23 11:21:16 +05:30
24339a8b51 Merge origin/main: keep the substring rule, read its tie
main had moved on with retrieval work validated against real queries —
minTokenHits (the word match needs two thirds of the label, not all of
it), separator folding so "Parle G"/"Parle-G"/"ParleG" all reach Parle-G,
the floor at 0.50 after "Paracetamol" came back as "Paneer Makhni 500ml"
at 0.304, and ties broken on cosine distance instead of name. All of that
is kept exactly as it was.

The conflict was in textScore: this branch replaced the substring rule
with a coverage formula to stop a bare brand name resolving to one
arbitrary product. That is the wrong half to change. The substring rule
scores every product of a brand 0.95 IDENTICALLY, and that tie is not the
bug — it is the signal. isAmbiguous reads it, so the branch's coverage
rewrite is dropped and the ambiguity layer alone does the work:

  "britannia" → all 258 rows tie at 0.95 → ambiguous: true + candidates
  "Parle G"   → folding and the single-character token still land it
  a real name → runner-up far behind → match, unchanged

Dropped with it: scanSpecificEnough, the per-hit text score, and the
proportional confirmation bonus — the flat +0.10 is back. Simpler, and it
leaves main's tuning untouched.

TestTextScoreRewardsSpecificityNotJustOverlap tested the removed formula
and is replaced by TestABrandNameScoresItsProductsIdentically, which
guards the tie itself: a formula that broke it on name length or word
count would bring the bug back.

Docs carry both rationales, and now say plainly that confidence stays
high on the ambiguous path — gate on `ambiguous`, never on `confidence`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 11:05:03 +05:30
01bc89ab77 Ask instead of guessing when a label fits several products
`"britannia"` is a substring of all 258 Britannia product names, and
textScore returned 0.95 for any product whose name contained the label.
So every one of them tied, the tie broke alphabetically, and the customer
was shown one arbitrary biscuit with "confidence": 0.95 and a price. Lens
hands back a bare wordmark often — it is usually the biggest thing printed
on a packet — so this was the common case, not an edge one. Found via the
example request in the mobile team's own proposal.

Scoring now asks both questions. A hit carries `score` (ranks) and `text`
(how specifically the label names THIS product: the harmonic mean of how
much of the label the product explains and how much of the product's name
the label explains, pack sizes dropped from both sides). A brand name
scores its products ~0.33 equally instead of 0.95 arbitrarily. The
"vector and text agree" bonus is now proportional to the text score, so a
weak match can no longer inflate a whole brand.

isAmbiguous reads that: the leader is a guess if anything is level with it
(margin) or if the label names no one product (specificity), and then the
response carries `ambiguous: true` with `candidates` — distinct products,
not pack sizes, at most ten, each marked with whether one of the
customer's stores has it in stock, available ones first. `match` is nil
and `stores` empty on that path: no price for a product nobody chose.
Erring towards asking is deliberate — a tap versus the wrong biscuit.

To act on a pick, /lookup now accepts `brand` + `catalogueid` instead of a
label and skips recognition entirely (also serves deep links and re-order).
New: ScanRepository.CatalogueRef, resolving via the brand tables discovered
from information_schema, never a name built from the request.

Also: scratch/cataloguedims now reports every vector column, not just
`embedding` — which is how we learned the catalogue also carries
img_vector(1024), filled on 1885 of 2124 rows. SCAN_TO_ORDER.md records
why that column stays unread for now and what would change it, alongside
why the app is not asked to compute vectors on the phone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 10:56:02 +05:30
18 changed files with 3284 additions and 56 deletions

View File

@@ -1,16 +1,63 @@
package controllers
import (
"log"
"net/http"
"strconv"
"strings"
"time"
"nearle/models"
"nearle/services"
"nearle/utils"
"github.com/gofiber/fiber/v2"
)
// attachWebSession hands a signed-in console user their session token.
//
// Added to the login response rather than served from a second endpoint, so the
// console receives it on the call it already makes and nothing changes about
// when or how it signs in.
//
// The claims come from the user's own record, which is the whole point: until
// now the console asserted its tenant on every request and was believed, and
// sealing it under a signature here is what makes `middleware.WebAuth` able to
// refuse a request naming somebody else's.
//
// `Issuperadmin` is copied across as the ONLY source of cross-tenant access.
// Not the role — `app_roles` calls roleid 1 "Super admin" and tenant onboarding
// wrote 1 for every shop owner, so trusting the role would promote every
// merchant on the platform.
//
// A failure to mint is logged and swallowed, deliberately, while
// WEB_AUTH_REQUIRED is off: a deployment that has not set a signing key yet must
// still be able to sign in, or shipping this takes the console down everywhere
// the secret is missing. Once enforcement is on, no token means no session —
// which is then the correct and loud failure.
//
// The parameter is the underlying map type rather than `fiber.Map`, because the
// two login paths do not agree on which fiber that is: `AppLogin` returns the
// v1 package's `Map` and `TenantWebLogin` the v2 one. Both are
// `map[string]any`, so taking that accepts either without dragging the
// old import into this file.
func attachWebSession(resp map[string]any, info models.TenantUserInfo) {
token, expires, err := utils.MintWebToken(utils.WebClaims{
Userid: info.Userid,
Tenantid: info.Tenantid,
Locationid: info.Locationid,
Roleid: info.Roleid,
Configid: info.Configid,
Superadmin: info.Issuperadmin,
}, time.Now())
if err != nil {
log.Printf("login: could not issue a console session for user %d: %v", info.Userid, err)
return
}
resp["token"] = token
resp["tokenexpiresat"] = expires.Unix()
}
type UserController struct {
userService services.UserService
}
@@ -179,7 +226,7 @@ func (ctl *UserController) AppLogin(c *fiber.Ctx) error {
})
}
_, resp, err := ctl.userService.AppLogin(user)
info, resp, err := ctl.userService.AppLogin(user)
if err != nil {
// Use resp.Code if present, fallback to 409
code := http.StatusConflict
@@ -189,6 +236,8 @@ func (ctl *UserController) AppLogin(c *fiber.Ctx) error {
return c.Status(code).JSON(resp)
}
attachWebSession(resp, info)
// ✅ Always return resp
return c.Status(http.StatusOK).JSON(resp)
}
@@ -244,6 +293,7 @@ func (ctl *UserController) TenantWebLogin(c *fiber.Ctx) error {
// Include tenant user info if login successful (code 200)
if code == fiber.StatusOK {
resp["details"] = info
attachWebSession(resp, info)
}
return c.Status(code).JSON(resp)

View File

@@ -8,6 +8,10 @@ recommend. When the customer taps a store and a size, a second call confirms
the shelf still has it — and if it does not, names the next-nearest store
that does.
When the label fits several products — `"britannia"` names 258 of them — it
answers with a short "did you mean?" list instead of picking one, because a
confident price on the wrong biscuit is worse than one extra tap.
Base path: `/live/api/v1/mob/scan`. Every response uses the usual envelope
`{ code, status, message, details }`; the shapes below are `details`.
@@ -17,7 +21,13 @@ Base path: `/live/api/v1/mob/scan`. Every response uses the usual envelope
photo ──Lens──▶ label
│
▼
POST /lookup ───▶ match + stores[] (recommended first)
POST /lookup ───▶ ambiguous:true + candidates[] "did you mean?"
│ │
│ customer taps one candidate
│ │
│ POST /lookup { brand, catalogueid }
│ │
└───▶ match + stores[] (recommended first) ◀──┘
│
customer taps a store + a size
│
@@ -26,11 +36,21 @@ photo ──Lens──▶ label
ok:false + alternative → offer the other store
```
**`/lookup` has two possible answers and the app must handle both.** A label
that names one product comes back with `match` + `stores`. A label that fits
several — a bare brand name like `"britannia"`, a generic word like
`"biscuits"` — comes back with `ambiguous: true` and `candidates`, and the
app asks the customer which one before any price is shown. Lens returns a
bare wordmark often, because it is usually the biggest thing printed on a
packet, so this is a normal path and not an error case.
`GET /stores` is for the "choose another shop" sheet: the customer's
registered stores, nearest first, independent of any product.
## `POST /lookup`
Note the `//` notes below are annotations, not JSON — strip them.
```json
{
"customerid": 5123,
@@ -39,16 +59,25 @@ registered stores, nearest first, independent of any product.
"longitude": 77.0290,
"tenantids": [1135, 1140], // optional: what the app THINKS the customer joined
"limit": 0 // optional: max stores, 0 = all
// Instead of a label: name the product outright. This is how you resolve
// a candidate the customer tapped, and how a deep link or a "buy again"
// skips recognition. With both set, `label` is ignored.
// "brand": "britannia", "catalogueid": 7
}
```
`label` is required **unless** `brand` and `catalogueid` are both given.
`tenantids` is verified, never trusted: the server intersects it with the
`tenantcustomers` table. Ids the customer is not actually registered with
come back in `unregistered_tenantids` — treat that as "refresh the local
list". A list that matches nothing at all is treated as stale and all
registered stores are used.
Response:
### Response A — one product identified
`ambiguous: false`, `match` set, `candidates` empty.
```json
{
@@ -59,6 +88,8 @@ Response:
"image": "https://…", "score": 0.94, "method": "vector+text"
},
"catalogue_variants": [ { "…same shape…": "100 g" }, { "…": "200 g" } ],
"ambiguous": false,
"candidates": [],
"confidence": 0.94,
"available": true,
"recommended_locationid": 20,
@@ -83,12 +114,57 @@ Response:
}
```
How to read it:
### Response B — several products fit, none clearly
- `match == null` → nothing recognised; show `message` and let them retry.
`confidence` below ~0.5 → recognised but unsure; confirm the name with the
customer before showing prices. `method: "text"` means no embedding model
was involved (not configured, or it timed out) — be a little more cautious.
`ambiguous: true`, `match: null`, `stores: []`. Show a "did you mean?" list.
```json
{
"label": "britannia",
"match": null,
"ambiguous": true,
"candidates": [
{ "brand": "britannia", "catalogueid": 23, "product_name": "Britannia Marie Gold",
"size": "250 g", "image": "https://…", "score": 0.95, "method": "text", "available": true },
{ "brand": "britannia", "catalogueid": 22, "product_name": "Britannia Good Day Butter Cookies",
"image": "https://…", "score": 0.95, "method": "text" },
{ "brand": "britannia", "catalogueid": 21, "product_name": "Britannia Good Day Cashew Cookies",
"image": "https://…", "score": 0.95, "method": "text" }
],
"confidence": 0.95,
"available": false,
"stores": [],
"catalogue_variants": [],
"message": "Which one is it? 1 of these 3 are in stock near you."
}
```
- **`confidence` is not low here, and that is not a bug.** "britannia" really
does appear in all three names, so relevance is high — what is missing is
*identification*. Gate on `ambiguous`, never on `confidence`: an app that
reads 0.95 as "sure enough to show a price" reintroduces the exact bug this
path exists to prevent.
- **`available` on a candidate** means at least one of the customer's
registered stores has it in stock right now. Candidates are ordered
available-first, so the list can show what is buyable before what is not
— and the field is absent (not `false`) when unavailable, so read it as
falsy, not as a required key.
- **To resolve a pick**, call `/lookup` again with that candidate's `brand`
and `catalogueid` and no label. You get Response A for that exact product,
with `method: "direct"` and `confidence: 1`.
- At most 10 candidates come back.
### How to read either response
- `match == null && !ambiguous` → nothing recognised; show `message` and let
them retry with a clearer photo.
- `ambiguous: true` → ask, do not guess. Never show a price on this path;
`stores` is deliberately empty.
- `confidence` below ~0.5 with a `match` → recognised but unsure; worth
confirming the name before showing prices. `method: "text"` means no
embedding model was involved (not configured, or it timed out) — be a
little more cautious. `method: "direct"` means the caller named the
product, so nothing was recognised at all.
- `stores` is ordered **in-stock first, then nearest**. Exactly one store has
`recommended: true` — the nearest with stock — and only when `available`
is true. Stores that sell it but have nothing on the shelf are still listed
@@ -205,6 +281,59 @@ Same `ScanStore` shape as inside `stores[]` above, without options.
- **Identity** is the `customerid` in the body, like every other mobile
endpoint here — there is no auth layer yet (see `SECURITY_HANDOFF.md`).
## Two decisions, and why
Both come from a proposal (2026-09-23) to have the app send vectors it
computed on the phone. Recorded here because the next person will ask.
### The app does not send `textvector`
An on-device MiniLM vector is only comparable to the catalogue's if the app
ships the identical model *and* tokenizer *and* pooling *and* normalisation;
a quantised tflite build usually drifts, and the failure is silent — the
ranking just gets worse. There is also nothing to gain: the server-side
embed is ~30 ms warm and the result is cached in Redis by label, so one
model call serves every customer who scans that product. A client-supplied
vector *defeats* that cache (the key would have to be the vector, not the
label), and 384 floats is ~5 KB of upload against ~12 bytes for
`"Milk Bikis"`. If the field ever arrives it can be accepted and validated,
but the app should not be asked to compute it.
**Send the full OCR text instead** if you want to give the server more to
work with — ~100 bytes, no model coupling, strictly more information than a
single label.
### The app does not send `imagevector` — yet
The catalogue *does* carry image vectors: every `brand_*` table has
`img_vector vector(1024)`, filled on 1885 of 2124 rows (empty in
`brand_haldirams`, `brand_kaleesuwari`, `brand_mdh`, `brand_zzsmoketest`).
That matches the proposed MobileNetV3-Small embedder, so the idea is
coherent and half-built — this flow simply does not read that column.
It stays unread for now because **Google Lens is already the image
recogniser, and a far better one**: photo → Lens → label is Google's product
recognition, trained on billions of images. Putting a 137M-parameter
ImageNet backbone searching 1885 vectors *behind* that adds little where
Lens succeeds, and MobileNetV3-Small — which struggles to tell one blue
biscuit wrapper from another — is unlikely to rescue the cases where Lens
fails. There is also an unverified dependency: the preprocessing the app
would use (BGR → centre crop → 224×224 INTER_AREA → RGB → `/255.0`) has to
match whatever the catalogue pipeline actually ran, or the search returns
confidently-ranked noise.
**What would change this:** the field data. Once live, count how often
`/lookup` returns `ambiguous: true` or nothing recognised. If Lens labels are
reliable, image search is polish; if that number is high, it becomes the
priority — and the first task is the cosine check (embed a known catalogue
product's image through the app's exact pipeline, compare with its stored
`img_vector`; ≈0.99 means the contract holds), not writing the query.
There is one non-recognition argument for it worth remembering: on-device
inference is free and needs no Google dependency, which matters if Cloud
Vision costs start to bite at volume. That is a business reason, not a
quality one.
## For backend developers
### Where the code is
@@ -219,7 +348,7 @@ Same `ScanStore` shape as inside `stores[]` above, without options.
| `utils/embedding.go` | `Embedder` interface, OpenAI-compatible and Gemini clients |
| `utils/geo.go` | coordinate parsing, haversine, opening hours, label tokenising |
| `config/config.go` | `EmbeddingConfig` and its validation |
| `scratch/cataloguedims` | read-only check of the catalogue's embedding width / fill |
| `scratch/cataloguedims` | read-only check of every catalogue vector column's width and fill |
### Try it locally
@@ -238,11 +367,13 @@ anything; a schema-only dump does not.
### Tests
`go test ./services -run 'Lookup|Confirm|Stores|CatalogueFamily'` drives
the whole pipeline through a fake repository (`services/scan_test.go`); no
database. `go test ./utils` covers both HTTP clients against `httptest`
servers, and the geo helpers. Add a case to `scan_test.go`'s fixture when
you change ranking — it is the spec.
`go test ./services -run 'Lookup|Confirm|Stores|Brand|Ambiguous|Specific|TextScore|Distinct|Naming'`
drives the whole pipeline through a fake repository
(`services/scan_test.go`); no database. `go test ./utils` covers both HTTP
clients against `httptest` servers, and the geo helpers. Add a case to
`scan_test.go`'s fixture when you change ranking — it is the spec, and
`newBrandLabelFixture` in particular is the regression guard for the
brand-name bug described under Scoring.
### Knobs (constants in `scanService.go`)
@@ -251,6 +382,8 @@ you change ranking — it is the spec.
| `scanLookupTimeout` | 5 s | whole lookup, including the model call |
| `scanCatalogueTopK` | 15 | rows taken from each brand table and from the merge |
| `scanMinScore` | 0.50 | below this the best hit is not shown as a match |
| `scanAmbiguityMargin` | 0.06 | how close the runner-up may be before the answer becomes a question |
| `scanMaxCandidates` | 10 | longest "did you mean?" list |
| `embedTimeout` (`utils/embedding.go`) | 4 s | one model call |
| `scanVectorTTL` / `scanHitsTTL` (`scanRepository.go`) | 7 d / 30 min | cache lifetimes |
@@ -274,6 +407,19 @@ Classic* at 0.9 (the "G" was dropped, so only "parle" matched either row),
and the name tie-break handed it to Monaco because a space precedes a hyphen
in ASCII. A confident, wrong answer — the kind no score floor can catch.
**When the substring rule ties, that tie is the answer.** A bare brand name
is a substring of every one of that brand's names, so all of them score 0.95
— identically, at a high score no floor would ever catch. Rather than
scoring around it, `isAmbiguous` reads it: if the runner-up is within
`scanAmbiguityMargin` of the leader, the reply becomes `ambiguous: true`
with `candidates` instead of a match (see Response B). Erring towards asking
is deliberate — one tap on a picture against the wrong biscuit. A label that
names one product leaves the runner-up far behind, so the common case is
untouched, and `services/scan_test.go`'s
`TestABrandNameScoresItsProductsIdentically` guards the tie itself: a
formula that broke it on name length or word count would bring the bug
back.
### Changing the embedding model
1. The catalogue team re-embeds `search_query` with the new model.

View File

@@ -4,6 +4,7 @@ import (
"nearle/controllers"
"nearle/repositories"
"nearle/services"
"nearle/services/tools"
"nearle/utils"
"gorm.io/gorm"
@@ -25,6 +26,13 @@ type Facade struct {
CatalogueUploadController *controllers.CatalogueUploadController
ScanController *controllers.ScanController
// Tools is what Nearle Buddy is allowed to do.
//
// Held on the facade because the assistant is not a module with a
// repository of its own — it is a door onto the services already built
// here, and every tool handler calls one of them rather than the database.
Tools *tools.Registry
// Held so the NATS consumer can reach the ingest without going through
// HTTP. Unexported: everything else should use the controller.
posService services.PosService
@@ -119,6 +127,22 @@ func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder) *Faca
scanService := services.NewScanService(scanRepo, embedder)
scanController := controllers.NewScanController(scanService)
// The assistant registry. Built last, because every tool it holds is a thin
// wrapper over a service constructed above.
//
// A registration error panics rather than being logged. A duplicate name or
// a tool with no description is a programming mistake, and a server that
// starts with a tool silently absent answers real questions with "I cannot
// do that" for a reason nobody can see from the outside.
toolRegistry := tools.New(tools.LogAudit{})
for _, tool := range []tools.Tool{
tools.StuckOrders(deliveriesService, nil),
} {
if err := toolRegistry.Register(tool); err != nil {
panic("assistant tools: " + err.Error())
}
}
return &Facade{
UserController: userController,
ProductController: productController,
@@ -134,6 +158,7 @@ func NewFacade(db *gorm.DB, catalogueDB *gorm.DB, embedder utils.Embedder) *Faca
LiveController: liveController,
CatalogueUploadController: catalogueUploadController,
ScanController: scanController,
Tools: toolRegistry,
posService: posService,
}
}

312
middleware/webauth.go Normal file
View File

@@ -0,0 +1,312 @@
package middleware
import (
"encoding/json"
"net/http"
"os"
"strconv"
"strings"
"time"
"nearle/services"
"nearle/utils"
"github.com/gofiber/fiber/v2"
)
// Authorisation for the console.
//
// The `/web` surface has never had any. The console keeps its login record in
// per-tab `sessionStorage` and sends no `Authorization` header, so every
// endpoint under `/v1/web` reads `tenantid` off the query string and believes
// it. Changing one number in a URL reads another merchant's orders, stock,
// staff and takings.
//
// This is the same hole `posauth.go` was written to close on the POS surface,
// and it is closed the same way, in the same order:
//
// 1. the caller holds a token this server signed, and
// 2. the tenant they are naming is the tenant inside that token.
//
// The second is the one that matters. A valid session is not a licence to name
// any tenant — it is a licence to name *your* tenant.
//
// ── Why this could not wait for the assistant ───────────────────────────────
//
// Nearle Buddy answers questions over this same data. Behind REST, reading
// another merchant's books takes knowing the endpoints, knowing the fields and
// iterating. Behind an assistant it is one sentence — "summarise the top ten
// tenants by revenue" — and the model assembles the cross-tenant answer itself,
// accurately and helpfully, because the data was in scope. The permission rules
// the assistant needs have nothing to stand on until this exists.
//
// ── What this does NOT yet do ───────────────────────────────────────────────
//
// It verifies what a request NAMES. It does not yet make handlers derive their
// scope from the session instead of from the wire, and it does not validate
// `partnerid`, `customerid` or `appuserid`, which are the other scoping ids
// some list endpoints accept. Those are the next step, and until they land a
// handler that scopes on one of them is still trusting the caller.
// WebLocalsKey names where the verified claims are parked for handlers.
const WebLocalsKey = "webclaims"
// webAuthRequired reports whether a request without a valid token is refused.
//
// Defaults to OFF, for the same reason POS enforcement does: the console is in
// use by real merchants right now, and its sign-in does not yet hand back a
// token. Switching enforcement on before the console sends one would lock every
// user out of a working product.
//
// So the order is: this middleware ships, sign-in starts issuing tokens, the
// console starts sending them, and `WEB_AUTH_REQUIRED=true` closes the door.
// While it is off a token is still VERIFIED when one is sent, and a request
// carrying a token for the wrong tenant is still refused — the flag only
// decides what happens to a request carrying none.
//
// This is a temporary state and should be short. An unauthenticated `/web`
// surface is the most serious thing in this codebase.
func webAuthRequired() bool {
return strings.EqualFold(strings.TrimSpace(os.Getenv("WEB_AUTH_REQUIRED")), "true")
}
// publicWebPaths are the endpoints that must work before anybody has a token.
//
// Sign-in, chiefly: guarding the login route with a session token means nobody
// can ever obtain one. Kept as suffixes rather than full paths so the group
// prefix can move without silently locking the door.
var publicWebPaths = []string{
"/users/applogin",
"/users/weblogin",
"/tenant/weblogin",
// First-password-set runs before a session exists, from a link in the
// invitation mail.
"/users/setpassword",
}
func isPublicWebPath(path string) bool {
lower := strings.ToLower(path)
for _, suffix := range publicWebPaths {
if strings.HasSuffix(lower, suffix) {
return true
}
}
return false
}
// webLocationChecker is the only question this middleware asks of the database:
// does this tenant own this branch? Narrowed to one method so the guard can be
// tested without a database, and so it cannot quietly grow a second dependency.
type webLocationChecker interface {
LocationAllowed(tenantID, locationID int) (bool, error)
}
// WebAuth verifies the console session and pins the request to its tenant.
func WebAuth(pos services.PosService) fiber.Handler { return webAuthWith(pos) }
func webAuthWith(locations webLocationChecker) fiber.Handler {
return func(c *fiber.Ctx) error {
if isPublicWebPath(c.Path()) {
return c.Next()
}
token := webBearerToken(c)
if token == "" {
if webAuthRequired() {
return webUnauthorized(c, "a session token is required; sign in again")
}
// A console that predates tokens. Allowed through unpinned, which is
// exactly the state this middleware exists to end — see
// webAuthRequired.
return c.Next()
}
claims, err := utils.ParseWebToken(token, time.Now())
if err != nil {
// Always refused, flag or no flag. A token that does not verify is a
// stronger signal than no token at all: nothing sends a broken one by
// accident.
return webUnauthorized(c, err.Error())
}
// Nearle's own staff work across every tenant and legitimately name any
// of them. Checked once, here, rather than at each test below, so the
// exemption is a single visible branch instead of three.
if !claims.IsPlatformAccount() {
if requested := requestedTenant(c); requested > 0 && requested != claims.Tenantid {
return webForbidden(c, "this session cannot reach tenant "+strconv.Itoa(requested))
}
// A request can also scope by branch alone, naming no tenant at all,
// so pinning the tenant is not enough on its own.
if requested := requestedWebLocation(c); requested > 0 && requested != claims.Locationid {
allowed, err := locations.LocationAllowed(claims.Tenantid, requested)
if err != nil {
return c.Status(http.StatusServiceUnavailable).JSON(fiber.Map{
"code": http.StatusServiceUnavailable, "status": false,
"message": "could not verify branch access",
})
}
if !allowed {
return webForbidden(c, "this session cannot reach branch "+strconv.Itoa(requested))
}
}
}
c.Locals(WebLocalsKey, claims)
return c.Next()
}
}
// webBearerToken reads the session out of the request.
//
// `Authorization: Bearer …` only. The POS reader next door also accepts
// `X-Pos-Token`, because shop routers between a till and this server strip
// Authorization headers on plain HTTP and a terminal that cannot authenticate
// is a shop that cannot trade. The console has no such problem — it is a
// browser on HTTPS — so it gets the one form, and a second accepted header is
// a second thing to get wrong.
func webBearerToken(c *fiber.Ctx) string {
header := strings.TrimSpace(c.Get("Authorization"))
if header == "" {
return ""
}
if after, found := strings.CutPrefix(header, "Bearer "); found {
return strings.TrimSpace(after)
}
if !strings.Contains(header, " ") {
return header
}
return ""
}
// requestedTenant reads the tenant a request is naming, from wherever it put it.
//
// Query first, because that is where every `/web` list endpoint carries it, then
// the body, because the writes do not: `createdeliveries`, `publishproduct` and
// the rest post JSON. Checking only the query would leave every call that
// CHANGES another tenant's data unguarded, which is the wrong half to skip.
func requestedTenant(c *fiber.Ctx) int {
for _, key := range []string{"tenantid", "tenant_id"} {
if raw := strings.TrimSpace(c.Query(key)); raw != "" {
if id, err := strconv.Atoi(raw); err == nil && id > 0 {
return id
}
}
}
return bodyScopeID(c, "tenantid", "tenant_id")
}
// requestedWebLocation reads the branch a request is naming.
//
// Separate from the POS reader's `requestedLocation` because the two surfaces
// spell it differently: POS routes use `store_id`, the console uses
// `locationid`. Both spellings are read here anyway — a shared endpoint is
// cheaper to allow for than to discover.
func requestedWebLocation(c *fiber.Ctx) int {
for _, key := range []string{"locationid", "location_id", "store_id"} {
if raw := strings.TrimSpace(c.Query(key)); raw != "" {
if id, err := strconv.Atoi(raw); err == nil && id > 0 {
return id
}
}
}
return bodyScopeID(c, "locationid", "location_id", "store_id")
}
// bodyScopeID pulls a scoping id out of a JSON request body.
//
// Decoded loosely rather than into a request type, on purpose: this runs before
// the handler and must not refuse anything the handler would have accepted. A
// body that will not parse here is left for the handler to reject with its own
// message, and a request shape that changes later must not silently stop being
// authorised.
//
// `c.Body()` returns buffered bytes, so reading here does not consume the
// stream the handler goes on to parse.
//
// An ARRAY body — `createdeliveries` posts one — is walked too. A batch naming
// another tenant in its elements is precisely the call worth guarding, and a
// probe that only understood objects would wave it through.
func bodyScopeID(c *fiber.Ctx, keys ...string) int {
body := c.Body()
if len(body) == 0 || len(body) > 8<<20 {
return 0
}
var raw json.RawMessage = body
trimmed := strings.TrimLeft(string(body), " \t\r\n")
if strings.HasPrefix(trimmed, "[") {
var elements []json.RawMessage
if err := json.Unmarshal(body, &elements); err != nil {
return 0
}
for _, element := range elements {
if id := scopeIDFromObject(element, keys); id > 0 {
return id
}
}
return 0
}
return scopeIDFromObject(raw, keys)
}
func scopeIDFromObject(raw json.RawMessage, keys []string) int {
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
return 0
}
for _, key := range keys {
if id := asScopeID(fields[key]); id > 0 {
return id
}
}
return 0
}
// asScopeID reads an id that may have been sent as a number or as a string.
//
// Both spellings are on the wire today — the console sends numbers, some app
// callers send strings — and a probe that understood only one would return 0
// for the other, which reads as "named no tenant" and waves the request past
// the check.
func asScopeID(raw json.RawMessage) int {
if len(raw) == 0 {
return 0
}
var number int
if err := json.Unmarshal(raw, &number); err == nil {
return number
}
var text string
if err := json.Unmarshal(raw, &text); err == nil {
if id, err := strconv.Atoi(strings.TrimSpace(text)); err == nil {
return id
}
}
return 0
}
func webUnauthorized(c *fiber.Ctx, message string) error {
return c.Status(http.StatusUnauthorized).JSON(fiber.Map{
"code": http.StatusUnauthorized, "status": false, "message": message,
})
}
func webForbidden(c *fiber.Ctx, message string) error {
return c.Status(http.StatusForbidden).JSON(fiber.Map{
"code": http.StatusForbidden, "status": false, "message": message,
})
}
// WebClaimsFrom returns the verified session on a request, if it carried one.
//
// The second return distinguishes "no token" from "a token claiming tenant 0",
// which is a platform account and a real answer. A handler that treated the two
// alike would give an unauthenticated caller the one session that reads
// everything.
func WebClaimsFrom(c *fiber.Ctx) (utils.WebClaims, bool) {
claims, ok := c.Locals(WebLocalsKey).(utils.WebClaims)
return claims, ok
}

273
middleware/webauth_test.go Normal file
View File

@@ -0,0 +1,273 @@
package middleware
import (
"net/http/httptest"
"strings"
"testing"
"time"
"nearle/utils"
"github.com/gofiber/fiber/v2"
)
const webTestSecret = "a-test-signing-key-long-enough"
// fakeLocations answers the tenant-owns-branch question without a database.
//
// `owned` is the branch the tenant genuinely has; anything else is refused, and
// `fails` makes the lookup itself error so the unavailable path can be reached.
type fakeLocations struct {
tenant int
owned int
fails bool
}
func (f fakeLocations) LocationAllowed(tenantID, locationID int) (bool, error) {
if f.fails {
return false, errFakeLookup
}
return tenantID == f.tenant && locationID == f.owned, nil
}
type fakeErr struct{}
func (fakeErr) Error() string { return "lookup unavailable" }
var errFakeLookup = fakeErr{}
// call runs one request through the middleware and reports the status.
//
// The handler behind it always succeeds, so any non-200 came from the guard.
func call(t *testing.T, locations webLocationChecker, token, method, target, body string) int {
t.Helper()
app := fiber.New()
app.Use("/live/api/v1/web", webAuthWith(locations))
app.All("/live/api/v1/web/*", func(c *fiber.Ctx) error { return c.SendStatus(fiber.StatusOK) })
req := httptest.NewRequest(method, target, strings.NewReader(body))
if body != "" {
req.Header.Set("Content-Type", "application/json")
}
if token != "" {
req.Header.Set("Authorization", "Bearer "+token)
}
resp, err := app.Test(req)
if err != nil {
t.Fatalf("calling: %v", err)
}
return resp.StatusCode
}
func tokenFor(t *testing.T, claims utils.WebClaims) string {
t.Helper()
token, _, err := utils.MintWebToken(claims, time.Now())
if err != nil {
t.Fatalf("minting: %v", err)
}
return token
}
/* ── The hole this exists to close ─────────────────────────────────────── */
func TestASessionCannotNameAnotherTenant(t *testing.T) {
// One number in a URL. Before this middleware it read another merchant's
// orders, stock, staff and takings.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 904, Tenantid: 1147})
own := call(t, fakeLocations{}, session, "GET", "/live/api/v1/web/orders/tenant/getorders?tenantid=1147", "")
if own != fiber.StatusOK {
t.Fatalf("a session was refused its own tenant: %d", own)
}
other := call(t, fakeLocations{}, session, "GET", "/live/api/v1/web/orders/tenant/getorders?tenantid=916", "")
if other != fiber.StatusForbidden {
t.Fatalf("tenant 916 was readable with a tenant 1147 session: %d", other)
}
}
func TestAWriteCannotNameAnotherTenantInItsBody(t *testing.T) {
// The half that would be easy to skip. Reads carry `tenantid` in the query;
// the calls that CHANGE things post JSON, so a query-only check leaves every
// write unguarded.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 904, Tenantid: 1147})
body := `{"tenantid":916,"productname":"Milk Bikis"}`
got := call(t, fakeLocations{}, session, "POST", "/live/api/v1/web/products/create", body)
if got != fiber.StatusForbidden {
t.Fatalf("a write into tenant 916 was allowed: %d", got)
}
}
func TestABatchCannotSmuggleAnotherTenantInAnArray(t *testing.T) {
// `createdeliveries` posts an array. A probe that only understood objects
// would wave through exactly the call that creates work in another
// merchant's shop.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 904, Tenantid: 1147})
body := `[{"orderheaderid":1,"tenantid":916}]`
got := call(t, fakeLocations{}, session, "POST", "/live/api/v1/web/deliveries/createdeliveries", body)
if got != fiber.StatusForbidden {
t.Fatalf("a batch naming tenant 916 was allowed: %d", got)
}
}
func TestATenantSentAsAStringIsStillChecked(t *testing.T) {
// Both spellings are on the wire. A probe that understood only numbers
// returns 0 for `"916"`, which reads as "named no tenant" and passes.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 904, Tenantid: 1147})
got := call(t, fakeLocations{}, session, "POST", "/live/api/v1/web/products/create", `{"tenantid":"916"}`)
if got != fiber.StatusForbidden {
t.Fatalf("a string tenant id slipped past: %d", got)
}
}
/* ── Scoping by branch alone ───────────────────────────────────────────── */
func TestABranchMustBelongToTheSessionsTenant(t *testing.T) {
// A request can scope by branch and name no tenant at all, so pinning the
// tenant is not sufficient on its own.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 904, Tenantid: 1147, Locationid: 1172})
locations := fakeLocations{tenant: 1147, owned: 1173}
mine := call(t, locations, session, "GET", "/live/api/v1/web/products/get?locationid=1173", "")
if mine != fiber.StatusOK {
t.Fatalf("a second branch of my own tenant was refused: %d", mine)
}
theirs := call(t, locations, session, "GET", "/live/api/v1/web/products/get?locationid=1185", "")
if theirs != fiber.StatusForbidden {
t.Fatalf("another tenant's branch was readable: %d", theirs)
}
}
func TestTheSessionsOwnBranchNeedsNoLookup(t *testing.T) {
// `fails: true` errors on any lookup, so reaching OK proves the home branch
// short-circuits before asking.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 904, Tenantid: 1147, Locationid: 1172})
got := call(t, fakeLocations{fails: true}, session, "GET", "/live/api/v1/web/products/get?locationid=1172", "")
if got != fiber.StatusOK {
t.Fatalf("the session's own branch was refused: %d", got)
}
}
func TestAFailedBranchLookupIsNotAPass(t *testing.T) {
// If the check cannot run, the answer is "cannot verify", never "allowed".
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 904, Tenantid: 1147, Locationid: 1172})
got := call(t, fakeLocations{fails: true}, session, "GET", "/live/api/v1/web/products/get?locationid=1185", "")
if got != fiber.StatusServiceUnavailable {
t.Fatalf("a broken lookup did not refuse: %d", got)
}
}
/* ── Tokens ────────────────────────────────────────────────────────────── */
func TestABrokenTokenIsAlwaysRefused(t *testing.T) {
// Refused whatever the flag says. Nothing sends a broken token by accident.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
t.Setenv("WEB_AUTH_REQUIRED", "false")
got := call(t, fakeLocations{}, "w1.rubbish.signature", "GET", "/live/api/v1/web/orders/tenant/getorders?tenantid=1147", "")
if got != fiber.StatusUnauthorized {
t.Fatalf("a forged token was not refused: %d", got)
}
}
func TestATillsTokenIsNotAConsoleSessionHere(t *testing.T) {
// A POS token is the same shape signed with the same key. If it verified
// here its `Locationid` would land where `Tenantid` is read.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
pos, _, err := utils.MintPosToken(utils.PosClaims{Userid: 7, Tenantid: 916, Locationid: 1185, Roleid: 8}, time.Now())
if err != nil {
t.Fatalf("minting a POS token: %v", err)
}
got := call(t, fakeLocations{}, pos, "GET", "/live/api/v1/web/orders/tenant/getorders?tenantid=1147", "")
if got != fiber.StatusUnauthorized {
t.Fatalf("a cashier's token was accepted on the console: %d", got)
}
}
/* ── The staged rollout ────────────────────────────────────────────────── */
func TestWithoutTheFlagAnUntokenedRequestStillWorks(t *testing.T) {
// The console in production sends no token yet. Locking it out before
// sign-in issues one would break a working product.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
t.Setenv("WEB_AUTH_REQUIRED", "false")
got := call(t, fakeLocations{}, "", "GET", "/live/api/v1/web/orders/tenant/getorders?tenantid=916", "")
if got != fiber.StatusOK {
t.Fatalf("an untokened request was refused while enforcement is off: %d", got)
}
}
func TestWithTheFlagAnUntokenedRequestIsRefused(t *testing.T) {
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
t.Setenv("WEB_AUTH_REQUIRED", "true")
got := call(t, fakeLocations{}, "", "GET", "/live/api/v1/web/orders/tenant/getorders?tenantid=916", "")
if got != fiber.StatusUnauthorized {
t.Fatalf("enforcement is on and an untokened request passed: %d", got)
}
}
func TestSignInStillWorksWithEnforcementOn(t *testing.T) {
// Guarding the login route with a session token means nobody can ever get
// one. This is the test that catches a locked-out deployment.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
t.Setenv("WEB_AUTH_REQUIRED", "true")
for _, path := range []string{
"/live/api/v1/web/users/applogin",
"/live/api/v1/web/tenant/weblogin",
} {
if got := call(t, fakeLocations{}, "", "POST", path, `{"authname":"a@b.c"}`); got != fiber.StatusOK {
t.Fatalf("%s was locked behind a session: %d", path, got)
}
}
}
/* ── The platform account ──────────────────────────────────────────────── */
func TestPlatformStaffMayNameAnyTenant(t *testing.T) {
// Nearle's own staff work across tenants and the console's /nearle pages
// depend on it.
t.Setenv("POS_TOKEN_SECRET", webTestSecret)
session := tokenFor(t, utils.WebClaims{Userid: 12, Superadmin: true, Roleid: 1})
got := call(t, fakeLocations{}, session, "GET", "/live/api/v1/web/orders/tenant/getorders?tenantid=916", "")
if got != fiber.StatusOK {
t.Fatalf("a platform session was refused tenant 916: %d", got)
}
}
func TestNoTokenIsNotAPlatformAccount(t *testing.T) {
// Tenant 0 is the session that reads everything, and Go's zero value is 0.
// A handler reading claims off a request that carried none would hand an
// anonymous caller exactly that session.
app := fiber.New()
var found bool
app.Get("/probe", func(c *fiber.Ctx) error {
_, found = WebClaimsFrom(c)
return c.SendStatus(fiber.StatusOK)
})
if _, err := app.Test(httptest.NewRequest("GET", "/probe", nil)); err != nil {
t.Fatalf("probing: %v", err)
}
if found {
t.Fatal("claims were reported present on a request that carried none")
}
}

View File

@@ -11,8 +11,16 @@ package models
type ScanLookupRequest struct {
Customerid int `json:"customerid"`
// What Lens read: "Milk Bikis", "Dabur Honey 500g". Free text, trimmed
// and capped by the service.
// and capped by the service. Not required when Brand and Catalogueid
// name a product outright.
Label string `json:"label"`
// A product the customer has already chosen, by its catalogue key —
// which is how the app resolves a `candidates` list from an earlier
// ambiguous lookup, and how a deep link or a re-order skips recognition
// altogether. When both are set the label is ignored and no catalogue
// search runs.
Brand string `json:"brand"`
Catalogueid int64 `json:"catalogueid"`
// Where the customer is right now. Optional: without it the customer's
// saved primary address is used, and without that stores are listed in
// registration order with no distance.
@@ -86,9 +94,15 @@ type ScanCatalogueMatch struct {
VariantKey string `json:"variant_key,omitempty"`
Image string `json:"image,omitempty"`
Score float64 `json:"score"`
// "vector", "vector+text" or "text" — how the score was produced. The app
// can be more cautious with a text-only match.
// "vector+text", "text" or "direct" — how the score was produced. The app
// can be more cautious with a text-only match; "direct" means the caller
// named the product by its catalogue key and nothing was recognised.
Method string `json:"method"`
// Set only on entries of `candidates`: at least one of the customer's
// registered stores has this product in stock right now. Candidates are
// ordered with the available ones first, so a "did you mean?" list can
// show what is actually buyable before what is not.
Available bool `json:"available,omitempty"`
}
// ScanLookupResponse is the answer to a scan.
@@ -96,10 +110,27 @@ type ScanLookupResponse struct {
Label string `json:"label"`
// The best catalogue product for the label, and the sizes of it the
// catalogue knows about (each a separate catalogue row).
//
// Match is nil when nothing was recognised, and also when several
// products matched equally well — see Ambiguous.
Match *ScanCatalogueMatch `json:"match"`
Variants []ScanCatalogueMatch `json:"catalogue_variants"`
// Several products fit the label and no one of them is a clear winner —
// which is what a bare brand name ("britannia") or a generic word
// ("biscuits") produces, and Lens returns those often because a
// wordmark is the most legible thing on a packet.
//
// When true: Match is nil, Stores is empty, and Candidates holds the
// products to offer as "did you mean?". Picking one means calling
// /lookup again with that candidate's `brand` and `catalogueid`.
//
// Guessing instead would mean showing a confident price for a product
// the customer did not photograph.
Ambiguous bool `json:"ambiguous"`
Candidates []ScanCatalogueMatch `json:"candidates"`
// 0..1. Below ~0.5 the app should confirm with the customer before
// showing prices.
// showing prices. With Ambiguous set this is the leader's score, which
// by definition the runner-up nearly equals.
Confidence float64 `json:"confidence"`
// Registered stores that stock the product, nearest first, in-stock
// first. Empty with Available=false when none does.

View File

@@ -94,6 +94,9 @@ type ScanRepository interface {
VectorSearch(ctx context.Context, vector []float32, limit int) ([]CatalogueHit, error)
TextSearch(ctx context.Context, label string, limit int) ([]CatalogueHit, error)
VectorSearchAvailable() bool
// CatalogueRef is one product named by its catalogue key, with its other
// pack sizes after it. Nothing is recognised or scored.
CatalogueRef(ctx context.Context, brand string, id int64) ([]CatalogueHit, error)
// cache
CachedVector(ctx context.Context, model, label string) ([]float32, bool)
@@ -546,6 +549,72 @@ func (r *scanRepository) TextSearch(ctx context.Context, label string, limit int
return hits, nil
}
// tableFor resolves a brand the caller named to a real catalogue table.
//
// The lookup is against the tables discovered from information_schema, never
// a string built from the request: table names cannot be parameterised in
// SQL, so the discovered map is what keeps this from being an injection
// point. Both the table suffix ("britannia") and a display name ("24 Mantra"
// → brand_24_mantra) resolve.
func (r *scanRepository) tableFor(ctx context.Context, brand string) (string, map[string]bool, error) {
tables, err := r.brandTables(ctx)
if err != nil {
return "", nil, err
}
for _, candidate := range []string{
"brand_" + strings.ToLower(strings.TrimSpace(brand)),
"brand_" + normaliseBrandKey(brand),
} {
if cols, ok := tables[candidate]; ok {
return candidate, cols, nil
}
}
return "", nil, ErrUnknownBrand
}
// CatalogueRef reads one product by (brand, id) and appends its other pack
// sizes — same variant_key where the catalogue assigned one, same name
// otherwise, matching how the search groups a family.
//
// Distance is 0 on every row: nothing here was ranked, the caller said which
// product they meant.
func (r *scanRepository) CatalogueRef(ctx context.Context, brand string, id int64) ([]CatalogueHit, error) {
table, cols, err := r.tableFor(ctx, brand)
if err != nil {
return nil, err
}
suffix := strings.TrimPrefix(table, "brand_")
columns := hitColumns(suffix, cols)
var self []CatalogueHit
err = r.catalogue.WithContext(ctx).Raw(fmt.Sprintf(
`SELECT %s, 0::float8 AS distance FROM %s WHERE id = ?`, columns, table), id).Scan(&self).Error
if err != nil {
return nil, err
}
if len(self) == 0 {
return nil, nil
}
var siblings []CatalogueHit
if cols["variant_key"] && strings.TrimSpace(self[0].VariantKey) != "" {
err = r.catalogue.WithContext(ctx).Raw(fmt.Sprintf(
`SELECT %s, 0::float8 AS distance FROM %s WHERE variant_key = ? AND id <> ? ORDER BY id`,
columns, table), self[0].VariantKey, id).Scan(&siblings).Error
} else {
err = r.catalogue.WithContext(ctx).Raw(fmt.Sprintf(
`SELECT %s, 0::float8 AS distance FROM %s WHERE LOWER(product_name) = LOWER(?) AND id <> ? ORDER BY id`,
columns, table), self[0].ProductName, id).Scan(&siblings).Error
}
if err != nil {
// The product itself was found; losing its other sizes is the smaller
// failure and the caller asked for this one.
log.Printf("scan: could not read pack sizes of %s#%d: %v", brand, id, err)
return self, nil
}
return append(self, siblings...), nil
}
func sortedKeys(m map[string]map[string]bool) []string {
keys := make([]string, 0, len(m))
for k := range m {

View File

@@ -2,6 +2,7 @@ package routes
import (
"nearle/facade"
"nearle/middleware"
"github.com/gofiber/fiber/v2"
)
@@ -10,6 +11,22 @@ func RegisterRoutes(app *fiber.App, f *facade.Facade) {
api := app.Group("/live/api")
// Console sessions.
//
// Mounted by PATH rather than on a group object, because the `/v1/web`
// routes are not one group — a dozen files each create their own
// (`/v1/web/users`, `/v1/web/orders`, `/v1/web/products`, …). Registered
// here, ahead of all of them, so a route added later is guarded by default
// rather than by somebody remembering to.
//
// `/v1/pos` is deliberately NOT covered: that is the terminal surface, it
// carries a different kind of token, and it has its own guard. But
// `/v1/web/pos` and `/v1/web/tenants` ARE, despite their names — both are
// console callers, and `createposuser` on the second mints till credentials,
// which until now it did on the strength of an unauthenticated request. The
// note above registerPosStaffConsoleRoutes asked for exactly this.
api.Use("/v1/web", middleware.WebAuth(f.PosService()))
RegisterUserRoutes(api, f)
RegisterProductRoutes(api, f)
RegisterOrderRoutes(api, f)

View File

@@ -53,28 +53,29 @@ func main() {
var cols []struct {
Relname string
Attname string
Typname string
Atttypmod int
}
if err := db.Raw(`
SELECT c.relname, t.typname, a.atttypmod
SELECT c.relname, a.attname, t.typname, a.atttypmod
FROM pg_attribute a
JOIN pg_class c ON c.oid = a.attrelid
JOIN pg_type t ON t.oid = a.atttypid
WHERE a.attname = 'embedding' AND c.relname LIKE 'brand\_%'
ORDER BY c.relname`).Scan(&cols).Error; err != nil {
WHERE t.typname = 'vector' AND a.attnum > 0 AND c.relname LIKE 'brand\_%'
ORDER BY c.relname, a.attname`).Scan(&cols).Error; err != nil {
log.Fatal(err)
}
if len(cols) == 0 {
fmt.Println("no brand_* table has an embedding column")
fmt.Println("no brand_* table has a vector column")
return
}
fmt.Printf("%-28s %-8s %5s %5s %5s\n", "table", "type", "dims", "rows", "embd")
fmt.Printf("%-24s %-16s %-8s %5s %5s %5s\n", "table", "column", "type", "dims", "rows", "filled")
for _, c := range cols {
var total, filled int64
db.Raw(fmt.Sprintf(`SELECT COUNT(1) FROM %s`, c.Relname)).Scan(&total)
db.Raw(fmt.Sprintf(`SELECT COUNT(1) FROM %s WHERE embedding IS NOT NULL`, c.Relname)).Scan(&filled)
fmt.Printf("%-28s %-8s %5d %5d %5d\n", c.Relname, c.Typname, c.Atttypmod, total, filled)
db.Raw(fmt.Sprintf(`SELECT COUNT(1) FROM %s WHERE %s IS NOT NULL`, c.Relname, c.Attname)).Scan(&filled)
fmt.Printf("%-24s %-16s %-8s %5d %5d %5d\n", c.Relname, c.Attname, c.Typname, c.Atttypmod, total, filled)
}
// nomic/bge emit unit vectors; a norm far from 1 means another pipeline.

View File

@@ -10,6 +10,7 @@ import (
"nearle/repositories"
"nearle/utils"
"sort"
"strconv"
"strings"
"sync"
"time"
@@ -54,6 +55,12 @@ const (
// came back as "Paneer Makhni 500ml" (0.304) — a near-miss on an
// unrelated row clears a floor set that close to the noise.
scanMinScore = 0.50
// How close the runner-up may be before the leader stops being an answer
// and the two become a question. See isAmbiguous.
scanAmbiguityMargin = 0.06
// A "did you mean?" list longer than this is not a choice, it is a
// catalogue — the customer is standing in a shop holding a packet.
scanMaxCandidates = 10
)
// ScanErrors the controller maps to statuses. Everything else is a 500.
@@ -82,11 +89,15 @@ func NewScanService(repo repositories.ScanRepository, embedder utils.Embedder) S
func (s *scanService) Lookup(ctx context.Context, req models.ScanLookupRequest) (*models.ScanLookupResponse, error) {
label := strings.TrimSpace(req.Label)
// The caller can name the product outright instead of describing it —
// how the app resolves a candidate the customer picked.
direct := strings.TrimSpace(req.Brand) != "" && req.Catalogueid > 0
if req.Customerid <= 0 {
return nil, fmt.Errorf("%w: customerid is required", ErrScanBadRequest)
}
if label == "" {
return nil, fmt.Errorf("%w: label is required", ErrScanBadRequest)
if label == "" && !direct {
return nil, fmt.Errorf("%w: label, or brand and catalogueid, is required", ErrScanBadRequest)
}
if len(label) > scanMaxLabelLen {
label = label[:scanMaxLabelLen]
@@ -126,6 +137,10 @@ func (s *scanService) Lookup(ctx context.Context, req models.ScanLookupRequest)
}()
go func() {
defer wg.Done()
if direct {
hits, method, matchErr = s.resolveRef(ctx, req.Brand, req.Catalogueid)
return
}
hits, method, matchErr = s.searchCatalogue(ctx, label)
}()
wg.Wait()
@@ -149,18 +164,32 @@ func (s *scanService) Lookup(ctx context.Context, req models.ScanLookupRequest)
Label: label,
Stores: []models.ScanStoreOffer{},
Variants: []models.ScanCatalogueMatch{},
Candidates: []models.ScanCatalogueMatch{},
}
// Verify the app's idea of the customer's tenants against the truth.
stores, resp.UnregisteredTenantids = restrictToTenants(stores, req.Tenantids)
if len(hits) == 0 || hits[0].score < scanMinScore {
switch {
case len(hits) == 0 && direct:
resp.Message = "That product is no longer in the catalogue."
return resp, nil
case len(hits) == 0, hits[0].score < scanMinScore:
resp.Message = "We couldn't recognise that product. Try a clearer photo of the front of the pack."
return resp, nil
}
best := hits[0]
family := catalogueFamily(hits)
distinct := distinctProducts(hits)
// Several products fit and none of them clearly wins — a bare brand name
// or a generic word. Ask rather than guess: naming one of them would put
// a confident price on a product the customer did not photograph.
if !direct && isAmbiguous(distinct) {
return s.candidatesResponse(ctx, resp, hits, distinct, method, stores)
}
best := distinct[0]
family := catalogueFamily(hits, best)
resp.Match = ptr(best.toMatch(method))
resp.Confidence = round3(best.score)
for _, h := range family {
@@ -179,11 +208,7 @@ func (s *scanService) Lookup(ctx context.Context, req models.ScanLookupRequest)
keys = append(keys, repositories.CatalogueKey{Brand: h.Brand, Catalogueid: h.ID, Imageid: h.ImageID})
names = append(names, h.ProductName)
}
locationids := make([]int, 0, len(stores))
for _, st := range stores {
locationids = append(locationids, st.Locationid)
}
rows, err := s.repo.StoreOptions(ctx, locationids, keys, names)
rows, err := s.repo.StoreOptions(ctx, locationIDs(stores), keys, names)
if err != nil {
return nil, err
}
@@ -213,6 +238,137 @@ func (s *scanService) Lookup(ctx context.Context, req models.ScanLookupRequest)
return resp, nil
}
// candidatesResponse answers an ambiguous label with the products to choose
// between, marking which of them the customer can actually buy right now.
//
// The availability read is the same StoreOptions query the confident path
// runs, widened to every candidate — so a "did you mean?" list can put the
// three that are in stock above the five that are not, instead of sending
// somebody to a shelf that has none of them.
func (s *scanService) candidatesResponse(ctx context.Context, resp *models.ScanLookupResponse,
hits, distinct []scoredHit, method string, stores []models.ScanStore) (*models.ScanLookupResponse, error) {
candidates := distinct
if len(candidates) > scanMaxCandidates {
candidates = candidates[:scanMaxCandidates]
}
resp.Ambiguous = true
resp.Confidence = round3(distinct[0].score)
// Every catalogue row belonging to a candidate, and a way back from what
// a tenant's product row carries to the candidate it stands for.
inCandidates := make(map[string]bool, len(candidates))
for _, c := range candidates {
inCandidates[c.productKey()] = true
}
var keys []repositories.CatalogueKey
var names []string
byImage := make(map[string]string)
byRef := make(map[string]string)
byName := make(map[string]string)
for _, h := range hits {
key := h.productKey()
if !inCandidates[key] {
continue
}
keys = append(keys, repositories.CatalogueKey{Brand: h.Brand, Catalogueid: h.ID, Imageid: h.ImageID})
names = append(names, h.ProductName)
if h.ImageID != "" {
byImage[h.ImageID] = key
}
byRef[refKey(h.Brand, h.ID)] = key
if n := strings.ToLower(strings.TrimSpace(h.ProductName)); n != "" {
byName[n] = key
}
}
stocked := make(map[string]bool)
if len(stores) > 0 && len(keys) > 0 {
rows, err := s.repo.StoreOptions(ctx, locationIDs(stores), keys, names)
if err != nil {
return nil, err
}
for _, row := range rows {
if row.Stock <= 0 {
continue
}
// Same precedence as optionFromRow: the stable key first.
if row.Imageid != "" {
if key, ok := byImage[row.Imageid]; ok {
stocked[key] = true
continue
}
}
if row.Catalogueid > 0 {
if key, ok := byRef[refKey(row.Productbrand, row.Catalogueid)]; ok {
stocked[key] = true
continue
}
}
if key, ok := byName[strings.ToLower(strings.TrimSpace(row.Productname))]; ok {
stocked[key] = true
}
}
}
for _, c := range candidates {
m := c.toMatch(method)
m.Available = stocked[c.productKey()]
resp.Candidates = append(resp.Candidates, m)
}
// Buyable first; within each group the search's own ranking stands.
sort.SliceStable(resp.Candidates, func(i, j int) bool {
return resp.Candidates[i].Available && !resp.Candidates[j].Available
})
available := 0
for _, c := range resp.Candidates {
if c.Available {
available++
}
}
if available > 0 {
resp.Message = fmt.Sprintf("Which one is it? %d of these %d are in stock near you.",
available, len(resp.Candidates))
} else {
resp.Message = fmt.Sprintf("Which one is it? We found %d products that could match.",
len(resp.Candidates))
}
return resp, nil
}
// resolveRef reads the product the caller named, with its pack sizes. No
// recognition, so every row scores 1 and the method says so.
func (s *scanService) resolveRef(ctx context.Context, brand string, id int64) ([]scoredHit, string, error) {
rows, err := s.repo.CatalogueRef(ctx, brand, id)
if err != nil {
switch {
case errors.Is(err, repositories.ErrCatalogueDBUnavailable):
return nil, "direct", ErrScanCatalogueDown
case errors.Is(err, repositories.ErrUnknownBrand):
return nil, "direct", fmt.Errorf("%w: unknown brand %q", ErrScanBadRequest, brand)
}
return nil, "direct", err
}
hits := make([]scoredHit, 0, len(rows))
for _, row := range rows {
hits = append(hits, scoredHit{CatalogueHit: row, score: 1})
}
return hits, "direct", nil
}
func refKey(brand string, id int64) string {
return strings.ToLower(strings.TrimSpace(brand)) + "#" + strconv.FormatInt(id, 10)
}
func locationIDs(stores []models.ScanStore) []int {
ids := make([]int, 0, len(stores))
for _, st := range stores {
ids = append(ids, st.Locationid)
}
return ids
}
// ── Confirm ─────────────────────────────────────────────────────────────────
func (s *scanService) Confirm(ctx context.Context, req models.ScanConfirmRequest) (*models.ScanConfirmResponse, error) {
@@ -530,6 +686,11 @@ func (s *scanService) embed(ctx context.Context, label string) ([]float32, error
// The whole label as a substring of the name is near-certain; otherwise the
// share of label words found in name+title, scaled so that "all of them"
// stops short of the substring case.
//
// A label that is a substring of MANY names — a bare brand, "britannia" —
// therefore scores them all 0.95, identically. That tie is not a flaw to
// score around: it is the signal, and isAmbiguous reads it to answer "did
// you mean?" rather than letting the sort order pick a winner.
func textScore(h repositories.CatalogueHit, label string, tokens []string) float64 {
name := strings.ToLower(h.ProductName)
hay := name + " " + strings.ToLower(h.Title)
@@ -561,27 +722,64 @@ func textScore(h repositories.CatalogueHit, label string, tokens []string) float
return 0.8 * float64(found) / float64(len(tokens))
}
// catalogueFamily is the best hit and its other pack sizes: same brand, and
// the same variant_key when the catalogue assigned one, else the same name.
// Every member is a separate catalogue row a shop may have imported.
func catalogueFamily(hits []scoredHit) []scoredHit {
if len(hits) == 0 {
return nil
// productKey identifies a product across its pack sizes: the catalogue's own
// variant_key where it assigned one, the name otherwise, always within a
// brand. Two rows sharing it are 100 g and 200 g of one thing; two rows that
// do not are different products to choose between.
func productKey(h repositories.CatalogueHit) string {
if k := strings.TrimSpace(h.VariantKey); k != "" {
return h.Brand + "/" + strings.ToLower(k)
}
best := hits[0]
family := []scoredHit{best}
for _, h := range hits[1:] {
if h.Brand != best.Brand {
return h.Brand + "/" + strings.ToLower(strings.TrimSpace(h.ProductName))
}
// productKey of a scored hit — scoredHit embeds the row it scored.
func (h scoredHit) productKey() string { return productKey(h.CatalogueHit) }
// distinctProducts keeps the best-scoring row of each product, in rank
// order — the list of things the customer could actually be shown to choose
// between, as opposed to the same product listed four times in four sizes.
func distinctProducts(hits []scoredHit) []scoredHit {
seen := make(map[string]bool, len(hits))
out := make([]scoredHit, 0, len(hits))
for _, h := range hits {
key := h.productKey()
if seen[key] {
continue
}
switch {
case best.VariantKey != "" && h.VariantKey != "":
if h.VariantKey == best.VariantKey {
seen[key] = true
out = append(out, h)
}
return out
}
// isAmbiguous reports that naming the leader as THE match would be a guess
// dressed up as an answer, because something else is level with it.
//
// A margin rather than an absolute threshold: what matters is not how high
// the best score is but whether anything is tied with it. A bare brand name
// is a substring of every one of that brand's names, so textScore gives them
// all 0.95 — a perfect tie at a HIGH score, which no floor would catch.
//
// Erring towards asking is deliberate. Asking costs the customer one tap on
// a picture; guessing wrong costs them the wrong biscuit and costs us the
// belief that the scanner works. A label that names one product leaves the
// runner-up far behind, so the common case is unaffected.
func isAmbiguous(distinct []scoredHit) bool {
return len(distinct) >= 2 && distinct[1].score >= distinct[0].score-scanAmbiguityMargin
}
// catalogueFamily is `of` and its other pack sizes, drawn from hits.
func catalogueFamily(hits []scoredHit, of scoredHit) []scoredHit {
key := of.productKey()
family := make([]scoredHit, 0, 4)
for _, h := range hits {
if h.productKey() == key {
family = append(family, h)
}
case strings.EqualFold(strings.TrimSpace(h.ProductName), strings.TrimSpace(best.ProductName)):
family = append(family, h)
}
if len(family) == 0 {
return []scoredHit{of}
}
return family
}

View File

@@ -3,10 +3,13 @@ package services
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"nearle/models"
"nearle/repositories"
"nearle/utils"
)
/*
@@ -30,9 +33,13 @@ type fakeScanRepo struct {
options []repositories.StoreOptionRow
at map[int]*repositories.StoreOptionRow // productid → row
ref []repositories.CatalogueHit
refErr error
askedKeys []repositories.CatalogueKey
askedNames []string
askedLocs []int
askedRef string
cachedHits map[string][]repositories.CatalogueHit
}
@@ -69,6 +76,10 @@ func (f *fakeScanRepo) TextSearch(context.Context, string, int) ([]repositories.
return f.text, nil
}
func (f *fakeScanRepo) VectorSearchAvailable() bool { return f.hasVec }
func (f *fakeScanRepo) CatalogueRef(_ context.Context, brand string, id int64) ([]repositories.CatalogueHit, error) {
f.askedRef = fmt.Sprintf("%s#%d", brand, id)
return f.ref, f.refErr
}
func (f *fakeScanRepo) CachedVector(context.Context, string, string) ([]float32, bool) {
return nil, false
}
@@ -423,7 +434,7 @@ func TestCatalogueFamilyGroupsByVariantKeyThenName(t *testing.T) {
{CatalogueHit: milkBikis200, score: 0.88},
{CatalogueHit: repositories.CatalogueHit{Brand: "parle", ProductName: "Milk Bikis", VariantKey: "milk_bikis"}, score: 0.5},
}
family := catalogueFamily(hits)
family := catalogueFamily(hits, hits[0])
if len(family) != 2 || family[1].ID != 8 {
t.Fatalf("family should be the two britannia sizes, got %+v", family)
}
@@ -432,11 +443,229 @@ func TestCatalogueFamilyGroupsByVariantKeyThenName(t *testing.T) {
a := scoredHit{CatalogueHit: repositories.CatalogueHit{Brand: "b", ID: 1, ProductName: "Honey"}}
b := scoredHit{CatalogueHit: repositories.CatalogueHit{Brand: "b", ID: 2, ProductName: "honey "}}
c := scoredHit{CatalogueHit: repositories.CatalogueHit{Brand: "b", ID: 3, ProductName: "Honey Lite"}}
if family := catalogueFamily([]scoredHit{a, b, c}); len(family) != 2 {
if family := catalogueFamily([]scoredHit{a, b, c}, a); len(family) != 2 {
t.Errorf("name match should join 1 and 2 only, got %+v", family)
}
}
/*
Ambiguity.
Lens hands back whatever was most legible on the packet, and on a packet that
is very often the brand wordmark alone. "britannia" fits 258 catalogue rows
equally well, so there is no best one. textScore gives all of them 0.95 —
correctly, the label IS in every one of those names — and with nothing to
read that tie, the sort order picked a winner and the customer was shown one
arbitrary biscuit with "confidence": 0.95 and a price. These tests are the
contract that it asks instead.
*/
// Three different Britannia products, of which the customer's stores stock
// one. Text-only: no embedder, which is also how production runs until the
// model is configured.
func newBrandLabelFixture() *fakeScanRepo {
cashew := repositories.CatalogueHit{Brand: "britannia", ID: 21, ProductName: "Britannia Good Day Cashew Cookies", VariantKey: "good_day_cashew", ImageID: "britannia_good_day_cashew"}
butter := repositories.CatalogueHit{Brand: "britannia", ID: 22, ProductName: "Britannia Good Day Butter Cookies", VariantKey: "good_day_butter", ImageID: "britannia_good_day_butter"}
marie := repositories.CatalogueHit{Brand: "britannia", ID: 23, ProductName: "Britannia Marie Gold", VariantKey: "marie_gold", ImageID: "britannia_marie_gold"}
return &fakeScanRepo{
exists: true,
stores: fixtureStores(),
text: []repositories.CatalogueHit{cashew, butter, marie},
options: []repositories.StoreOptionRow{
{Tenantid: 2, Locationid: 20, Productid: 220, Productname: "Britannia Marie Gold",
Productbrand: "britannia", Catalogueid: 23, Imageid: "britannia_marie_gold", Price: 30, Stock: 4},
},
}
}
func TestABareBrandNameAsksInsteadOfGuessing(t *testing.T) {
repo := newBrandLabelFixture()
svc := NewScanService(repo, nil)
resp, err := svc.Lookup(context.Background(), models.ScanLookupRequest{
Customerid: 5, Label: "britannia", Latitude: "11.035", Longitude: "77.035",
})
if err != nil {
t.Fatal(err)
}
if !resp.Ambiguous {
t.Fatalf("a bare brand name must not resolve to one product, got match %+v", resp.Match)
}
if resp.Match != nil {
t.Errorf("Match must be nil while ambiguous, got %+v", resp.Match)
}
if len(resp.Stores) != 0 {
t.Errorf("no store or price may be quoted for a product the customer has not chosen, got %d offers", len(resp.Stores))
}
if len(resp.Variants) != 0 {
t.Errorf("pack sizes belong to a chosen product, got %+v", resp.Variants)
}
if len(resp.Candidates) != 3 {
t.Fatalf("want the three distinct Britannia products, got %d: %+v", len(resp.Candidates), resp.Candidates)
}
// The one the customer can actually buy is offered first.
if !resp.Candidates[0].Available || resp.Candidates[0].Catalogueid != 23 {
t.Errorf("the stocked product should lead the list, got %+v", resp.Candidates[0])
}
for _, c := range resp.Candidates[1:] {
if c.Available {
t.Errorf("only Marie Gold is stocked, but %s reports available", c.ProductName)
}
}
// Note what confidence does NOT say here. The label appears verbatim in
// all three names, so relevance is high — and the answer is still a
// question. An app that gated on `confidence` instead of `ambiguous`
// would show a price for the wrong biscuit, which is the whole bug.
if resp.Confidence < 0.9 {
t.Errorf("a verbatim brand match scores high; %v suggests the scoring changed", resp.Confidence)
}
if !strings.Contains(resp.Message, "Which one") {
t.Errorf("the message should ask, got %q", resp.Message)
}
}
func TestAnAmbiguousLabelWithNoStockStillLists(t *testing.T) {
repo := newBrandLabelFixture()
repo.options = nil
svc := NewScanService(repo, nil)
resp, err := svc.Lookup(context.Background(), models.ScanLookupRequest{Customerid: 5, Label: "britannia"})
if err != nil {
t.Fatal(err)
}
if !resp.Ambiguous || len(resp.Candidates) != 3 {
t.Fatalf("want three candidates, got ambiguous=%v %d", resp.Ambiguous, len(resp.Candidates))
}
for _, c := range resp.Candidates {
if c.Available {
t.Errorf("%s cannot be available with no stock anywhere", c.ProductName)
}
}
}
// The other half of the contract: a label that does name a product must not
// start asking questions.
func TestASpecificLabelStillWinsOutright(t *testing.T) {
repo := newBrandLabelFixture()
svc := NewScanService(repo, nil)
resp, err := svc.Lookup(context.Background(), models.ScanLookupRequest{
Customerid: 5, Label: "good day cashew",
})
if err != nil {
t.Fatal(err)
}
if resp.Ambiguous {
t.Fatalf("a label naming one product should resolve, got candidates %+v", resp.Candidates)
}
if resp.Match == nil || resp.Match.Catalogueid != 21 {
t.Fatalf("want the cashew cookies, got %+v", resp.Match)
}
if len(resp.Candidates) != 0 {
t.Errorf("candidates belong to an ambiguous answer, got %+v", resp.Candidates)
}
}
// The property the ambiguity check rests on: a label that is a substring of
// several names scores them EQUALLY. Nothing downstream can tell "did you
// mean?" from "found it" if a formula breaks that tie on name length, word
// count or anything else incidental — which is how one arbitrary Britannia
// biscuit used to come back with a price on it.
func TestABrandNameScoresItsProductsIdentically(t *testing.T) {
cashew := repositories.CatalogueHit{ProductName: "Britannia Good Day Cashew Cookies 200g"}
butter := repositories.CatalogueHit{ProductName: "Britannia Good Day Butter Cookies 100g"}
// Deliberately a much shorter name: length must not become a tie-breaker.
marie := repositories.CatalogueHit{ProductName: "Britannia Marie Gold"}
tokens := utils.SearchTokens("britannia")
a, b, c := textScore(cashew, "britannia", tokens), textScore(butter, "britannia", tokens), textScore(marie, "britannia", tokens)
if a != b || b != c {
t.Fatalf("a brand must score its products equally, got %.3f / %.3f / %.3f", a, b, c)
}
if a == 0 {
t.Fatal("the brand name is in every one of those names; scoring it 0 would hide them all")
}
// And a label that does name a product must NOT tie with its siblings,
// or everything would be a question.
specific := utils.SearchTokens("good day cashew")
if textScore(cashew, "good day cashew", specific) <= textScore(butter, "good day cashew", specific) {
t.Error("a label naming one product must outscore its siblings")
}
if none := textScore(cashew, "dabur honey", utils.SearchTokens("dabur honey")); none != 0 {
t.Errorf("nothing in common should score 0, got %.3f", none)
}
}
func TestDistinctProductsCollapsesPackSizes(t *testing.T) {
hits := []scoredHit{
{CatalogueHit: milkBikis, score: 1},
{CatalogueHit: milkBikis200, score: 0.9},
{CatalogueHit: goodDay, score: 0.5},
}
distinct := distinctProducts(hits)
if len(distinct) != 2 || distinct[0].ID != 7 || distinct[1].ID != 9 {
t.Fatalf("two sizes of one product are one choice, got %+v", distinct)
}
}
// Picking a candidate: the app sends the key instead of a description, and
// nothing is recognised at all.
func TestNamingTheProductSkipsRecognition(t *testing.T) {
repo := newLookupFixture()
repo.ref = []repositories.CatalogueHit{milkBikis, milkBikis200}
// If recognition ran, these would decide the answer instead.
repo.text = []repositories.CatalogueHit{goodDay}
repo.vector = []repositories.CatalogueHit{goodDay}
svc := NewScanService(repo, fakeEmbedder{vec: []float32{0.1}})
resp, err := svc.Lookup(context.Background(), models.ScanLookupRequest{
Customerid: 5, Brand: "britannia", Catalogueid: 7,
Latitude: "11.035", Longitude: "77.035",
})
if err != nil {
t.Fatal(err)
}
if repo.askedRef != "britannia#7" {
t.Fatalf("the catalogue should have been asked for that exact product, got %q", repo.askedRef)
}
if resp.Match == nil || resp.Match.Catalogueid != 7 || resp.Match.Method != "direct" {
t.Fatalf("want a direct match on 7, got %+v", resp.Match)
}
if resp.Ambiguous || resp.Confidence != 1 {
t.Errorf("a named product is not a guess: ambiguous=%v confidence=%v", resp.Ambiguous, resp.Confidence)
}
if len(resp.Variants) != 2 {
t.Errorf("its pack sizes should come with it, got %+v", resp.Variants)
}
if !resp.Available || resp.RecommendedLocationid != 20 {
t.Errorf("stores are resolved exactly as for a recognised product, got %+v", resp.Stores)
}
}
func TestAMissingCatalogueRefIsNotAMatch(t *testing.T) {
repo := newLookupFixture()
repo.ref = nil
svc := NewScanService(repo, nil)
resp, err := svc.Lookup(context.Background(), models.ScanLookupRequest{
Customerid: 5, Brand: "britannia", Catalogueid: 999,
})
if err != nil {
t.Fatal(err)
}
if resp.Match != nil || resp.Ambiguous || len(resp.Stores) != 0 {
t.Fatalf("a product that is gone is not a match, got %+v", resp)
}
if !strings.Contains(resp.Message, "no longer") {
t.Errorf("the message should say the product is gone, got %q", resp.Message)
}
}
// A vector neighbour that is merely not-quite-unrelated used to clear the old
// 0.30 floor: in production "Paracetamol" came back as "Paneer Makhni 500ml"
// on a 0.304 similarity. Correct labels land near 0.92, so nothing this weak

162
services/tools/audit.go Normal file
View File

@@ -0,0 +1,162 @@
package tools
import (
"context"
"encoding/json"
"log"
"sort"
"strings"
"time"
)
// The audit trail.
//
// One row per call, including every refusal — the refusals are the interesting
// ones. A registry that recorded only successes would answer "did anything try
// to read another tenant?" with silence, which reads the same as "no".
//
// ── Why the sink is an interface ────────────────────────────────────────────
//
// Phase 1 writes to the log, because a table is a migration and this needs to
// work before that lands. Nothing else in the package knows that: the registry
// holds an `AuditSink`, so the database sink arrives later as a second
// implementation and no call site changes.
//
// ── Writes record intent, not outcome ───────────────────────────────────────
//
// Reads record what happened, which is all a read can be asked for. When write
// tools arrive they must record the ATTEMPT before the call leaves, not the
// result after it returns: a crash mid-write has to leave a trace that it was
// tried, and a row written only on success is a row that is missing exactly
// when it is needed.
const (
OutcomeOK = "ok"
OutcomeRefused = "refused"
OutcomeFailed = "failed"
)
// AuditEntry is one attempt to use a tool.
type AuditEntry struct {
At time.Time
Agent string
Tool string
Scope string
Userid int
Tenantid int
// The arguments as the handler received them — validated and defaulted, not
// as the model sent them. What actually ran is what is worth keeping.
Args map[string]any
// ok | refused | failed. `refused` is the guard saying no; `failed` is the
// handler breaking. Collapsing the two would hide a broken tool inside a
// count of things working as designed.
Outcome string
Detail string
Rows int
Took time.Duration
}
// AuditSink is where entries go.
//
// No error returned, deliberately. An audit sink that can fail a call gives a
// full disk the power to take the assistant down; one that cannot means a lost
// row, which is worse in theory and better in practice. A sink that cares
// should retry or buffer internally.
type AuditSink interface {
Write(ctx context.Context, entry AuditEntry)
}
// DiscardAudit keeps nothing. For tests that are not about the audit trail.
type DiscardAudit struct{}
func (DiscardAudit) Write(context.Context, AuditEntry) {}
// LogAudit writes one line per call to the standard logger.
//
// A line rather than JSON per field, because this is read by a person tailing
// logs during the rollout. The database sink can be structured.
type LogAudit struct{}
func (LogAudit) Write(_ context.Context, entry AuditEntry) {
log.Printf("assistant: %s", entry.Line())
}
// Line renders an entry for a log.
//
// Arguments are rendered sorted so two identical calls produce identical lines
// and a grep for one of them finds both. Go's map iteration is randomised, so
// without the sort the same call logs differently every time.
func (e AuditEntry) Line() string {
var b strings.Builder
b.WriteString(e.Outcome)
b.WriteString(" ")
b.WriteString(e.Agent)
b.WriteString("/")
b.WriteString(e.Tool)
if e.Tenantid > 0 {
b.WriteString(" tenant=")
b.WriteString(itoa(e.Tenantid))
} else {
// Explicitly, rather than by omission: "no tenant" on an assistant call
// is either staff or a bug, and both are worth being able to search for.
b.WriteString(" tenant=none")
}
b.WriteString(" user=")
b.WriteString(itoa(e.Userid))
if len(e.Args) > 0 {
keys := make([]string, 0, len(e.Args))
for key := range e.Args {
keys = append(keys, key)
}
sort.Strings(keys)
parts := make([]string, 0, len(keys))
for _, key := range keys {
value, err := json.Marshal(e.Args[key])
if err != nil {
value = []byte("?")
}
parts = append(parts, key+"="+string(value))
}
b.WriteString(" args{")
b.WriteString(strings.Join(parts, " "))
b.WriteString("}")
}
if e.Outcome == OutcomeOK {
b.WriteString(" rows=")
b.WriteString(itoa(e.Rows))
}
if e.Detail != "" {
b.WriteString(" detail=")
value, err := json.Marshal(e.Detail)
if err != nil {
b.WriteString("?")
} else {
b.Write(value)
}
}
b.WriteString(" took=")
b.WriteString(e.Took.Round(time.Millisecond).String())
return b.String()
}
func itoa(n int) string {
value, _ := json.Marshal(n)
return string(value)
}
// CollectAudit keeps entries in memory, for tests that ARE about the trail.
type CollectAudit struct{ Entries []AuditEntry }
func (c *CollectAudit) Write(_ context.Context, entry AuditEntry) {
c.Entries = append(c.Entries, entry)
}
func (c *CollectAudit) Last() (AuditEntry, bool) {
if len(c.Entries) == 0 {
return AuditEntry{}, false
}
return c.Entries[len(c.Entries)-1], true
}

431
services/tools/registry.go Normal file
View File

@@ -0,0 +1,431 @@
// Package tools is the registry every assistant call goes through.
//
// An agent does not reach the database. It names a tool, and this package
// decides whether that is allowed, whether the arguments make sense, who is
// asking, and what gets recorded — then runs a handler that was written by a
// person and tested.
//
// ── Why a registry rather than generated SQL ────────────────────────────────
//
// The usual reason is safety. Here there is a harder one: the fields on this
// backend do not mean what their names say, and it is measured and documented.
// `orders.deliverystatus` is an empty string on all 181 rows of tenant 1147.
// `orders.orderstatus` only ever carries pending, delivered or cancelled, so
// the six delivery stages in between never reach it. `deliveries.ridername`
// holds delivery statuses as often as names. `orders.deliverytype` is empty on
// every row in production, so filtering on it hides the entire list.
// `billedat` is local wall-clock labelled `Z`, which puts 19 of 20 bills in the
// future.
//
// A model writing SQL gets every one of those wrong, confidently, with no
// error — it reports a cancel rate from a column of empty strings and nobody
// can tell. A model calling a tool cannot, because the correction lives inside
// the handler with the measurement that justified it written beside it.
//
// So: no agent gets raw table access, and a tool that accepts a `where` string
// is a table with extra steps.
package tools
import (
"context"
"errors"
"fmt"
"sort"
"time"
)
// Scope separates reads from writes.
//
// Not decoration. A write tool goes through human approval before it runs
// (that is a later phase), and the registry is where the two are told apart, so
// the distinction has to be declared on the tool rather than inferred from its
// name.
type Scope string
const (
ScopeRead Scope = "read"
ScopeWrite Scope = "write"
)
// Kind is the type of one argument. Deliberately three.
//
// Enough for every tool that exists, and small enough that the validator is
// readable in one sitting. A tool wanting a nested object is a tool doing two
// things.
type Kind string
const (
KindInt Kind = "integer"
KindString Kind = "string"
KindBool Kind = "boolean"
)
// Field is one argument a tool accepts.
//
// `Max == 0` means unbounded, which is why no field here wants a negative
// range — none do, and a pointer per bound to express "unset" would cost every
// call site clarity to buy a case that has not come up.
type Field struct {
Name string
// What the model reads to decide what to put here. Written for the model,
// not for a developer: "minutes a job may sit unaccepted before it counts
// as stuck" beats "threshold".
Description string
Kind Kind
Required bool
Min, Max int
Default any
}
// Schema is a tool's argument contract.
type Schema struct{ Fields []Field }
var (
ErrUnknownTool = errors.New("no such tool")
ErrNotAllowed = errors.New("this agent may not use that tool")
ErrBadArgument = errors.New("argument is not valid")
ErrNoTenant = errors.New("caller names no tenant")
)
// Validate checks arguments and fills in defaults.
//
// Returns a NEW map rather than editing the caller's, and the returned map is
// what the handler sees. Anything not declared is dropped rather than passed
// through — a handler must never receive a key it did not ask for, or an
// argument the model invented becomes an argument the handler might one day
// start reading.
func (s Schema) Validate(args map[string]any) (map[string]any, error) {
clean := make(map[string]any, len(s.Fields))
for _, field := range s.Fields {
raw, sent := args[field.Name]
if !sent || raw == nil {
if field.Required {
return nil, fmt.Errorf("%w: %s is required", ErrBadArgument, field.Name)
}
if field.Default != nil {
clean[field.Name] = field.Default
}
continue
}
value, err := coerce(field, raw)
if err != nil {
return nil, err
}
clean[field.Name] = value
}
return clean, nil
}
// coerce turns what arrived into what the field declared.
//
// JSON numbers arrive as float64 whatever they looked like on the wire, so an
// integer field has to accept one and check it is whole. Reading it as an int
// directly would fail every call made over HTTP, which is all of them.
func coerce(field Field, raw any) (any, error) {
switch field.Kind {
case KindInt:
var n int
switch v := raw.(type) {
case int:
n = v
case int64:
n = int(v)
case float64:
if v != float64(int(v)) {
return nil, fmt.Errorf("%w: %s must be a whole number", ErrBadArgument, field.Name)
}
n = int(v)
default:
return nil, fmt.Errorf("%w: %s must be a number", ErrBadArgument, field.Name)
}
if n < field.Min {
return nil, fmt.Errorf("%w: %s must be at least %d", ErrBadArgument, field.Name, field.Min)
}
if field.Max > 0 && n > field.Max {
return nil, fmt.Errorf("%w: %s must be at most %d", ErrBadArgument, field.Name, field.Max)
}
return n, nil
case KindString:
text, ok := raw.(string)
if !ok {
return nil, fmt.Errorf("%w: %s must be text", ErrBadArgument, field.Name)
}
if field.Max > 0 && len(text) > field.Max {
return nil, fmt.Errorf("%w: %s is longer than %d characters", ErrBadArgument, field.Name, field.Max)
}
return text, nil
case KindBool:
flag, ok := raw.(bool)
if !ok {
return nil, fmt.Errorf("%w: %s must be true or false", ErrBadArgument, field.Name)
}
return flag, nil
}
return nil, fmt.Errorf("%w: %s has no type", ErrBadArgument, field.Name)
}
// JSONSchema renders the contract in the form a model and MCP both expect.
//
// Kept as a projection of `Schema` rather than the source of truth, so the
// validator and the description a model is given cannot drift: there is one
// declaration and this is a view of it.
func (s Schema) JSONSchema() map[string]any {
properties := map[string]any{}
required := []string{}
for _, field := range s.Fields {
property := map[string]any{
"type": string(field.Kind),
"description": field.Description,
}
if field.Kind == KindInt {
property["minimum"] = field.Min
if field.Max > 0 {
property["maximum"] = field.Max
}
}
if field.Default != nil {
property["default"] = field.Default
}
properties[field.Name] = property
if field.Required {
required = append(required, field.Name)
}
}
sort.Strings(required)
schema := map[string]any{
"type": "object",
"properties": properties,
// The model may not invent arguments. A tool that tolerated extras
// would make the schema a suggestion.
"additionalProperties": false,
}
if len(required) > 0 {
schema["required"] = required
}
return schema
}
// Caller is the verified session a tool runs on behalf of.
//
// Built from `middleware.WebAuth`'s claims and never from anything the model
// said. That is the whole arrangement: the model chooses the tool and the
// arguments, and has no say at all in whose data it reads.
type Caller struct {
Userid int
Tenantid int
Locationid int
// Nearle staff, from `app_users.issuperadmin` — the only thing that reads
// across tenants. NOT a role: `app_roles` calls roleid 1 "Super admin" and
// tenant onboarding wrote 1 for every shop owner.
Superadmin bool
}
// Request is what a handler receives.
type Request struct {
Args map[string]any
Caller Caller
}
// Int reads a validated integer argument.
//
// No error return, on purpose: by the time a handler runs, the schema has
// already refused anything that is not an int in range, so a second check here
// would be unreachable code that still has to be read.
func (r Request) Int(name string) int {
if v, ok := r.Args[name].(int); ok {
return v
}
return 0
}
func (r Request) String(name string) string {
if v, ok := r.Args[name].(string); ok {
return v
}
return ""
}
func (r Request) Bool(name string) bool {
if v, ok := r.Args[name].(bool); ok {
return v
}
return false
}
// Result is what a tool answers with.
//
// Rows, never prose. The handler returns structured data and the model does the
// phrasing; a handler that wrote sentences would mean two layers formatting the
// same fact, and they drift.
type Result struct {
// Concrete typed slices, marshalled by the caller. `any` here so handlers
// stay typed rather than every one of them building maps.
Rows any
Count int
// True when there were more rows than were returned, with `Note` saying so
// in words. An empty answer and a capped answer look identical to a model,
// and it will describe both as "none".
Truncated bool
Note string
// The console route showing the same rows, so an answer can carry a link to
// what proves it. Buddy states a conclusion; this is where a person checks.
Source string
// What the answer covers — "all branches", or a named one. A tool that
// omits it lets an answer about one branch be read as an answer about all.
Scope string
}
// Agent is a caller's allow-list.
//
// The registry refuses a tool an agent has not named, rather than the model
// declining to call it. A prompt is a request; this is a rule.
type Agent struct {
Name string
Tools []string
}
func (a Agent) Allows(tool string) bool {
for _, name := range a.Tools {
if name == tool {
return true
}
}
return false
}
// Tool is one thing an agent can do.
type Tool struct {
Name string
// What the model reads to choose between tools. The most load-bearing
// string in the package: a vague one produces a model that calls the wrong
// tool and explains the wrong number confidently.
Description string
Scope Scope
Schema Schema
Handler func(ctx context.Context, req Request) (Result, error)
}
// Registry holds the tools and is the only way to reach one.
type Registry struct {
tools map[string]Tool
audit AuditSink
now func() time.Time
}
func New(audit AuditSink) *Registry {
if audit == nil {
audit = DiscardAudit{}
}
return &Registry{tools: map[string]Tool{}, audit: audit, now: time.Now}
}
// Register adds a tool. Duplicate names are refused rather than overwritten:
// silently replacing a tool is how a permission check disappears.
func (r *Registry) Register(t Tool) error {
if t.Name == "" {
return errors.New("a tool needs a name")
}
if t.Handler == nil {
return fmt.Errorf("tool %q has no handler", t.Name)
}
if t.Description == "" {
return fmt.Errorf("tool %q has no description; a model cannot choose it", t.Name)
}
if _, taken := r.tools[t.Name]; taken {
return fmt.Errorf("tool %q is already registered", t.Name)
}
r.tools[t.Name] = t
return nil
}
// Definitions describes the tools one agent may use, for a model or for MCP.
//
// Built from the agent's allow-list rather than from everything registered, so
// a model is never told about a tool it would then be refused — which reads to
// a model as a malfunction and to a person as the assistant being broken.
func (r *Registry) Definitions(agent Agent) []map[string]any {
names := make([]string, 0, len(agent.Tools))
for _, name := range agent.Tools {
if _, ok := r.tools[name]; ok {
names = append(names, name)
}
}
sort.Strings(names)
out := make([]map[string]any, 0, len(names))
for _, name := range names {
tool := r.tools[name]
out = append(out, map[string]any{
"name": tool.Name,
"description": tool.Description,
"input_schema": tool.Schema.JSONSchema(),
})
}
return out
}
// Call is the one entry point, and it does five things in this order:
// find the tool, check the agent may use it, validate the arguments, confirm
// the caller is scoped to something, and run the handler — recording exactly
// one audit row whatever happens, including every refusal.
//
// The order matters. Arguments are validated before the handler sees them so a
// handler never defends itself, and the caller is checked before the handler
// runs so a tool cannot forget to.
func (r *Registry) Call(ctx context.Context, agent Agent, name string, args map[string]any, caller Caller) (Result, error) {
started := r.now()
entry := AuditEntry{
At: started,
Agent: agent.Name,
Tool: name,
Userid: caller.Userid,
Tenantid: caller.Tenantid,
Args: args,
}
finish := func(result Result, outcome string, detail string, err error) (Result, error) {
entry.Outcome = outcome
entry.Detail = detail
entry.Rows = result.Count
entry.Took = r.now().Sub(started)
r.audit.Write(ctx, entry)
return result, err
}
tool, known := r.tools[name]
if !known {
return finish(Result{}, OutcomeRefused, "unknown tool", fmt.Errorf("%w: %s", ErrUnknownTool, name))
}
entry.Scope = string(tool.Scope)
if !agent.Allows(name) {
return finish(Result{}, OutcomeRefused, "not on the agent's allow-list", fmt.Errorf("%w: %s cannot use %s", ErrNotAllowed, agent.Name, name))
}
clean, err := tool.Schema.Validate(args)
if err != nil {
return finish(Result{}, OutcomeRefused, err.Error(), err)
}
entry.Args = clean
// A caller scoped to nothing must not be treated as a caller scoped to
// everything. Go's zero value is 0, so an unset tenant and a platform
// account look identical unless staff status is asked for separately.
if caller.Tenantid <= 0 && !caller.Superadmin {
return finish(Result{}, OutcomeRefused, "no tenant on the caller", ErrNoTenant)
}
result, err := tool.Handler(ctx, Request{Args: clean, Caller: caller})
if err != nil {
return finish(Result{}, OutcomeFailed, err.Error(), err)
}
return finish(result, OutcomeOK, "", nil)
}

View File

@@ -0,0 +1,327 @@
package tools
import (
"context"
"errors"
"testing"
"time"
)
// The registry is the only way to reach a tool, so these are the checks that
// stand between a model and the data. Each one is a thing the model could ask
// for and must not get.
func okTool(name string) Tool {
return Tool{
Name: name,
Description: "a tool, for testing",
Scope: ScopeRead,
Schema: Schema{Fields: []Field{{
Name: "limit", Description: "how many", Kind: KindInt, Min: 1, Max: 50, Default: 10,
}}},
Handler: func(_ context.Context, req Request) (Result, error) {
return Result{Rows: []int{1, 2}, Count: 2, Scope: "all branches"}, nil
},
}
}
func registryWith(t *testing.T, tools ...Tool) (*Registry, *CollectAudit) {
t.Helper()
audit := &CollectAudit{}
r := New(audit)
for _, tool := range tools {
if err := r.Register(tool); err != nil {
t.Fatalf("registering %s: %v", tool.Name, err)
}
}
return r, audit
}
var anyone = Caller{Userid: 904, Tenantid: 1147}
/* ── The five jobs ─────────────────────────────────────────────────────── */
func TestAToolRunsAndReturnsRows(t *testing.T) {
r, _ := registryWith(t, okTool("thing"))
agent := Agent{Name: "orders", Tools: []string{"thing"}}
result, err := r.Call(context.Background(), agent, "thing", nil, anyone)
if err != nil {
t.Fatalf("calling: %v", err)
}
if result.Count != 2 {
t.Fatalf("rows lost: %d", result.Count)
}
}
func TestAnUnknownToolIsRefused(t *testing.T) {
r, _ := registryWith(t, okTool("thing"))
agent := Agent{Name: "orders", Tools: []string{"thing"}}
_, err := r.Call(context.Background(), agent, "invented", nil, anyone)
if !errors.Is(err, ErrUnknownTool) {
t.Fatalf("a tool the model made up was not refused: %v", err)
}
}
func TestAToolOffTheAllowListIsRefusedByTheRegistryNotTheModel(t *testing.T) {
// The whole point of the allow-list: a prompt is a request, this is a rule.
r, _ := registryWith(t, okTool("thing"), okTool("other"))
agent := Agent{Name: "orders", Tools: []string{"thing"}}
_, err := r.Call(context.Background(), agent, "other", nil, anyone)
if !errors.Is(err, ErrNotAllowed) {
t.Fatalf("an agent reached a tool it does not name: %v", err)
}
}
func TestBadArgumentsNeverReachTheHandler(t *testing.T) {
reached := false
tool := okTool("thing")
tool.Handler = func(context.Context, Request) (Result, error) {
reached = true
return Result{}, nil
}
r, _ := registryWith(t, tool)
agent := Agent{Name: "orders", Tools: []string{"thing"}}
_, err := r.Call(context.Background(), agent, "thing", map[string]any{"limit": 999}, anyone)
if !errors.Is(err, ErrBadArgument) {
t.Fatalf("an out-of-range argument was accepted: %v", err)
}
if reached {
t.Fatal("the handler ran on arguments the schema refused")
}
}
func TestACallerScopedToNothingIsNotACallerScopedToEverything(t *testing.T) {
// Go's zero value is 0, so an unset tenant and a platform account look
// identical unless staff status is asked for separately.
r, _ := registryWith(t, okTool("thing"))
agent := Agent{Name: "orders", Tools: []string{"thing"}}
_, err := r.Call(context.Background(), agent, "thing", nil, Caller{Userid: 904})
if !errors.Is(err, ErrNoTenant) {
t.Fatalf("a caller with no tenant was let through: %v", err)
}
if _, err := r.Call(context.Background(), agent, "thing", nil, Caller{Userid: 12, Superadmin: true}); err != nil {
t.Fatalf("staff were refused: %v", err)
}
}
/* ── Arguments ─────────────────────────────────────────────────────────── */
func TestDefaultsAreFilledIn(t *testing.T) {
var seen Request
tool := okTool("thing")
tool.Handler = func(_ context.Context, req Request) (Result, error) {
seen = req
return Result{}, nil
}
r, _ := registryWith(t, tool)
if _, err := r.Call(context.Background(), Agent{Name: "a", Tools: []string{"thing"}}, "thing", nil, anyone); err != nil {
t.Fatalf("calling: %v", err)
}
if seen.Int("limit") != 10 {
t.Fatalf("the default did not arrive: %d", seen.Int("limit"))
}
}
func TestAJSONNumberIsAcceptedAsAnInteger(t *testing.T) {
// Every argument arrives over HTTP, so an integer field that only accepted
// Go ints would refuse every real call.
var seen Request
tool := okTool("thing")
tool.Handler = func(_ context.Context, req Request) (Result, error) {
seen = req
return Result{}, nil
}
r, _ := registryWith(t, tool)
if _, err := r.Call(context.Background(), Agent{Name: "a", Tools: []string{"thing"}}, "thing", map[string]any{"limit": float64(20)}, anyone); err != nil {
t.Fatalf("a JSON number was refused: %v", err)
}
if seen.Int("limit") != 20 {
t.Fatalf("the value did not survive: %d", seen.Int("limit"))
}
}
func TestAFractionIsNotAWholeNumber(t *testing.T) {
r, _ := registryWith(t, okTool("thing"))
_, err := r.Call(context.Background(), Agent{Name: "a", Tools: []string{"thing"}}, "thing", map[string]any{"limit": 2.5}, anyone)
if !errors.Is(err, ErrBadArgument) {
t.Fatalf("2.5 was accepted as a count: %v", err)
}
}
func TestAnInventedArgumentIsDroppedNotPassedOn(t *testing.T) {
// A handler must never receive a key it did not declare, or an argument the
// model made up becomes one a handler might later start reading.
var seen Request
tool := okTool("thing")
tool.Handler = func(_ context.Context, req Request) (Result, error) {
seen = req
return Result{}, nil
}
r, _ := registryWith(t, tool)
args := map[string]any{"limit": 5, "tenantid": 916, "where": "1=1"}
if _, err := r.Call(context.Background(), Agent{Name: "a", Tools: []string{"thing"}}, "thing", args, anyone); err != nil {
t.Fatalf("calling: %v", err)
}
if _, present := seen.Args["tenantid"]; present {
t.Fatal("the model got to name a tenant")
}
if _, present := seen.Args["where"]; present {
t.Fatal("the model got to pass a where clause")
}
}
/* ── Registration ──────────────────────────────────────────────────────── */
func TestAToolCannotBeSilentlyReplaced(t *testing.T) {
// Overwriting a registered tool is how a permission check disappears.
r, _ := registryWith(t, okTool("thing"))
if err := r.Register(okTool("thing")); err == nil {
t.Fatal("a second tool took the same name")
}
}
func TestAToolWithoutADescriptionIsRefused(t *testing.T) {
// A model chooses between tools by their descriptions. One without is a
// tool that gets called for the wrong question.
r := New(nil)
tool := okTool("thing")
tool.Description = ""
if err := r.Register(tool); err == nil {
t.Fatal("a tool with no description was registered")
}
}
/* ── What the model is told ────────────────────────────────────────────── */
func TestAnAgentIsOnlyToldAboutToolsItMayUse(t *testing.T) {
// Describing a tool the agent would then be refused reads to a model as a
// malfunction, and to a person as the assistant being broken.
r, _ := registryWith(t, okTool("thing"), okTool("other"))
defs := r.Definitions(Agent{Name: "orders", Tools: []string{"thing"}})
if len(defs) != 1 || defs[0]["name"] != "thing" {
t.Fatalf("the agent was told about the wrong tools: %v", defs)
}
}
func TestTheSchemaForbidsInventedArguments(t *testing.T) {
r, _ := registryWith(t, okTool("thing"))
defs := r.Definitions(Agent{Name: "orders", Tools: []string{"thing"}})
schema, _ := defs[0]["input_schema"].(map[string]any)
if schema["additionalProperties"] != false {
t.Fatal("the schema lets the model add its own arguments")
}
}
/* ── The audit trail ───────────────────────────────────────────────────── */
func TestEveryCallLeavesExactlyOneRow(t *testing.T) {
r, audit := registryWith(t, okTool("thing"))
agent := Agent{Name: "orders", Tools: []string{"thing"}}
_, _ = r.Call(context.Background(), agent, "thing", nil, anyone)
if len(audit.Entries) != 1 {
t.Fatalf("a successful call wrote %d rows", len(audit.Entries))
}
entry, _ := audit.Last()
if entry.Outcome != OutcomeOK || entry.Tool != "thing" || entry.Tenantid != 1147 {
t.Fatalf("the row does not describe the call: %+v", entry)
}
}
func TestARefusalIsAudited(t *testing.T) {
// The refusals are the interesting ones. A trail of successes answers
// "did anything try to read another tenant?" with silence, which reads the
// same as "no".
r, audit := registryWith(t, okTool("thing"), okTool("other"))
_, _ = r.Call(context.Background(), Agent{Name: "orders", Tools: []string{"thing"}}, "other", nil, anyone)
entry, ok := audit.Last()
if !ok || entry.Outcome != OutcomeRefused {
t.Fatalf("a refusal left no trace: %+v", entry)
}
if entry.Detail == "" {
t.Fatal("the refusal does not say why")
}
}
func TestABrokenToolIsFailedNotRefused(t *testing.T) {
// Collapsing the two hides a broken tool inside a count of things working
// as designed.
tool := okTool("thing")
tool.Handler = func(context.Context, Request) (Result, error) {
return Result{}, errors.New("the database is down")
}
r, audit := registryWith(t, tool)
_, err := r.Call(context.Background(), Agent{Name: "a", Tools: []string{"thing"}}, "thing", nil, anyone)
if err == nil {
t.Fatal("a broken handler reported success")
}
entry, _ := audit.Last()
if entry.Outcome != OutcomeFailed {
t.Fatalf("a handler error was recorded as %q", entry.Outcome)
}
}
func TestTheAuditKeepsWhatRanNotWhatWasSent(t *testing.T) {
// Defaults applied, invented keys dropped. What actually executed is the
// thing worth being able to read back.
r, audit := registryWith(t, okTool("thing"))
_, _ = r.Call(context.Background(), Agent{Name: "a", Tools: []string{"thing"}}, "thing",
map[string]any{"tenantid": 916}, anyone)
entry, _ := audit.Last()
if _, present := entry.Args["tenantid"]; present {
t.Fatal("the audit kept an argument the handler never saw")
}
if entry.Args["limit"] != 10 {
t.Fatalf("the applied default is missing from the trail: %+v", entry.Args)
}
}
func TestAnAuditLineIsStableBetweenIdenticalCalls(t *testing.T) {
// Go randomises map iteration, so without sorting the same call logs
// differently every time and a grep for one of them finds one of them.
entry := AuditEntry{
At: time.Now(), Agent: "orders", Tool: "thing", Userid: 904, Tenantid: 1147,
Args: map[string]any{"b": 2, "a": 1, "c": 3}, Outcome: OutcomeOK,
}
first := entry.Line()
for range 20 {
if entry.Line() != first {
t.Fatalf("two renderings of one entry differ:\n%s\n%s", first, entry.Line())
}
}
}
func TestAnAuditLineSaysWhenThereWasNoTenant(t *testing.T) {
// Explicitly, rather than by omission: no tenant on an assistant call is
// either staff or a bug, and both are worth being able to search for.
entry := AuditEntry{Agent: "orders", Tool: "thing", Userid: 12, Outcome: OutcomeOK}
if got := entry.Line(); !contains(got, "tenant=none") {
t.Fatalf("a tenantless call is invisible in the log: %s", got)
}
}
func contains(haystack, needle string) bool {
return len(haystack) >= len(needle) && (func() bool {
for i := 0; i+len(needle) <= len(haystack); i++ {
if haystack[i:i+len(needle)] == needle {
return true
}
}
return false
})()
}

View File

@@ -0,0 +1,256 @@
package tools
import (
"context"
"fmt"
"sort"
"strings"
"time"
"nearle/models"
)
// "Which orders are stuck?" — the first tool, and a real one.
//
// A delivery that is still `pending` some time after it was handed out is a job
// nobody has picked up. The rider may not have seen it, may have no device, may
// have put the phone down. The console cannot tell which, and does not need to:
// the wait itself is the fact worth surfacing, and every one of these is a
// customer waiting without knowing why.
//
// ── Derived, never remembered ───────────────────────────────────────────────
//
// Nothing stores "this job went unaccepted". It is computed from `assigntime`
// and `orderstatus`, both of which every delivery read already returns, so the
// answer does not depend on anybody having been watching when it happened.
//
// ── Why the two thresholds ──────────────────────────────────────────────────
//
// Ten minutes is worth a look; twenty-five needs somebody now. A single
// threshold either cries wolf at three minutes — which trains people to ignore
// it, and an ignored flag is worse than none — or stays silent until the
// customer has already called.
const (
// StuckLookMinutes is when a wait becomes worth a glance.
StuckLookMinutes = 10
// StuckNowMinutes is when it needs a person.
StuckNowMinutes = 25
// stuckMaxRows caps one answer. A model handed four hundred rows summarises
// them into a sentence nobody can check; a dispatcher can act on ten.
stuckMaxRows = 50
)
// DeliveryReader is the one thing this tool needs from the rest of the app.
//
// Narrowed to a single method so the tool can be tested without a database, and
// so it cannot quietly grow a second dependency. The real implementation is
// `services.DeliveriesService`.
type DeliveryReader interface {
GetDeliveries(input models.DeliveryQuery) []models.Deliveryinfo
}
// StuckOrder is one row of the answer.
//
// Field names are what the model will read back to a person, so they say what
// they mean: `WaitingMinutes`, not `delta`.
type StuckOrder struct {
Deliveryid int `json:"deliveryid"`
Orderid string `json:"orderid"`
Rider string `json:"rider,omitempty"`
Branch string `json:"branch,omitempty"`
Customer string `json:"customer,omitempty"`
AssignedAt string `json:"assigned_at"`
WaitingMinutes int `json:"waiting_minutes"`
// "look" or "now" — which bucket the wait falls in. Returned rather than
// left to the model to work out from the number, so the threshold is
// decided in one place and cannot be re-invented in a sentence.
Urgency string `json:"urgency"`
Action string `json:"action"`
}
// StuckOrders builds the tool.
//
// `now` is injected so the tests can ask what the board looked like at a fixed
// instant. Production passes `time.Now`.
func StuckOrders(deliveries DeliveryReader, now func() time.Time) Tool {
if now == nil {
now = time.Now
}
return Tool{
Name: "stuck_orders",
Description: "Deliveries a rider has been given but has not accepted yet, oldest first. " +
"Use for questions about jobs that are stuck, not moving, unaccepted, or riders who have not started. " +
"Returns the wait in minutes and what to do about each one.",
Scope: ScopeRead,
Schema: Schema{Fields: []Field{{
Name: "minutes_waiting",
Description: "Only count jobs unaccepted for at least this many minutes. Defaults to 10.",
Kind: KindInt,
Min: 1,
Max: 720,
Default: StuckLookMinutes,
}}},
Handler: func(_ context.Context, req Request) (Result, error) {
// The tenant comes from the verified session, never from an
// argument. There is deliberately no `tenantid` field on the schema
// above: a tool that accepted one would let the model be talked into
// reading somebody else's shop, and the model is the one part of
// this system that can be argued with.
if req.Caller.Tenantid <= 0 {
return Result{}, fmt.Errorf("stuck_orders needs a tenant; staff must pick one first")
}
threshold := req.Int("minutes_waiting")
if threshold <= 0 {
threshold = StuckLookMinutes
}
rows := deliveries.GetDeliveries(models.DeliveryQuery{
Tenantid: req.Caller.Tenantid,
// The caller's own branch when they have one. A branch user asking
// "what is stuck?" means their shop; an admin with no home branch
// means all of them.
Locationid: req.Caller.Locationid,
Pagesize: 500,
Pageno: 1,
})
at := now()
stuck := make([]StuckOrder, 0, 8)
for _, row := range rows {
waited, ok := unacceptedFor(row, at)
if !ok || waited < time.Duration(threshold)*time.Minute {
continue
}
minutes := int(waited.Minutes())
urgency, action := "look", "Check the rider has seen it."
if minutes >= StuckNowMinutes {
urgency, action = "now", "Call the rider, or give the job to somebody else."
}
stuck = append(stuck, StuckOrder{
Deliveryid: row.Deliveryid,
Orderid: row.Orderid,
// `ridername` is not reliably a name — on tenant 916 every
// rider has delivery statuses in that column too, and for two
// of five the status is the MORE common value. Excluded by
// vocabulary rather than by a hand-written list.
Rider: riderName(row.Ridername),
Branch: row.Locationname,
Customer: row.Deliverycustomer,
AssignedAt: row.Assigntime,
WaitingMinutes: minutes,
Urgency: urgency,
Action: action,
})
}
// Worst first, then longest waiting. This is a worklist, not a log:
// the row to deal with next belongs at the top.
sort.SliceStable(stuck, func(i, j int) bool {
if stuck[i].Urgency != stuck[j].Urgency {
return stuck[i].Urgency == "now"
}
return stuck[i].WaitingMinutes > stuck[j].WaitingMinutes
})
result := Result{
Count: len(stuck),
Source: "/admin/dispatch",
Scope: scopeWords(req.Caller),
}
if len(stuck) > stuckMaxRows {
result.Truncated = true
result.Note = fmt.Sprintf(
"%d jobs are waiting; the %d longest are listed. Say so — do not describe this as the full list.",
len(stuck), stuckMaxRows)
stuck = stuck[:stuckMaxRows]
}
result.Rows = stuck
return result, nil
},
}
}
// unacceptedFor is how long a job has sat with nobody accepting it.
//
// Only `pending` counts. A job the rider accepted, picked up or delivered is
// not stuck however old it is, and `rejected` or `skipped` is a different
// problem with a different answer — those are somebody's to reassign, not to
// chase.
//
// A stamp that will not parse returns false rather than 1970. Reading an
// unparseable `assigntime` as the epoch would report every such row as fifty
// years late, which is the kind of number that gets a whole screen ignored.
func unacceptedFor(row models.Deliveryinfo, now time.Time) (time.Duration, bool) {
if strings.ToLower(strings.TrimSpace(row.Orderstatus)) != "pending" {
return 0, false
}
assigned, ok := parseStamp(row.Assigntime)
if !ok {
return 0, false
}
waited := now.Sub(assigned)
if waited < 0 {
// A clock ahead of ours, not a job from the future.
return 0, false
}
return waited, true
}
// parseStamp reads `assigntime` as the writer actually writes it.
//
// Local wall-clock with no zone — `2026-09-23 14:05:31` — which is what
// `stampNow` produces. Parsed in the server's own location rather than UTC,
// because reading local digits as UTC would put every job five and a half hours
// out in India and turn a fresh assignment into a four-hour wait.
func parseStamp(raw string) (time.Time, bool) {
text := strings.TrimSpace(raw)
if text == "" {
return time.Time{}, false
}
for _, layout := range []string{
"2006-01-02 15:04:05",
"2006-01-02T15:04:05",
time.RFC3339,
} {
if at, err := time.ParseInLocation(layout, text, time.Local); err == nil {
return at, true
}
}
return time.Time{}, false
}
// riderName keeps a name and drops a status wearing one.
//
// `deliveries.ridername` carries both. Measured on tenant 916: rider 897 has
// "Varun" 69 times and "delivered" 75, so neither "first non-empty" nor "most
// common" finds the name. The statuses are excluded by vocabulary, so a status
// added to the ladder is excluded the same day.
func riderName(raw string) string {
name := strings.ToLower(strings.TrimSpace(raw))
if name == "" {
return ""
}
for _, status := range []string{
"pending", "accepted", "arrived", "picked", "active",
"skipped", "rejected", "delivered", "cancelled", "waiting",
} {
if name == status {
return ""
}
}
return strings.TrimSpace(raw)
}
// scopeWords says what the answer covers, in words a person would use.
func scopeWords(caller Caller) string {
if caller.Locationid > 0 {
return "this branch"
}
return "all branches"
}

View File

@@ -0,0 +1,305 @@
package tools
import (
"context"
"testing"
"time"
"nearle/models"
)
var stuckNow = time.Date(2026, 9, 23, 14, 0, 0, 0, time.Local)
// fakeDeliveries stands in for the deliveries service, and records what it was
// asked — the query matters as much as the answer, because that is where the
// tenant scope either is or is not.
type fakeDeliveries struct {
rows []models.Deliveryinfo
last models.DeliveryQuery
}
func (f *fakeDeliveries) GetDeliveries(input models.DeliveryQuery) []models.Deliveryinfo {
f.last = input
return f.rows
}
// assignedMinutesAgo writes the stamp the way `stampNow` does: local
// wall-clock, no zone.
func assignedMinutesAgo(n int) string {
return stuckNow.Add(-time.Duration(n) * time.Minute).Format("2006-01-02 15:04:05")
}
func job(over models.Deliveryinfo) models.Deliveryinfo {
if over.Orderstatus == "" {
over.Orderstatus = "pending"
}
if over.Assigntime == "" {
over.Assigntime = assignedMinutesAgo(30)
}
return over
}
func runStuck(t *testing.T, rows []models.Deliveryinfo, args map[string]any, caller Caller) (Result, *fakeDeliveries) {
t.Helper()
deliveries := &fakeDeliveries{rows: rows}
r := New(nil)
if err := r.Register(StuckOrders(deliveries, func() time.Time { return stuckNow })); err != nil {
t.Fatalf("registering: %v", err)
}
result, err := r.Call(context.Background(), Agent{Name: "orders", Tools: []string{"stuck_orders"}}, "stuck_orders", args, caller)
if err != nil {
t.Fatalf("calling: %v", err)
}
return result, deliveries
}
func rowsOf(t *testing.T, result Result) []StuckOrder {
t.Helper()
rows, ok := result.Rows.([]StuckOrder)
if !ok {
t.Fatalf("rows are not stuck orders: %T", result.Rows)
}
return rows
}
/* ── The tenant is never the model's to choose ─────────────────────────── */
func TestTheTenantComesFromTheSessionNotTheArguments(t *testing.T) {
// The single most important property of the whole registry. The model picks
// the tool and the arguments; it has no say in whose data is read.
_, deliveries := runStuck(t, nil, map[string]any{"tenantid": 916}, Caller{Userid: 904, Tenantid: 1147})
if deliveries.last.Tenantid != 1147 {
t.Fatalf("the query ran against tenant %d", deliveries.last.Tenantid)
}
}
func TestTheToolDoesNotEvenAcceptATenantArgument(t *testing.T) {
// Belt and braces: the schema must not have the field at all, so there is
// nothing to argue the model into filling in.
tool := StuckOrders(&fakeDeliveries{}, nil)
for _, field := range tool.Schema.Fields {
if field.Name == "tenantid" || field.Name == "locationid" {
t.Fatalf("the schema offers %q for the model to set", field.Name)
}
}
}
func TestABranchUserIsScopedToTheirBranch(t *testing.T) {
_, deliveries := runStuck(t, nil, nil, Caller{Userid: 904, Tenantid: 1147, Locationid: 1172})
if deliveries.last.Locationid != 1172 {
t.Fatalf("a branch user read branch %d", deliveries.last.Locationid)
}
}
func TestStaffMustPickATenantFirst(t *testing.T) {
// A platform account passes the registry's caller check but cannot ask this
// question of "everyone" — the answer would span merchants.
deliveries := &fakeDeliveries{}
r := New(nil)
_ = r.Register(StuckOrders(deliveries, func() time.Time { return stuckNow }))
_, err := r.Call(context.Background(), Agent{Name: "orders", Tools: []string{"stuck_orders"}},
"stuck_orders", nil, Caller{Userid: 12, Superadmin: true})
if err == nil {
t.Fatal("staff read stuck orders across every tenant at once")
}
}
/* ── What counts as stuck ──────────────────────────────────────────────── */
func TestOnlyPendingJobsAreStuck(t *testing.T) {
// A job the rider accepted, picked up or delivered is not stuck however old
// it is. Rejected and skipped are a different problem with a different fix.
rows := []models.Deliveryinfo{
job(models.Deliveryinfo{Deliveryid: 1, Orderstatus: "pending"}),
job(models.Deliveryinfo{Deliveryid: 2, Orderstatus: "accepted"}),
job(models.Deliveryinfo{Deliveryid: 3, Orderstatus: "picked"}),
job(models.Deliveryinfo{Deliveryid: 4, Orderstatus: "delivered"}),
job(models.Deliveryinfo{Deliveryid: 5, Orderstatus: "rejected"}),
job(models.Deliveryinfo{Deliveryid: 6, Orderstatus: "skipped"}),
}
result, _ := runStuck(t, rows, nil, anyone)
if result.Count != 1 || rowsOf(t, result)[0].Deliveryid != 1 {
t.Fatalf("expected only the pending job: %+v", result.Rows)
}
}
func TestTheCasingTheRiderAppActuallyWrites(t *testing.T) {
// Fiesta stores status as free text and the casing varies between writers.
rows := []models.Deliveryinfo{job(models.Deliveryinfo{Deliveryid: 1, Orderstatus: "Pending"})}
result, _ := runStuck(t, rows, nil, anyone)
if result.Count != 1 {
t.Fatal("a capitalised status stopped counting")
}
}
func TestAFewMinutesOfSilenceIsOrdinary(t *testing.T) {
// Flagging this would train people to ignore the flag, and an ignored flag
// is worse than none.
rows := []models.Deliveryinfo{
job(models.Deliveryinfo{Deliveryid: 1, Assigntime: assignedMinutesAgo(3)}),
job(models.Deliveryinfo{Deliveryid: 2, Assigntime: assignedMinutesAgo(9)}),
}
result, _ := runStuck(t, rows, nil, anyone)
if result.Count != 0 {
t.Fatalf("a nine-minute wait was reported as stuck: %+v", result.Rows)
}
}
func TestTenMinutesIsALookAndTwentyFiveNeedsSomebody(t *testing.T) {
rows := []models.Deliveryinfo{
job(models.Deliveryinfo{Deliveryid: 1, Assigntime: assignedMinutesAgo(12)}),
job(models.Deliveryinfo{Deliveryid: 2, Assigntime: assignedMinutesAgo(40)}),
}
result, _ := runStuck(t, rows, nil, anyone)
got := rowsOf(t, result)
// Worst first: the forty-minute job leads.
if got[0].Deliveryid != 2 || got[0].Urgency != "now" {
t.Fatalf("the urgent job is not first: %+v", got)
}
if got[1].Urgency != "look" {
t.Fatalf("a twelve-minute wait was not a look: %+v", got[1])
}
if got[0].Action == got[1].Action {
t.Fatal("both buckets suggest the same action, so the bucket says nothing")
}
}
func TestTheThresholdCanBeRaised(t *testing.T) {
rows := []models.Deliveryinfo{
job(models.Deliveryinfo{Deliveryid: 1, Assigntime: assignedMinutesAgo(12)}),
job(models.Deliveryinfo{Deliveryid: 2, Assigntime: assignedMinutesAgo(40)}),
}
result, _ := runStuck(t, rows, map[string]any{"minutes_waiting": 30}, anyone)
if result.Count != 1 || rowsOf(t, result)[0].Deliveryid != 2 {
t.Fatalf("the threshold was ignored: %+v", result.Rows)
}
}
func TestTheWaitIsReportedInMinutes(t *testing.T) {
rows := []models.Deliveryinfo{job(models.Deliveryinfo{Deliveryid: 1, Assigntime: assignedMinutesAgo(40)})}
result, _ := runStuck(t, rows, nil, anyone)
if got := rowsOf(t, result)[0].WaitingMinutes; got != 40 {
t.Fatalf("waiting minutes reported as %d", got)
}
}
/* ── Stamps that cannot be read ────────────────────────────────────────── */
func TestAnUnreadableAssigntimeIsSkippedNotDatedTo1970(t *testing.T) {
// Reading it as the epoch would report the row as fifty years late, which
// is the kind of number that gets a whole screen ignored.
// Built without `job()`, which fills a default stamp in — the helper would
// hand the parser a valid date and the test would pass without testing.
rows := []models.Deliveryinfo{
{Deliveryid: 1, Orderstatus: "pending", Assigntime: ""},
{Deliveryid: 2, Orderstatus: "pending", Assigntime: "not a date"},
{Deliveryid: 3, Orderstatus: "pending", Assigntime: " "},
}
result, _ := runStuck(t, rows, nil, anyone)
if result.Count != 0 {
t.Fatalf("an unreadable stamp produced a row: %+v", result.Rows)
}
}
func TestALocalStampIsNotReadAsUTC(t *testing.T) {
// `assigntime` is local wall-clock with no zone. Read as UTC it would be
// five and a half hours out in India, turning a fresh assignment into a
// four-hour wait.
rows := []models.Deliveryinfo{job(models.Deliveryinfo{Deliveryid: 1, Assigntime: assignedMinutesAgo(30)})}
result, _ := runStuck(t, rows, nil, anyone)
if got := rowsOf(t, result)[0].WaitingMinutes; got != 30 {
t.Fatalf("a local stamp read as %d minutes instead of 30", got)
}
}
func TestAJobFromTheFutureIsAClockNotAWait(t *testing.T) {
rows := []models.Deliveryinfo{job(models.Deliveryinfo{Deliveryid: 1, Assigntime: assignedMinutesAgo(-20)})}
result, _ := runStuck(t, rows, nil, anyone)
if result.Count != 0 {
t.Fatalf("a stamp in the future was reported as a wait: %+v", result.Rows)
}
}
/* ── The rider's name ──────────────────────────────────────────────────── */
func TestAStatusInTheRiderColumnIsNotAName(t *testing.T) {
// Measured on tenant 916: rider 897 carries "Varun" 69 times and
// "delivered" 75, so neither "first non-empty" nor "most common" finds the
// name. A model handed "delivered" would tell somebody to call a rider
// called Delivered.
rows := []models.Deliveryinfo{
job(models.Deliveryinfo{Deliveryid: 1, Ridername: "delivered"}),
job(models.Deliveryinfo{Deliveryid: 2, Ridername: "Varun"}),
}
result, _ := runStuck(t, rows, nil, anyone)
got := rowsOf(t, result)
byID := map[int]string{}
for _, row := range got {
byID[row.Deliveryid] = row.Rider
}
if byID[1] != "" {
t.Fatalf("a status was reported as a rider: %q", byID[1])
}
if byID[2] != "Varun" {
t.Fatalf("a real name was dropped: %q", byID[2])
}
}
/* ── Saying when the answer is partial ─────────────────────────────────── */
func TestACappedAnswerSaysSo(t *testing.T) {
// An empty answer and a capped answer look identical to a model, and it
// will describe both as "none".
rows := make([]models.Deliveryinfo, 0, 60)
for i := range 60 {
rows = append(rows, job(models.Deliveryinfo{Deliveryid: i + 1, Assigntime: assignedMinutesAgo(30 + i)}))
}
result, _ := runStuck(t, rows, nil, anyone)
if !result.Truncated {
t.Fatal("sixty rows came back as a complete answer")
}
if result.Note == "" {
t.Fatal("the cap is not explained in words the model will repeat")
}
if len(rowsOf(t, result)) != stuckMaxRows {
t.Fatalf("the cap did not apply: %d rows", len(rowsOf(t, result)))
}
if result.Count != 60 {
t.Fatalf("the true total was lost: %d", result.Count)
}
}
func TestAnEmptyBoardIsAnAnswer(t *testing.T) {
result, _ := runStuck(t, nil, nil, anyone)
if result.Count != 0 || result.Truncated {
t.Fatalf("an empty board was not answered cleanly: %+v", result)
}
if result.Scope == "" {
t.Fatal("even an empty answer must say what it covered")
}
}
/* ── Evidence ──────────────────────────────────────────────────────────── */
func TestTheAnswerCarriesWhereToCheckIt(t *testing.T) {
// Buddy states a conclusion; this is where a person goes to see the rows.
result, _ := runStuck(t, []models.Deliveryinfo{job(models.Deliveryinfo{Deliveryid: 1})}, nil, anyone)
if result.Source == "" {
t.Fatal("the answer links to nothing")
}
}
func TestTheScopeIsStatedSoOneBranchIsNotReadAsAll(t *testing.T) {
all, _ := runStuck(t, nil, nil, Caller{Userid: 904, Tenantid: 1147})
one, _ := runStuck(t, nil, nil, Caller{Userid: 904, Tenantid: 1147, Locationid: 1172})
if all.Scope == one.Scope {
t.Fatalf("one branch and all branches report the same scope: %q", all.Scope)
}
}

180
utils/webtoken.go Normal file
View File

@@ -0,0 +1,180 @@
package utils
import (
"crypto/hmac"
"encoding/base64"
"encoding/json"
"fmt"
"strings"
"time"
)
// Session tokens for the console.
//
// The same construction as the POS token next door — `base64url(payload).
// base64url(hmac-sha256)`, signed with the same key, deliberately not JWT —
// and for the same reasons, which `postoken.go` sets out in full. What differs
// is who is carrying it and what it is allowed to say.
//
// ── Why the console needs one at all ────────────────────────────────────────
//
// It has never had one. The console keeps its login record in `sessionStorage`
// and sends no `Authorization` header, so every `/web` endpoint has been taking
// `tenantid` off the query string and believing it. That is the same hole
// `middleware/posauth.go` was written to close on the POS surface — its own
// header describes a till naming another shop's id in a URL and being trusted —
// except that on the web surface nothing has closed it yet.
//
// ── A browser is not a till ─────────────────────────────────────────────────
//
// The POS token lasts thirty days because a shop signs a terminal in once and
// expects it to keep billing through reboots and dead networks. A browser tab
// is the opposite: the console already drops its session when the tab closes,
// because `sessionStorage` is per-tab by design. So the expiry here is a
// backstop for a tab left open, not the thing that ends the session, and a
// working day is the right order of magnitude.
//
// ── What the claims may say ─────────────────────────────────────────────────
//
// `Tenantid` is the load-bearing field, as `Locationid` is for POS. It is taken
// from the user's own record at sign-in and sealed under the signature, so a
// request can no longer name whichever tenant it likes.
//
// Platform access — Nearle's own staff, who work across every tenant and
// legitimately need to — rides on `Superadmin`, and NOT on the tenant being
// zero, nor on any role id.
//
// Both of those shortcuts are wrong, and the console learned it the hard way.
// `app_roles` calls roleid 1 "Super admin" and tenant onboarding wrote 1 for
// every shop owner, so a role test hands platform access to every merchant on
// the system. And a `Tenantid == 0` test promotes any user row whose tenant was
// never filled in — a missing field becoming the one session that reads
// everything. The real signal is `app_users.issuperadmin`, a column somebody
// has to deliberately set.
type WebClaims struct {
Userid int `json:"uid"`
// The tenant this session is pinned to. Every read and write stays inside
// it unless Superadmin says otherwise.
Tenantid int `json:"tid"`
// Nearle staff, from `app_users.issuperadmin`. The only thing that lifts
// the tenant pin; see above for the two tests that look equivalent and are
// not.
Superadmin bool `json:"sa,omitempty"`
// The user's home branch, where they have one. Not a restriction on its
// own: a tenant admin with six shops reads all six, and the check that
// decides which is `LocationAllowed` against the tenant, not this field.
Locationid int `json:"lid,omitempty"`
Roleid int `json:"rid"`
Configid int `json:"cid,omitempty"`
Issuedat int64 `json:"iat"`
Expiresat int64 `json:"exp"`
}
// WebTokenTTL is how long a console session stays valid.
//
// Twelve hours: longer than a shift, shorter than a week. The tab closing is
// what normally ends the session, so this only decides how long a tab left open
// overnight keeps working — and a person coming back the next morning signing
// in again is a reasonable thing to ask, where the same demand of a till
// mid-trade is not.
const WebTokenTTL = 12 * time.Hour
// IsPlatformAccount reports whether these claims may read across tenants.
//
// One function rather than `claims.Tenantid == 0` written out at each call
// site, so the rule can be found, tested, and changed in one place. Every
// cross-tenant decision in the middleware goes through it.
func (c WebClaims) IsPlatformAccount() bool { return c.Superadmin }
// MintWebToken issues a session for a signed-in console user.
//
// Shares `posTokenSecret` with the POS token: one signing key for the
// deployment, one place it can be missing, one error when it is. A second
// variable would be a second thing to forget.
func MintWebToken(claims WebClaims, now time.Time) (string, time.Time, error) {
secret, err := posTokenSecret()
if err != nil {
return "", time.Time{}, err
}
expires := now.Add(WebTokenTTL)
claims.Issuedat = now.Unix()
claims.Expiresat = expires.Unix()
payload, err := json.Marshal(claims)
if err != nil {
return "", time.Time{}, err
}
encoded := base64.RawURLEncoding.EncodeToString(payload)
return webTokenPrefix + encoded + "." + sign(encoded, secret), expires, nil
}
// webTokenPrefix keeps the two token kinds apart on the wire.
//
// Without it a POS token and a console token are the same shape signed with the
// same key, so one would verify as the other and a cashier's token would parse
// into web claims with `Locationid` in the seat `Tenantid` should occupy. The
// prefix is checked before the signature and is the reason `ParseWebToken`
// cannot accept a till's session.
const webTokenPrefix = "w1."
// ParseWebToken verifies a console token and returns what it claims.
//
// The order is the same as the POS parser's and matters for the same reason:
// nothing in the payload is trusted — not the expiry, not the tenant — until
// the signature has been checked. Reading `exp` from an unverified payload is
// taking the caller's word for when their own token runs out.
func ParseWebToken(token string, now time.Time) (WebClaims, error) {
secret, err := posTokenSecret()
if err != nil {
return WebClaims{}, err
}
raw := strings.TrimSpace(token)
after, found := strings.CutPrefix(raw, webTokenPrefix)
if !found {
return WebClaims{}, fmt.Errorf("not a console session token")
}
encoded, signature, found := strings.Cut(after, ".")
if !found || encoded == "" || signature == "" {
return WebClaims{}, fmt.Errorf("malformed session token")
}
// Constant time, so the right signature cannot be learned a byte at a time
// from how long the comparison took.
if !hmac.Equal([]byte(signature), []byte(sign(encoded, secret))) {
return WebClaims{}, fmt.Errorf("session token signature does not verify")
}
payload, err := base64.RawURLEncoding.DecodeString(encoded)
if err != nil {
return WebClaims{}, fmt.Errorf("malformed session token")
}
var claims WebClaims
if err := json.Unmarshal(payload, &claims); err != nil {
return WebClaims{}, fmt.Errorf("malformed session token")
}
if claims.Expiresat > 0 && now.Unix() >= claims.Expiresat {
return WebClaims{}, fmt.Errorf("session has expired; sign in again")
}
// A token naming nobody authorises nothing, and must not be mistaken for one
// authorising everything.
if claims.Userid <= 0 {
return WebClaims{}, fmt.Errorf("session token names no user")
}
// A tenant session must name its tenant. Staff are the only accounts that
// may carry none, and they have to say so explicitly.
if claims.Tenantid <= 0 && !claims.Superadmin {
return WebClaims{}, fmt.Errorf("session token names no tenant")
}
return claims, nil
}
// WebTokenConfigured reports whether console sessions can be issued at all.
func WebTokenConfigured() bool { return PosTokenConfigured() }

216
utils/webtoken_test.go Normal file
View File

@@ -0,0 +1,216 @@
package utils
import (
"strings"
"testing"
"time"
)
func TestAConsoleSessionSurvivesTheRoundTrip(t *testing.T) {
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
token, expires, err := MintWebToken(WebClaims{
Userid: 904, Tenantid: 1147, Locationid: 1172, Roleid: 3, Configid: 2,
}, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
claims, err := ParseWebToken(token, now.Add(time.Hour))
if err != nil {
t.Fatalf("parsing a token we just issued: %v", err)
}
if claims.Tenantid != 1147 || claims.Userid != 904 {
t.Fatalf("the identity did not survive: user %d tenant %d", claims.Userid, claims.Tenantid)
}
if claims.Locationid != 1172 || claims.Roleid != 3 {
t.Fatalf("branch or role lost: location %d role %d", claims.Locationid, claims.Roleid)
}
if !expires.After(now) {
t.Fatalf("expiry is not in the future: %v", expires)
}
}
/* ── The two token kinds must not be interchangeable ────────────────────── */
func TestATillsTokenIsNotAConsoleSession(t *testing.T) {
// The whole reason `webTokenPrefix` exists. Both tokens are the same shape
// signed with the same key, so without the prefix a POS token verifies as a
// web one — and its `Locationid` would land where `Tenantid` is read, which
// is the field every permission decision is made on.
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
pos, _, err := MintPosToken(PosClaims{Userid: 7, Tenantid: 916, Locationid: 1185, Roleid: 8}, now)
if err != nil {
t.Fatalf("minting a POS token: %v", err)
}
if _, err := ParseWebToken(pos, now); err == nil {
t.Fatal("a cashier's token was accepted as a console session")
}
}
func TestAConsoleSessionIsNotATillsToken(t *testing.T) {
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
web, _, err := MintWebToken(WebClaims{Userid: 904, Tenantid: 1147}, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
if _, err := ParsePosToken(web, now); err == nil {
t.Fatal("a console session was accepted at the till")
}
}
/* ── Forgery and tampering ─────────────────────────────────────────────── */
func TestATamperedTenantDoesNotVerify(t *testing.T) {
// The attack this is all for: take a valid session, change the tenant, read
// somebody else's shop.
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
token, _, err := MintWebToken(WebClaims{Userid: 904, Tenantid: 1147}, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
body, signature, _ := strings.Cut(strings.TrimPrefix(token, webTokenPrefix), ".")
forged := webTokenPrefix + body[:len(body)-1] + "X" + "." + signature
if _, err := ParseWebToken(forged, now); err == nil {
t.Fatal("an edited payload verified")
}
}
func TestATokenSignedWithAnotherKeyIsRefused(t *testing.T) {
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
withSecret(t, "a-completely-different-signing-key")
token, _, err := MintWebToken(WebClaims{Userid: 904, Tenantid: 1147}, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
withSecret(t, testSecret)
if _, err := ParseWebToken(token, now); err == nil {
t.Fatal("a token signed with someone else's key verified")
}
}
func TestNoSigningKeyMeansNoSessions(t *testing.T) {
t.Setenv("POS_TOKEN_SECRET", "")
t.Setenv("JWT_SECRET_KEY", "")
if _, _, err := MintWebToken(WebClaims{Userid: 1, Tenantid: 1}, time.Now()); err == nil {
t.Fatal("a session was issued with no signing key")
}
if WebTokenConfigured() {
t.Fatal("reported configured with no signing key")
}
}
/* ── Expiry ────────────────────────────────────────────────────────────── */
func TestASessionExpires(t *testing.T) {
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
token, _, err := MintWebToken(WebClaims{Userid: 904, Tenantid: 1147}, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
if _, err := ParseWebToken(token, now.Add(WebTokenTTL-time.Minute)); err != nil {
t.Fatalf("refused inside its life: %v", err)
}
if _, err := ParseWebToken(token, now.Add(WebTokenTTL+time.Minute)); err == nil {
t.Fatal("a tab left open overnight still authorised")
}
}
/* ── The platform account ──────────────────────────────────────────────── */
func TestPlatformAccessComesFromSuperadminAndNothingElse(t *testing.T) {
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
token, _, err := MintWebToken(WebClaims{Userid: 12, Superadmin: true, Roleid: 1}, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
claims, err := ParseWebToken(token, now)
if err != nil {
t.Fatalf("a staff session was refused: %v", err)
}
if !claims.IsPlatformAccount() {
t.Fatal("issuperadmin did not grant platform access")
}
}
func TestRoleid1IsAMerchantNotAPlatformOperator(t *testing.T) {
// `app_roles` calls roleid 1 "Super admin", and tenant onboarding wrote 1
// for every shop owner. A role test here would hand cross-tenant access to
// every merchant on the system — the console already had to fix this once.
claims := WebClaims{Userid: 904, Tenantid: 1147, Roleid: 1}
if claims.IsPlatformAccount() {
t.Fatal("roleid 1 claimed platform access")
}
}
func TestAMissingTenantIsNotAPlatformAccount(t *testing.T) {
// The other near-miss: a user row whose tenant was never filled in must not
// become the one session that reads everything. Go's zero value is 0, so
// this is exactly what a forgotten field looks like.
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
claims := WebClaims{Userid: 904, Tenantid: 0}
if claims.IsPlatformAccount() {
t.Fatal("a missing tenant claimed platform access")
}
token, _, err := MintWebToken(claims, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
if _, err := ParseWebToken(token, now); err == nil {
t.Fatal("a session naming no tenant and claiming no staff status verified")
}
}
func TestATokenNamingNobodyIsRefused(t *testing.T) {
// A token that authorises nothing must not be mistaken for one that
// authorises everything.
withSecret(t, testSecret)
now := time.Date(2026, 9, 23, 9, 0, 0, 0, time.UTC)
token, _, err := MintWebToken(WebClaims{Userid: 0, Tenantid: 1147}, now)
if err != nil {
t.Fatalf("minting: %v", err)
}
if _, err := ParseWebToken(token, now); err == nil {
t.Fatal("a token naming no user verified")
}
}
/* ── Shape ─────────────────────────────────────────────────────────────── */
func TestMalformedTokensAreRefusedWithoutPanicking(t *testing.T) {
withSecret(t, testSecret)
now := time.Now()
for _, token := range []string{
"", " ", "w1.", "w1..", "w1.onlyonepart",
"w1.!!!not-base64!!!.sig", "no-prefix.payload.sig",
} {
if _, err := ParseWebToken(token, now); err == nil {
t.Fatalf("accepted a malformed token: %q", token)
}
}
}