backend_fiesta/POS_TERMINAL_INGEST.md

# POS terminal ingest

How an in-store Nearle POS till reaches this backend. The terminal side of the
contract is specified in the POS repository at `docs/sync-contract.md`; this
covers what was built here and how to turn it on.

## What a terminal expects, and why it matters

A till stores every bill in its own SQLite database the moment a sale
completes, and keeps it for **seven days after we acknowledge it**. It marks a
bill synced if — and only if — the bill's id appears in the `accepted` list of
our reply.

That single rule drives every decision below:

- **Silence is not acceptance.** No reply, an empty reply, a 200 with no body:
  all leave the bill on the till, and it is sent again. This is the correct
  behaviour when we are struggling, and it is why a failing ingest never
  acknowledges.
- **A duplicate is a success.** Delivery is at-least-once. A lost ack makes a
  terminal re-send bills we already hold, and calling those failures would
  strand a day of takings. The ingest recognises them and accepts them without
  touching stock again.
- **A rejection is a decision.** Naming an id in `rejected` stops the till
  retrying it and waits for a person. Right for "this bill is malformed", wrong
  for "the database is having a bad minute".

## Two ways in, one code path

Both transports call `services.PosService`, so a bill arriving over MQTT and
one arriving over HTTP cannot diverge.

### Where a bill lands

Counter sales are written to **`pos_orders` / `pos_order_items`**, not to
`orders`. A bill is a different document from an app order: it carries a
cashier, a terminal, a rounding adjustment, promo campaigns, loyalty movement
and a payment split across several tenders, none of which `orders` has anywhere
to put. Forcing one into the other's shape loses whichever fields do not fit,
silently.

**Stock is not separate.** A counter sale writes the same `productstocks`
"out" rows an app order does, through the shared helpers in `stockLedger.go` —
the same row locks, the same availability check, the same availability re-sync.
Two stock ledgers would mean the catalogue pull sends a till figures that ignore
the till's own trading, and it would oversell.

Existing revenue queries were extended to include `pos_orders`
(`GetRevenueSummary`, `GetSalesSummary`), so dashboards do not understate a shop
that runs a counter. **Any new report has to remember to do the same** — that is
the standing cost of the split.

### HTTP

| Method | Path | Purpose |
|---|---|---|
| `POST` | `/live/api/v1/pos/orders` | Completed bills |
| `POST` | `/live/api/v1/pos/customers` | Shoppers registered at a till |
| `GET` | `/live/api/v1/pos/catalogue` | Product pull. Query: `store_id`, `since`, `page`, `page_size` |

These answer with a **bare ack**, not the usual `{code, message, status}`
envelope — the terminal reads `accepted` from the top level of the body:

```json
{ "batch_id": "9f1c…", "accepted": ["order-uuid"], "rejected": {} }
```

Status codes carry the rest of the contract:

- **200** — batch processed. Individual bills may still be refused; the ack says which.
- **4xx** — the request is wrong (unknown outlet, bad store id). The till halts and shows a person.
- **5xx** — outcome unknown. The till keeps everything and retries with backoff.

### MQTT

The broker is **Eclipse Mosquitto 2.1.2** at `66.116.225.226:1883`, shared with
the rider app (`nearle/riders/#`) and the doormile project (`doormile/riders/#`).

There is **no NATS** in this deployment. NATS servers exist for other projects
on other hosts, but their ports are closed from here and their configs carry no
`mqtt {}` block, so they expose no MQTT gateway. The NATS consumer that briefly
lived in this package has been deleted rather than left to rot.

| Env | Purpose |
|---|---|
| `MQTT_URL` | `tcp://66.116.225.226:1883`. **Unset disables MQTT ingest** — HTTP still works |
| `MQTT_USER` / `MQTT_PASSWORD` | Broker credentials |
| `MQTT_CLIENT_ID` | Defaults to `nearle-pos-ingest`. **Must be unique per replica** — a second connection with the same id evicts the first, and the two would fight in a loop |

| Topic | Direction |
|---|---|
| `nearle/pos/+/+/order` | till → us |
| `nearle/pos/+/+/customer` | till → us |
| `nearle/pos/+/+/health` | till → us, every 30s |
| `nearle/pos/{loc}/{terminal}/ack` | us → till |
| `nearle/pos/{loc}/catalogue` | us → every till at a shop |

Subscribed with `CleanSession(false)` and a stable client id, so a brief restart
resumes rather than missing what arrived meanwhile. Re-subscribes on every
reconnect, because a broker that did not persist the session would otherwise
come back subscribed to nothing.

**Do not treat the broker as durable storage.** Two measured facts make that
unsafe:

- `max_queued_messages` is at its default of **1000**. If this backend is down
  long enough for a hundred tills to exceed that, Mosquitto silently drops the
  overflow.
- Mosquitto's `autosave_interval` defaults to **30 minutes**, so a hard kill can
  lose up to half an hour of persisted state.

Neither loses a bill, and that is the whole point of the acknowledgement design:
a dropped message is simply never acked, so the terminal keeps its copy and
sends it again. The broker is a transport, not a ledger.

**The store and terminal are read from the topic, never from the body.** A till
that could name its own store in the payload could redirect another counter's
acknowledgements.

## Configuring a terminal

Settings → Connectivity & sync → Configure.

| Field | Value |
|---|---|
| Store ID | **The numeric `locationid`.** Not a name — the tenant is resolved from it |
| Terminal name | Whatever staff call the till |
| Transport | `HTTP` or `MQTT` |
| Base URL (HTTP) | `https://your-host/live/api/v1/pos` |
| Broker host / port (MQTT) | `66.116.225.226`, port `1883`, **TLS off** (8883 is not configured) |

`store_id` carrying the locationid is load-bearing: `resolvePosStore` looks the
tenant up from it and refuses a location that is not registered. A terminal
cannot name its tenant.

## Mapping decisions

Worth knowing before the first bill lands.

- **Idempotency** is a unique index on `pos_orders.terminalorderid` — the UUID
  minted at the till — plus a Postgres advisory lock held for the life of the
  transaction, so a redelivery arriving concurrently waits and then sees the
  committed row rather than racing past the check.
- **`businessdate` is the day the sale was rung**, not the day it arrived. A
  till that was offline overnight uploads yesterday's bills this morning, and
  they belong to yesterday. Every daily figure keys on this.
- **Line amounts are scaled onto the bill total.** The till sends each line at
  its pre-apportionment value while the header carries the total after
  bill-level discounts. Left alone the item rows would sum to the subtotal and
  every report that adds up lines would disagree with the one reading the
  header.
- **Payment mode** is the largest tender on a split bill; the full split is
  kept verbatim in `paymentsjson` for drawer reconciliation.
- **Fractional quantities round *up* for stock.** `productstocks.quantity` is an
  integer column, so 1.5 kg of onions cannot be recorded exactly. Rounding up
  never under-deducts, so recorded stock is never higher than the shelf. The app
  order path truncates instead (1.5 → 1), which under-deducts; that behaviour was
  left untouched rather than silently changed for live traffic. **Making the
  column numeric is the real fix.**
- **Customers** match on `contactno` within the outlet's `applocationid`, so a
  shopper registered at a till and one who installed the app become one row.
  Registrations are **insert-if-absent** — never an update, so a profile
  corrected at head office is not reverted by a terminal replaying an old
  capture.
- **Catalogue** answers `is_delta: false` and is therefore a full snapshot. The
  terminal withdraws every product a snapshot omits, so this must stay true
  while the query returns everything stocked at the outlet.
- **Barcodes** come from `products.productsku` — there is no barcode column.
  Scanning at the till matches on it, so SKUs must be the scannable code for
  barcode scanning to work.

## Terminal health

Every till publishes a heartbeat to `nearle/pos/{loc}/{terminal}/health` every
**30 seconds**. It is stored in Redis, never in Postgres.

| Env | Purpose |
|---|---|
| `REDIS_HOST` / `REDIS_PORT` | **Unset disables presence.** Point at the same Redis the express backend uses |
| `REDIS_USER` / `REDIS_PASSWORD` / `REDIS_DB` | Defaults `default`, empty, `0` |

```
pos:terminal:{terminalcode}          HASH, TTL 90s
pos:location:{locationid}:terminals  SET,  no TTL
```

The TTL is the whole design. A heartbeat is a fact with an expiry date: a till
that loses power stops refreshing, the key expires, and it disappears from the
board with nothing having to notice. In Postgres this would need ~288,000 writes
a day across a hundred tills *and* a reaper job, because a row saying "online"
cannot age out by itself.

90 seconds is three missed beats. Two would make an ordinary GPRS hiccup look
like a dead till; five would take two and a half minutes to notice a real one.

The set has **no TTL**, mirroring `city:{tenantid}:active_deliveries` in the
express backend: it is an index of what exists, not a claim that any of it is
alive. Membership means "this till has been seen here"; liveness is whether the
hash still exists.

Keys are namespaced `pos:*` and do not collide with express's `delivery:*`,
`city:*` or `rider_*`. **Worth keeping that way** — a shared datastore only stays
safe while each writer's keys are obviously its own.

A heartbeat is **never acknowledged**. Presence is fire-and-forget: a till that
stopped selling because a dashboard was busy would be a self-inflicted outage.

Read it back:

| Method | Path |
|---|---|
| `GET` | `/live/api/v1/pos/health/terminal?terminal_id=T4A9` |
| `GET` | `/live/api/v1/pos/health/location?location_id=12` |

A till whose key has expired comes back marked `offline` rather than being
omitted — omitting it would make a dead terminal indistinguishable from one that
was never installed, and the dead one is exactly what somebody is looking for.

What a heartbeat carries: identity and app version; **queue depth**
(`pending_bills`, `pending_registrations`, `oldest_pending_at`) — the numbers
that make a silent sync failure visible; **today's trading** (`today_bills`,
`today_amount`, `last_bill_at`) — a till that is connected but has rung nothing
in three hours is usually a jammed printer or an absent cashier; and device
state.

## Not built

- **Catalogue deltas.** Every pull is a full snapshot. Fine for a few hundred
  products, worth revisiting at a few thousand.
- **Loyalty coming back down.** The uplink deliberately carries no points or
  spend — those belong to the bill stream, which is idempotent and sees every
  counter. Nothing yet computes them centrally and sends them to the tills, so
  a shopper's balance at a till is that till's view.
- **Device authentication.** A terminal is trusted with a locationid. Signed
  device tokens are the obvious next step before this is exposed publicly.
- **Battery and free storage in the heartbeat.** The reporter has a hook for
  them, but this build collects neither — they need platform packages a desktop
  build has no use for. Fields that are not collected are **omitted**, not sent
  as zero: a board showing every till at 0% battery is worse than one showing
  nothing.

## Broker accounts

Applied 2026-08-03 on `66.116.225.226`. Two scoped accounts now exist alongside
`admin`, with an ACL at `/mosquitto/config/acl` referenced from
`mosquitto.conf`.

| User | May publish | May subscribe |
|---|---|---|
| `pos_terminal` | `nearle/pos/+/+/{order,customer,status,health}` | `nearle/pos/+/+/{ack,command}`, `nearle/pos/+/catalogue` |
| `pos_ingest` | `nearle/pos/+/+/{ack,command}`, `nearle/pos/+/catalogue` | `nearle/pos/+/+/{order,customer,health,status}` |
| `admin` | everything — **deliberately unchanged** | everything |

A till therefore cannot publish to `nearle/riders/#` or `doormile/#`, and cannot
write its own ack topic — only the ingest may do that. Verified by publishing as
`pos_terminal` to all four and watching which arrived: the order did, the other
three did not.

**`admin` was left unrestricted on purpose.** Its credentials are compiled into
the rider app, so narrowing it here would cut off the live rider fleet without
warning. The right next step is:

```conf
user admin
topic readwrite nearle/riders/#
topic readwrite doormile/#
```

but only once someone has confirmed nothing else authenticates as `admin`.
Until then the ACL changes nothing for it — which is why applying it was safe.

Rollback, if ever needed:

```bash
cp /root/Mqtt/backup-<timestamp>/{mosquitto.conf,passwd} /root/Mqtt/config/
docker restart mqtt_broker
```

**Still outstanding on the broker:**

- **No TLS.** Port 8883 is not configured. Bills carry customer names and mobile
  numbers, and they travel in the clear. Traefik on the same host already
  terminates 443, so certificates exist to borrow from.
- **`passwd` is world-readable.** Mosquitto warns about it and future versions
  will refuse to load it. Tightening it means `chown 1883:1883` as well as
  `chmod`, because the broker runs as uid 1883 and a root-owned 0600 file would
  stop it starting.
- **Credentials in source.** `admin` is in the rider APK, Redis is hardcoded in
  the express backend, and Postgres was in this repository's git history until
  2026-08-03. The POS accounts above are the only ones not in any source tree —
  keep it that way.

## Capacity

Current load, measured: **~0.9 msg/s inbound**, 5 connected clients, 112
retained messages totalling 8 KB.

A hundred tills add roughly 3.3 msg/s steady (a 1 KB heartbeat each per 30s)
plus bursts of up to ~50 KB when a sale batch goes up. That is 3–4× current
traffic and well within what Mosquitto handles on any VPS. The broker will not
be the bottleneck; Postgres write throughput on bill ingest is the thing to
watch instead.