Ingest counter sales from the POS terminals, over MQTT and HTTP
A till holds every bill in its own SQLite database and keeps it for seven days after we acknowledge it, marking one synced only when its id comes back in an ack. Everything here follows from that. Silence is not acceptance, so a failing ingest publishes nothing at all and the terminal simply sends again. A duplicate is a success, because at-least-once delivery means a lost ack legitimately re-delivers bills we already hold, and calling those failures would strand a day of takings on the till. Deduplication is a unique index on the terminal's UUID plus an advisory lock held for the transaction. Bills land in pos_orders / pos_order_items rather than orders: a counter bill carries a cashier, a terminal, a rounding adjustment, promos, loyalty movement and a payment split that orders has nowhere to put, and forcing one into the other loses whatever does not fit. Stock is *not* split — a counter sale writes the same productstocks rows an app order does, through helpers extracted from createOrderTx so the rule that prevents overselling has one implementation rather than two. GetRevenueSummary and GetSalesSummary were extended to union the new table in; any new report has to remember the same. Terminal health goes to Redis under a 90-second TTL, sharing the instance the express backend uses. A heartbeat is a fact with an expiry date: a till that loses power stops refreshing and ages off the board by itself, where a Postgres row would need ~288k writes a day and a reaper. Proven end to end against the live estate before commit: a bill over HTTP and one over the real Mosquitto broker, the same bill three times producing one row and one stock movement, and a heartbeat arriving on the health endpoint. All probe data was removed afterwards. Four things that only surfaced against real data. An unset jsonb column failed the very first bill. Product SKUs are unusable as barcodes — 6,245 products share 93 SKUs and "1" covers 5,794 of them — against the till's unique index, so barcodes fall back to the product id. A taxpercent of -1 exists and would have put negative GST in a filed slab. And a product with id 0 exists, which can never be billed and is now skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Suriya
321
POS_TERMINAL_INGEST.md
Normal file
321
POS_TERMINAL_INGEST.md
Normal file
@@ -0,0 +1,321 @@
|
||||
# POS terminal ingest
|
||||
|
||||
How an in-store Nearle POS till reaches this backend. The terminal side of the
|
||||
contract is specified in the POS repository at `docs/sync-contract.md`; this
|
||||
covers what was built here and how to turn it on.
|
||||
|
||||
## What a terminal expects, and why it matters
|
||||
|
||||
A till stores every bill in its own SQLite database the moment a sale
|
||||
completes, and keeps it for **seven days after we acknowledge it**. It marks a
|
||||
bill synced if — and only if — the bill's id appears in the `accepted` list of
|
||||
our reply.
|
||||
|
||||
That single rule drives every decision below:
|
||||
|
||||
- **Silence is not acceptance.** No reply, an empty reply, a 200 with no body:
|
||||
all leave the bill on the till, and it is sent again. This is the correct
|
||||
behaviour when we are struggling, and it is why a failing ingest never
|
||||
acknowledges.
|
||||
- **A duplicate is a success.** Delivery is at-least-once. A lost ack makes a
|
||||
terminal re-send bills we already hold, and calling those failures would
|
||||
strand a day of takings. The ingest recognises them and accepts them without
|
||||
touching stock again.
|
||||
- **A rejection is a decision.** Naming an id in `rejected` stops the till
|
||||
retrying it and waits for a person. Right for "this bill is malformed", wrong
|
||||
for "the database is having a bad minute".
|
||||
|
||||
## Two ways in, one code path
|
||||
|
||||
Both transports call `services.PosService`, so a bill arriving over MQTT and
|
||||
one arriving over HTTP cannot diverge.
|
||||
|
||||
### Where a bill lands
|
||||
|
||||
Counter sales are written to **`pos_orders` / `pos_order_items`**, not to
|
||||
`orders`. A bill is a different document from an app order: it carries a
|
||||
cashier, a terminal, a rounding adjustment, promo campaigns, loyalty movement
|
||||
and a payment split across several tenders, none of which `orders` has anywhere
|
||||
to put. Forcing one into the other's shape loses whichever fields do not fit,
|
||||
silently.
|
||||
|
||||
**Stock is not separate.** A counter sale writes the same `productstocks`
|
||||
"out" rows an app order does, through the shared helpers in `stockLedger.go` —
|
||||
the same row locks, the same availability check, the same availability re-sync.
|
||||
Two stock ledgers would mean the catalogue pull sends a till figures that ignore
|
||||
the till's own trading, and it would oversell.
|
||||
|
||||
Existing revenue queries were extended to include `pos_orders`
|
||||
(`GetRevenueSummary`, `GetSalesSummary`), so dashboards do not understate a shop
|
||||
that runs a counter. **Any new report has to remember to do the same** — that is
|
||||
the standing cost of the split.
|
||||
|
||||
### HTTP
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `POST` | `/live/api/v1/pos/orders` | Completed bills |
|
||||
| `POST` | `/live/api/v1/pos/customers` | Shoppers registered at a till |
|
||||
| `GET` | `/live/api/v1/pos/catalogue` | Product pull. Query: `store_id`, `since`, `page`, `page_size` |
|
||||
|
||||
These answer with a **bare ack**, not the usual `{code, message, status}`
|
||||
envelope — the terminal reads `accepted` from the top level of the body:
|
||||
|
||||
```json
|
||||
{ "batch_id": "9f1c…", "accepted": ["order-uuid"], "rejected": {} }
|
||||
```
|
||||
|
||||
Status codes carry the rest of the contract:
|
||||
|
||||
- **200** — batch processed. Individual bills may still be refused; the ack says which.
|
||||
- **4xx** — the request is wrong (unknown outlet, bad store id). The till halts and shows a person.
|
||||
- **5xx** — outcome unknown. The till keeps everything and retries with backoff.
|
||||
|
||||
### MQTT
|
||||
|
||||
The broker is **Eclipse Mosquitto 2.1.2** at `66.116.225.226:1883`, shared with
|
||||
the rider app (`nearle/riders/#`) and the doormile project (`doormile/riders/#`).
|
||||
|
||||
There is **no NATS** in this deployment. NATS servers exist for other projects
|
||||
on other hosts, but their ports are closed from here and their configs carry no
|
||||
`mqtt {}` block, so they expose no MQTT gateway. The NATS consumer that briefly
|
||||
lived in this package has been deleted rather than left to rot.
|
||||
|
||||
| Env | Purpose |
|
||||
|---|---|
|
||||
| `MQTT_URL` | `tcp://66.116.225.226:1883`. **Unset disables MQTT ingest** — HTTP still works |
|
||||
| `MQTT_USER` / `MQTT_PASSWORD` | Broker credentials |
|
||||
| `MQTT_CLIENT_ID` | Defaults to `nearle-pos-ingest`. **Must be unique per replica** — a second connection with the same id evicts the first, and the two would fight in a loop |
|
||||
|
||||
| Topic | Direction |
|
||||
|---|---|
|
||||
| `nearle/pos/+/+/order` | till → us |
|
||||
| `nearle/pos/+/+/customer` | till → us |
|
||||
| `nearle/pos/+/+/health` | till → us, every 30s |
|
||||
| `nearle/pos/{loc}/{terminal}/ack` | us → till |
|
||||
| `nearle/pos/{loc}/catalogue` | us → every till at a shop |
|
||||
|
||||
Subscribed with `CleanSession(false)` and a stable client id, so a brief restart
|
||||
resumes rather than missing what arrived meanwhile. Re-subscribes on every
|
||||
reconnect, because a broker that did not persist the session would otherwise
|
||||
come back subscribed to nothing.
|
||||
|
||||
**Do not treat the broker as durable storage.** Two measured facts make that
|
||||
unsafe:
|
||||
|
||||
- `max_queued_messages` is at its default of **1000**. If this backend is down
|
||||
long enough for a hundred tills to exceed that, Mosquitto silently drops the
|
||||
overflow.
|
||||
- Mosquitto's `autosave_interval` defaults to **30 minutes**, so a hard kill can
|
||||
lose up to half an hour of persisted state.
|
||||
|
||||
Neither loses a bill, and that is the whole point of the acknowledgement design:
|
||||
a dropped message is simply never acked, so the terminal keeps its copy and
|
||||
sends it again. The broker is a transport, not a ledger.
|
||||
|
||||
**The store and terminal are read from the topic, never from the body.** A till
|
||||
that could name its own store in the payload could redirect another counter's
|
||||
acknowledgements.
|
||||
|
||||
## Configuring a terminal
|
||||
|
||||
Settings → Connectivity & sync → Configure.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Store ID | **The numeric `locationid`.** Not a name — the tenant is resolved from it |
|
||||
| Terminal name | Whatever staff call the till |
|
||||
| Transport | `HTTP` or `MQTT` |
|
||||
| Base URL (HTTP) | `https://your-host/live/api/v1/pos` |
|
||||
| Broker host / port (MQTT) | `66.116.225.226`, port `1883`, **TLS off** (8883 is not configured) |
|
||||
|
||||
`store_id` carrying the locationid is load-bearing: `resolvePosStore` looks the
|
||||
tenant up from it and refuses a location that is not registered. A terminal
|
||||
cannot name its tenant.
|
||||
|
||||
## Mapping decisions
|
||||
|
||||
Worth knowing before the first bill lands.
|
||||
|
||||
- **Idempotency** is a unique index on `pos_orders.terminalorderid` — the UUID
|
||||
minted at the till — plus a Postgres advisory lock held for the life of the
|
||||
transaction, so a redelivery arriving concurrently waits and then sees the
|
||||
committed row rather than racing past the check.
|
||||
- **`businessdate` is the day the sale was rung**, not the day it arrived. A
|
||||
till that was offline overnight uploads yesterday's bills this morning, and
|
||||
they belong to yesterday. Every daily figure keys on this.
|
||||
- **Line amounts are scaled onto the bill total.** The till sends each line at
|
||||
its pre-apportionment value while the header carries the total after
|
||||
bill-level discounts. Left alone the item rows would sum to the subtotal and
|
||||
every report that adds up lines would disagree with the one reading the
|
||||
header.
|
||||
- **Payment mode** is the largest tender on a split bill; the full split is
|
||||
kept verbatim in `paymentsjson` for drawer reconciliation.
|
||||
- **Fractional quantities round *up* for stock.** `productstocks.quantity` is an
|
||||
integer column, so 1.5 kg of onions cannot be recorded exactly. Rounding up
|
||||
never under-deducts, so recorded stock is never higher than the shelf. The app
|
||||
order path truncates instead (1.5 → 1), which under-deducts; that behaviour was
|
||||
left untouched rather than silently changed for live traffic. **Making the
|
||||
column numeric is the real fix.**
|
||||
- **Customers** match on `contactno` within the outlet's `applocationid`, so a
|
||||
shopper registered at a till and one who installed the app become one row.
|
||||
Registrations are **insert-if-absent** — never an update, so a profile
|
||||
corrected at head office is not reverted by a terminal replaying an old
|
||||
capture.
|
||||
- **Catalogue** answers `is_delta: false` and is therefore a full snapshot. The
|
||||
terminal withdraws every product a snapshot omits, so this must stay true
|
||||
while the query returns everything stocked at the outlet.
|
||||
- **Barcodes** come from `products.productsku` — there is no barcode column.
|
||||
Scanning at the till matches on it, so SKUs must be the scannable code for
|
||||
barcode scanning to work.
|
||||
|
||||
## Terminal health
|
||||
|
||||
Every till publishes a heartbeat to `nearle/pos/{loc}/{terminal}/health` every
|
||||
**30 seconds**. It is stored in Redis, never in Postgres.
|
||||
|
||||
| Env | Purpose |
|
||||
|---|---|
|
||||
| `REDIS_HOST` / `REDIS_PORT` | **Unset disables presence.** Point at the same Redis the express backend uses |
|
||||
| `REDIS_USER` / `REDIS_PASSWORD` / `REDIS_DB` | Defaults `default`, empty, `0` |
|
||||
|
||||
```
|
||||
pos:terminal:{terminalcode} HASH, TTL 90s
|
||||
pos:location:{locationid}:terminals SET, no TTL
|
||||
```
|
||||
|
||||
The TTL is the whole design. A heartbeat is a fact with an expiry date: a till
|
||||
that loses power stops refreshing, the key expires, and it disappears from the
|
||||
board with nothing having to notice. In Postgres this would need ~288,000 writes
|
||||
a day across a hundred tills *and* a reaper job, because a row saying "online"
|
||||
cannot age out by itself.
|
||||
|
||||
90 seconds is three missed beats. Two would make an ordinary GPRS hiccup look
|
||||
like a dead till; five would take two and a half minutes to notice a real one.
|
||||
|
||||
The set has **no TTL**, mirroring `city:{tenantid}:active_deliveries` in the
|
||||
express backend: it is an index of what exists, not a claim that any of it is
|
||||
alive. Membership means "this till has been seen here"; liveness is whether the
|
||||
hash still exists.
|
||||
|
||||
Keys are namespaced `pos:*` and do not collide with express's `delivery:*`,
|
||||
`city:*` or `rider_*`. **Worth keeping that way** — a shared datastore only stays
|
||||
safe while each writer's keys are obviously its own.
|
||||
|
||||
A heartbeat is **never acknowledged**. Presence is fire-and-forget: a till that
|
||||
stopped selling because a dashboard was busy would be a self-inflicted outage.
|
||||
|
||||
Read it back:
|
||||
|
||||
| Method | Path |
|
||||
|---|---|
|
||||
| `GET` | `/live/api/v1/pos/health/terminal?terminal_id=T4A9` |
|
||||
| `GET` | `/live/api/v1/pos/health/location?location_id=12` |
|
||||
|
||||
A till whose key has expired comes back marked `offline` rather than being
|
||||
omitted — omitting it would make a dead terminal indistinguishable from one that
|
||||
was never installed, and the dead one is exactly what somebody is looking for.
|
||||
|
||||
What a heartbeat carries: identity and app version; **queue depth**
|
||||
(`pending_bills`, `pending_registrations`, `oldest_pending_at`) — the numbers
|
||||
that make a silent sync failure visible; **today's trading** (`today_bills`,
|
||||
`today_amount`, `last_bill_at`) — a till that is connected but has rung nothing
|
||||
in three hours is usually a jammed printer or an absent cashier; and device
|
||||
state.
|
||||
|
||||
## Not built
|
||||
|
||||
- **Catalogue deltas.** Every pull is a full snapshot. Fine for a few hundred
|
||||
products, worth revisiting at a few thousand.
|
||||
- **Loyalty coming back down.** The uplink deliberately carries no points or
|
||||
spend — those belong to the bill stream, which is idempotent and sees every
|
||||
counter. Nothing yet computes them centrally and sends them to the tills, so
|
||||
a shopper's balance at a till is that till's view.
|
||||
- **Device authentication.** A terminal is trusted with a locationid. Signed
|
||||
device tokens are the obvious next step before this is exposed publicly.
|
||||
- **Battery and free storage in the heartbeat.** The reporter has a hook for
|
||||
them, but this build collects neither — they need platform packages a desktop
|
||||
build has no use for. Fields that are not collected are **omitted**, not sent
|
||||
as zero: a board showing every till at 0% battery is worse than one showing
|
||||
nothing.
|
||||
|
||||
## Broker hardening — before a hundred tills join
|
||||
|
||||
Measured on the live broker, not assumed. None of this is caused by the POS
|
||||
work; all of it gets worse the moment bills start flowing.
|
||||
|
||||
**There is no ACL file.** `allow_anonymous false` is set and auth is by password
|
||||
file, but with no `acl_file` every authenticated user is unrestricted on every
|
||||
topic. The rider app ships `admin` credentials **hardcoded in its APK**, so
|
||||
anyone who decompiles it today has full publish and subscribe over `nearle/#`
|
||||
*and* `doormile/#` — a second project's traffic. Adding POS puts every shop's
|
||||
takings behind the same credential.
|
||||
|
||||
A scoped account is two commands and a container restart:
|
||||
|
||||
```bash
|
||||
# A user for the tills, and one for this backend.
|
||||
mosquitto_passwd -b /mosquitto/config/passwd pos_terminal '<strong-unique-pw>'
|
||||
mosquitto_passwd -b /mosquitto/config/passwd pos_ingest '<different-pw>'
|
||||
```
|
||||
|
||||
```conf
|
||||
# /mosquitto/config/acl — then add `acl_file /mosquitto/config/acl` to mosquitto.conf
|
||||
|
||||
# Tills: publish their own traffic, read only their own acks and their shop's
|
||||
# catalogue. The %c substitution binds a client to its own topics, so one till
|
||||
# cannot read another's.
|
||||
user pos_terminal
|
||||
topic write nearle/pos/+/+/order
|
||||
topic write nearle/pos/+/+/customer
|
||||
topic write nearle/pos/+/+/status
|
||||
topic write nearle/pos/+/+/health
|
||||
topic read nearle/pos/+/+/ack
|
||||
topic read nearle/pos/+/+/command
|
||||
topic read nearle/pos/+/catalogue
|
||||
|
||||
# This backend: the mirror image.
|
||||
user pos_ingest
|
||||
topic read nearle/pos/+/+/order
|
||||
topic read nearle/pos/+/+/customer
|
||||
topic read nearle/pos/+/+/health
|
||||
topic read nearle/pos/+/+/status
|
||||
topic write nearle/pos/+/+/ack
|
||||
topic write nearle/pos/+/+/command
|
||||
topic write nearle/pos/+/catalogue
|
||||
|
||||
# Existing projects, scoped to what they already use.
|
||||
user admin
|
||||
topic readwrite nearle/riders/#
|
||||
topic readwrite doormile/#
|
||||
```
|
||||
|
||||
Tighten `pos_terminal` further with per-terminal credentials if you want one
|
||||
till unable to read another's acks at all; the pattern above trusts tills within
|
||||
the fleet but not outside it.
|
||||
|
||||
**There is no TLS.** Port 8883 is not configured and is closed. Rider GPS
|
||||
travels in the clear today; POS bills carry customer names and mobile numbers,
|
||||
which is a different category of exposure on a shared network. Adding a listener
|
||||
means certs plus republishing the port, i.e. recreating the container — worth
|
||||
doing before rollout rather than after.
|
||||
|
||||
**Two more, from the audit:**
|
||||
|
||||
- The broker password and the workolik NATS password differ only in
|
||||
capitalisation. Diverge them when creating the scoped users.
|
||||
- Confirm on the host whether the broker was started from the compose file or
|
||||
from a bare `docker run` before editing the compose file and expecting it to
|
||||
take effect — there is precedent in this estate for compose existing but not
|
||||
being the deploy path.
|
||||
|
||||
## Capacity
|
||||
|
||||
Current load, measured: **~0.9 msg/s inbound**, 5 connected clients, 112
|
||||
retained messages totalling 8 KB.
|
||||
|
||||
A hundred tills add roughly 3.3 msg/s steady (a 1 KB heartbeat each per 30s)
|
||||
plus bursts of up to ~50 KB when a sale batch goes up. That is 3–4× current
|
||||
traffic and well within what Mosquitto handles on any VPS. The broker will not
|
||||
be the bottleneck; Postgres write throughput on bill ingest is the thing to
|
||||
watch instead.
|
||||
Reference in New Issue
Block a user