paho delivers on one goroutine, so bills were committed strictly one
after another. Each is a full Postgres transaction — advisory lock,
dedup, stock row locks, availability check, four inserts, commit — which
is 10-30ms, so the ceiling was roughly 30-100 bills a second and a
shop's backlog draining after an outage took minutes to land.
A fixed pool behind a bounded queue, rather than a goroutine per
message. Unbounded concurrency would open a transaction per message and
exhaust the connection pool under a storm, stalling every one of them at
once — a slow minute turned into a dead one. When the queue fills,
submit blocks: paho stops acknowledging, the broker's in-flight window
fills, it stops sending, and the backpressure reaches the till, which
holds its bills and retries. Slow, but nothing is dropped.
Heartbeats get their own pool. Sharing one would let a backlog of bills
delay presence, so every till would appear to go dark at exactly the
moment the system was busiest — the worst time to be blind to which
counters are alive.
Payloads are copied on the way in. paho reuses its buffer once a handler
returns and the work now happens after that, so a queued bill would
otherwise be read as whatever message arrived next — silently, and as
valid JSON often enough to commit the wrong sale.
One bug found by its own test: submit-after-stop selected between a
done-channel and the job channel, and once both were ready Go picks at
random. Picking the send panics on a closed channel. It would have shown
up in production as an occasional crash during shutdown and nowhere
else. Now guarded by an RWMutex held across the send, so the queue
cannot be closed under one in progress.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deployed as a StatefulSet with three replicas, and MQTT has no queue
groups — every subscriber receives every message. All three pods would
commit the same bill and publish three acks. Nothing double-counts,
because the ingest deduplicates on the till's UUID and holds an advisory
lock, but it is three times the database work and three times the
traffic for one sale.
Ordinal 0 consumes; the others stay idle. A StatefulSet already
guarantees stable unique ordinals, so this is a deterministic election
with no lock, no lease and no new dependency. If that pod dies the set
recreates it and tills hold their bills meanwhile, which is what they
are built to do.
POS_MQTT_CONSUMER=always/never overrides it for deployments that are not
a StatefulSet. Anything without an ordinal name — a Deployment pod, a
bare container, local development — consumes, because a lone instance
that silently refused to would be a far more confusing failure than one
that did.
The client id now defaults to the pod name rather than a constant. Two
connections sharing an id evict each other in a reconnect loop that
looks exactly like a flapping network, and takes a while to recognise as
anything else.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every pull was a full snapshot, so a shop with a thousand products
re-sent all of them to correct one price. The response now carries a
revision the terminal stores and hands back, and a pull that supplies
one gets only what moved: the product row, its row at that outlet, or
its stock ledger. Stock is included because a shop's count drifts from a
till's on every sale rung at another counter, and a delta that ignored
it would let that drift persist until someone forced a full pull.
The dangerous part is the flag, not the filter. A response marked
is_delta:false tells the terminal to withdraw every product it does not
mention — so a filtered result carrying that label empties the shelf.
Both are now derived from one value, and there is no path through the
function that filters without also setting the flag.
Everything ambiguous resolves toward the snapshot. A revision that is
malformed, empty, or issued to another outlet yields a zero cutoff and a
complete response; the opposite would leave a terminal permanently
missing changes with nothing to show for it. The revision advances only
on the final page, so a terminal that abandons a paginated pull cannot
end up holding one that claims it saw pages it never received. And the
stamp is taken a second in the past, because a product written during
the same second the query ran would otherwise fall on the wrong side of
the next cutoff and be skipped for good.
A delta still cannot withdraw a deleted product — removing a row from
productlocations leaves no tombstone — so a periodic pull without a
revision is what collects those.
Verified against the live outlet: a full pull of 12, a delta returning
only the one product whose price had changed, and pagination that stays
exact now that productid <= 0 is excluded in SQL rather than after the
LIMIT.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The ingest only ever wrote. A bill that reached pos_orders was safe and
completely unreachable — no screen in the product could show it, and the
only way to see a day's counter takings was to query the database by
hand.
Three endpoints: a paged bill list, one bill with its lines, and a
summary split the three ways somebody actually asks for — by tender for
reconciling a drawer, by day for a chart, by till for an outlet running
several counters.
locationid is required on all of them and is the authorisation boundary,
so a caller cannot page through another shop's takings by omitting a
parameter. Fetching a bill under the wrong outlet returns 404 even when
the reference is a real one.
Dates match businessdate rather than arrival, because a till that was
offline overnight uploads yesterday's bills this morning and they belong
to yesterday. The list is ordered by billedat for the same reason —
sorting by arrival would interleave a recovered backlog through today.
Unlike the ingest handlers these answer in the usual envelope: they are
read by the web app, not by a terminal, and nothing about them is bound
to the till's contract.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two scoped users, pos_terminal and pos_ingest, with an ACL that keeps a
till to its own topics: it can publish its bills, registrations and
heartbeats, read its own acks, and nothing else. It cannot reach
nearle/riders/# or doormile/#, and cannot forge an ack — only the ingest
writes those.
admin is deliberately left unrestricted. Its credentials are compiled
into the rider app, so narrowing it would cut off the live fleet without
warning; that change needs someone to confirm nothing else uses it
first. Because admin's entry grants everything, applying the ACL changed
nothing for existing traffic — verified by watching riders 852 and 1114
keep publishing battery, speed and periodic logs throughout.
The scoping was verified by publishing as pos_terminal to four topics
and observing which arrived: the order did, the rider topic, the
doormile topic and its own ack topic did not.
Config and password file were backed up first; the rollback is one cp
and a container restart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It has been in the repository since the initial commit carrying the live
database host, user and password. Removing it from the index stops that
getting worse; the credentials are still in history and should be
rotated, which needs coordinating with everything that reads them.
Deployments should pass configuration as container environment rather
than shipping a file — a file on disk is one `git add -f` away from
being committed again.
Also closes the last untested path: a shopper registration published
over the broker rather than posted over HTTP. All three MQTT topics —
order, customer and health — have now been fired against the live
Mosquitto instance and acknowledged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A till holds every bill in its own SQLite database and keeps it for
seven days after we acknowledge it, marking one synced only when its id
comes back in an ack. Everything here follows from that.
Silence is not acceptance, so a failing ingest publishes nothing at all
and the terminal simply sends again. A duplicate is a success, because
at-least-once delivery means a lost ack legitimately re-delivers bills
we already hold, and calling those failures would strand a day of
takings on the till. Deduplication is a unique index on the terminal's
UUID plus an advisory lock held for the transaction.
Bills land in pos_orders / pos_order_items rather than orders: a counter
bill carries a cashier, a terminal, a rounding adjustment, promos,
loyalty movement and a payment split that orders has nowhere to put, and
forcing one into the other loses whatever does not fit. Stock is *not*
split — a counter sale writes the same productstocks rows an app order
does, through helpers extracted from createOrderTx so the rule that
prevents overselling has one implementation rather than two.
GetRevenueSummary and GetSalesSummary were extended to union the new
table in; any new report has to remember the same.
Terminal health goes to Redis under a 90-second TTL, sharing the
instance the express backend uses. A heartbeat is a fact with an expiry
date: a till that loses power stops refreshing and ages off the board by
itself, where a Postgres row would need ~288k writes a day and a reaper.
Proven end to end against the live estate before commit: a bill over
HTTP and one over the real Mosquitto broker, the same bill three times
producing one row and one stock movement, and a heartbeat arriving on
the health endpoint. All probe data was removed afterwards.
Four things that only surfaced against real data. An unset jsonb column
failed the very first bill. Product SKUs are unusable as barcodes — 6,245
products share 93 SKUs and "1" covers 5,794 of them — against the till's
unique index, so barcodes fall back to the product id. A taxpercent of
-1 exists and would have put negative GST in a filed slab. And a product
with id 0 exists, which can never be billed and is now skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/utils/notifyuser has been commented out since the initial commit, so every
rider push the admin consoles have ever sent returned 404. Riders were
assigned deliveries and never told, and the failure surfaced as a generic
"notification failed" that read like a transient network fault.
Nothing else was missing. The handler, the FcmNotification model, the
Firebase service account and the Dockerfile line that copies that account
into the image were all already in place — only the route registration was
absent, which is why the gap survived this long.
Verified against Google: with the route registered, FCM authenticates the
service account and returns a specific rejection for a deliberately invalid
token rather than a 404. No push was sent to a real rider, so the final hop
to a device is still unproven.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The offline-sales import required one workbook per outlet and a store
picked in the UI. A merchant running several branches had to download,
fill and upload a file per branch, and the picker defaulted to the
tenant's first outlet — so an admin who never touched it silently
credited the wrong store, which no validation could catch because the
file and the selection agreed with each other.
One workbook now covers every branch. getsaletemplate takes locationid=0
(the default) to span the tenant, stamping tenantid, locationid and the
store name onto every row, and that row's locationid is what decides
which branch a sale is deducted from. The INNER JOIN on tenantlocations
confines it to outlets the tenant owns, so a template can never disclose
another merchant's catalogue.
uploadofflinesales accordingly takes locationid on each bill. The
locationid on the request itself becomes a scope constraint rather than
a destination: left at 0 the bills go where their rows say, and set to a
branch it pins the upload there and refuses anything else. That is what
holds a store user to their own store — the pin comes from their session,
so editing the locationid column in the spreadsheet changes nothing.
Every branch referenced is checked against the tenant regardless.
Branch context and catalogue are resolved once per branch and reused; a
workbook covering six outlets would otherwise re-run both queries for
every bill in it.
Duplicate detection is now per branch. Bill numbers only have to be
unique within a store, since counter books at different outlets
routinely restart numbering at 1, and treating a shared number as a
repeat would have silently dropped a real sale.
Verified against tenant 1087, whose two branches both stock product
6998 at 100 units: a single upload of two bills moved 1097 to 97 and
1135 to 95 independently; the same bill number at both branches imported
as two separate orders; an upload pinned to 1097 imported its own bill
and refused the 1135 one; a row naming another tenant's outlet was
refused; and re-uploading the file deducted nothing. All five test
orders were cancelled afterwards and both branches confirmed back at 100.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Order ids were duplicating in production: 160 distinct (tenant, orderid) pairs
are shared by more than one order, worst of them "1135-1" on 108 orders, and
every order tenant 1147 has ever placed is numbered "1147-1".
getSequenceno read MAX(seqno)+1 and updateSeqno incremented, both against
r.db rather than the order's transaction and separated by the whole order
insert. Two concurrent orders therefore read the same number before either
wrote, and an order that rolled back still consumed one. Three further
defects made it worse:
- A NULL orderseqno made COALESCE(MAX(orderseqno) + 1, 1) evaluate
NULL + 1 = NULL and fall through to a hardcoded "<tenantid>-1". The
increment then computed NULL + 1 = NULL too, so the counter could never
leave NULL and every subsequent order reused that same id.
- Tenants with several ordersequences rows (tenant 1135 has ~25) hit a
GROUP BY returning multiple rows, of which Scan kept the first
arbitrarily, while the increment updated all of them.
- A tenant with no row at all fell back to "<tenantid>-1" indefinitely,
because nothing ever created one.
nextSequenceNo replaces both functions with a single UPDATE ... RETURNING run
inside the caller's transaction, so the counter row stays locked until the
order commits and concurrent orders queue rather than collide. A NULL seeds
from the tenant's existing order count — at least as high as any number
already issued, so recovery cannot reissue a used id — the counter is pinned
to the tenant's lowest sequenceid so reads and writes address one row, and a
missing row is created on first use.
Verified against production data in rolled-back transactions: tenant 1147
(NULL) now yields 1147-9, 1147-10, ...; tenant 1135 (NULL plus duplicate rows)
1135-356 onward; tenant 916 keeps its 916-2024115209 subprefix format; an
unknown tenant creates its row and starts at 1. Eight concurrent allocations
produced eight distinct ids. Two real orders through the API returned 1147-9
and 1147-10, then were cancelled with stock restoring to its baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
products.productstatus is a per-product lifecycle field holding
"Active"/"Inactive". CreateProductStock overwrote it with "available" on every
stock receipt — an availability value written into a lifecycle column — which
destroyed the real lifecycle state of the rows it touched. 136 products now
read "available" and 12 "outofstock" with no way to recover what they were.
A single column on products cannot express availability anyway: the same
product can be stocked at one outlet and empty at another. That fact belongs
to productlocations.status, which SyncProductLocationStatus already derives
from the ledger, so the receipt path now updates only that and leaves
productstatus alone. UpdateProductStatus remains available as an explicit
admin operation; it is simply no longer called as a side effect of stock
movement.
GetProductCount counted available/outofstock off the same corrupted column and
returned near-nonsense as a result: across 6245 products it matched
'available' on 136 and 'outofstock' on 12, leaving 6097 — the real answer —
uncounted under "Active". It now derives both from the ledger, counting a
product available when it holds positive stock at any of the tenant's outlets,
so total = available + outofstock (6245 = 22 + 6223).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stock shown in the console did not match the productstocks ledger, and two
product endpoints were failing outright. Every cause was on the read side or
in how the availability flag was maintained; the ledger writes themselves
(CreateOrder's "out" entry, cancellation's "in" entry) were already correct.
Read fixes, repositories/productRepository.go:
- GetProductStocks returned SQLSTATE 42803 on every call: bare a.tenantid /
a.stocktype / a.status under GROUP BY a.productid. The per-ledger-row
columns are now aggregated and the grouping covers the identity columns.
- FetchFilteredProducts filtered on an alias `e` that no query defines, so
every /getallproducts call carrying a locationid failed with SQLSTATE 42P01
instead of returning products.
- FetchFilteredProducts joined productlocations on productid alone and joined
a (productid, locationid)-grouped stock subquery on productid alone, so a
product carried by three outlets came back three times, each row showing
another outlet's quantity and status. Both are now tenant-scoped subqueries
collapsed to one row per product and scoped to the outlet when one is given.
- GetProductStocks and FetchFilteredProducts compared stocktype = 'in'
case-sensitively. Production holds 'in' and 'IN' both, so uppercase receipts
were silently dropped from the balance: one outlet reported 0 for a product
holding 50, another reported 0 for twelve products holding 200-840.
- GetStockStatement summed opening over stockdate <= CURRENT_DATE, making it
arithmetically identical to closing. The Inventory ledger showed the same
number in both columns on every row, which reads as stock never moving.
Availability flag:
productlocations.status was maintained by two different rules — the order path
derived it from the balance, the receiving path set 'available' on any "in"
entry regardless of the resulting balance. A partial restock that left the
balance at or below zero marked a product sellable, and a flag set by an old
order never cleared for stock that arrived by a route the API did not own.
Both paths now derive the flag from the live balance through one rule:
SyncProductLocationStatus (receiving side) and syncProductLocationStatus
(order side, inside the caller's transaction). ReactivateProductLocations is
replaced by the former; the service no longer filters refs by stocktype, since
the direction of the movement is no longer what decides the flag. A row that
has already drifted now repairs itself on its next ledger entry.
Verified against the live database: all four stock endpoints return matching
balances, /getallproducts no longer duplicates rows, and the flag sync was
exercised in both directions inside a rolled-back transaction.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The order-placement stock check (read available qty, then insert an "out"
deduction) had no row lock, so two concurrent orders for the same product
could both pass the availability check before either committed its
deduction, overselling the item. Locks each ordered product's
productlocations row with SELECT ... FOR UPDATE up front, in a fixed
(productid, locationid) order across all items so overlapping concurrent
orders contend for locks in the same sequence instead of deadlocking.
The mobile app needs an "All" tile to browse every product regardless of
category. Rather than inserting a real app_category row (which would break
once any category-scoped product filter treats it as an actual, empty
category), the service now prepends a synthetic entry with categoryid=0 —
reusing the "0 = no category filter" convention GetAllProducts already
implements in FetchFilteredProducts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The store/branch QR code the app scans is just {tenantid, locationid} JSON, but the
onboarding response previously discarded the DB-assigned locationid, so the frontend
had no way to render a store's QR right after onboarding without a separate lookup.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two gaps found while auditing the order/stock flow:
- Receiving stock via an approved stock request only updated
products.productstatus (a global per-product flag). A location
flagged outofstock by CreateOrder never got reactivated, since
nothing touched productlocations.status on the way back in.
CreateProductStock now reactivates the specific
(tenant, location, product) row to "available" for every "in"
entry, the counterpart to how it gets flagged out.
- getproductbyvariant returned no stock info at all, so the app could
only find out a product was unavailable from the 409 at order time.
It now accepts an optional locationid and, when passed, returns
live productstock (same SUM(in)-SUM(out) formula the order check
uses) and locationstatus per product. Omitting locationid keeps the
old response shape.
Added MOBILE_ORDER_VERIFICATION.md as a handoff doc for the mobile
team covering the expected request/response shapes and how to verify
their integration.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every order created after orderheaderid 146119 had zero rows in
orderdetails — the header saved fine but items never made it in,
which also meant the stock pre-validation loop (it iterates over
data.Items) never ran, so an order could go through without ever
checking stock. Root cause: whichever client is sending these sends
items as a sibling of "orders" rather than nested inside it, a shape
neither existing parse strategy captures, so encoding/json silently
dropped it.
Add a third parse fallback for that sibling-items shape, and reject
any order with zero items outright instead of letting it through as
a phantom header-only row.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CreateTenantLocation hardcoded the new location and its auto-spawned
manager login to InActive regardless of what the caller sent (the
frontend already sends Active). AppLogin checks account status before
checking whether a password is set, so a new store's login could
never reach the password-setup screen — permanently stuck on
"Inactive Account. Contact admin."
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CreateTenantUser copied the tenant form's fields onto the new
app_users row via copier.Copy, but the onboarding form never sends a
configid, so it defaulted to 0. AppLogin's GetUserByAuthname always
queries configid=1 for the web login, so any tenant onboarded through
this path was permanently unable to log in by email ("Email not
found") no matter what was typed. Set it explicitly, same as roleid.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PUT /users/update doubles as the password-setup/reset call (userid +
password only) for the new frontend create-password flow, and had no
validation on that field at all.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
issuperadmin on app_users (schema change already applied to the DB)
is now returned by AppLogin/TenantWebLogin's login response, gated
server-side rather than via a client-supplied roleid. Fixes the login
enrichment query (GetTenantUserById) to LEFT JOIN app_location instead
of INNER JOIN — a tenantless super-admin row was previously silently
dropped, so the login "succeeded" but returned an empty struct.
Also exposes createtenantuser on the web route group (was mob-only).
No repository changes were needed for location auto-provisioning:
GORM was already inserting the nested tenantlocations association on
tenant creation, it just wasn't reachable from the web.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
subcategoryid is no longer required to import a catalogue product —
it's a display/grouping hint elsewhere (the codebase already has an
"Uncategorized" fallback for subcategoryid=0), so requiring it was
pure friction with no correctness payoff.
Fixes the category picker at the root: categoryid 2, which tenant
1135's real products actually use, has no row in productcategories at
all (not a filter bug — the master data is genuinely missing it).
Rather than inventing category master data, adds
GET /products/gettenantcategories, which lists categories a tenant's
own products actually use (falling back to a synthesized label when
the master table has no name), so the import category picker always
offers something real instead of an incomplete global list.
Also relaxes the subcategory lookup's tenant filter to include
unowned/global rows (tenantid NULL or 0), not just exact tenant
matches — categoryid 2's real subcategories carry no tenant at all,
so the strict filter was hiding them even when a tenant wanted one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds a separate CatalogueDB (pgvector) connection alongside the main
nearledb, plus a new catalogue module (repository/service/controller/
routes) to browse it by brand, category, and keyword, with brand
optional so the whole ~237-product catalogue can be browsed unfiltered.
Adds the actual bridge: importing a catalogue product snapshots it into
the tenant's own products table (keyed on brand+catalogueid, since a
catalogue row's bare id is only unique within its own brand table),
then links it via the existing productlocations upsert. Re-importing
tops up stock and refreshes price instead of duplicating. Also adds an
imported-refs endpoint so the frontend can badge already-imported items
without diffing full product lists, and wires the new AWS S3 image
store used to resolve catalogue product photos.
Bumps Go/Docker to 1.24 for the AWS SDK dependency this needs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
