nearle_daily/backend_fiesta
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Ingest counter sales from the POS terminals, over MQTT and HTTP

Browse Source 
A till holds every bill in its own SQLite database and keeps it for
seven days after we acknowledge it, marking one synced only when its id
comes back in an ack. Everything here follows from that.

Silence is not acceptance, so a failing ingest publishes nothing at all
and the terminal simply sends again. A duplicate is a success, because
at-least-once delivery means a lost ack legitimately re-delivers bills
we already hold, and calling those failures would strand a day of
takings on the till. Deduplication is a unique index on the terminal's
UUID plus an advisory lock held for the transaction.

Bills land in pos_orders / pos_order_items rather than orders: a counter
bill carries a cashier, a terminal, a rounding adjustment, promos,
loyalty movement and a payment split that orders has nowhere to put, and
forcing one into the other loses whatever does not fit. Stock is *not*
split — a counter sale writes the same productstocks rows an app order
does, through helpers extracted from createOrderTx so the rule that
prevents overselling has one implementation rather than two.
GetRevenueSummary and GetSalesSummary were extended to union the new
table in; any new report has to remember the same.

Terminal health goes to Redis under a 90-second TTL, sharing the
instance the express backend uses. A heartbeat is a fact with an expiry
date: a till that loses power stops refreshing and ages off the board by
itself, where a Postgres row would need ~288k writes a day and a reaper.

Proven end to end against the live estate before commit: a bill over
HTTP and one over the real Mosquitto broker, the same bill three times
producing one row and one stock movement, and a heartbeat arriving on
the health endpoint. All probe data was removed afterwards.

Four things that only surfaced against real data. An unset jsonb column
failed the very first bill. Product SKUs are unusable as barcodes — 6,245
products share 93 SKUs and "1" covers 5,794 of them — against the till's
unique index, so barcodes fall back to the product id. A taxpercent of
-1 exists and would have put negative GST in a filed slab. And a product
with id 0 exists, which can never be billed and is now skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Suriya
2026-08-03 17:48:00 +05:30
parent 583cd89063
commit e3459a0f1c
23 changed files with 3679 additions and 171 deletions
Download Patch File Download Diff File Expand all files Collapse all files

212
models/pos.go Normal file
View File

@@ -0,0 +1,212 @@
package models
// Wire format for the Nearle POS terminal.
//
// These types mirror what the till actually publishes, field for field. The
// terminal is the fixed side of this contract: it is installed on a hundred
// machines that cannot all be updated at once, so the names here follow its
// JSON rather than this codebase's usual Go casing.
//
// The authoritative description lives in the terminal repository at
// docs/sync-contract.md.
// PosOrderItem is one line of a counter bill.
//
// Productid arrives as a string because the till stores catalogue ids as text.
// It carries the numeric products.productid this backend issued during a
// catalogue pull, so it parses back to an int on arrival.
type PosOrderItem struct {
Productid string `json:"product_id"`
Barcode string `json:"barcode"`
Name string `json:"name"`
Quantity float64 `json:"quantity"`
Unitprice float64 `json:"unit_price"`
Discount float64 `json:"discount"`
Gstrate float64 `json:"gst_rate"`
Tax float64 `json:"tax"`
Linetotal float64 `json:"line_total"`
}
// PosOrderCustomer is the shopper snapshot carried on the bill itself.
//
// Deliberately thin. The full profile travels on its own uplink; this exists so
// a bill can be attached to somebody even when their registration has not
// arrived yet.
type PosOrderCustomer struct {
Id string `json:"id"`
Mobile string `json:"mobile"`
Name string `json:"name"`
}
// PosOrderPayment is one tender against a bill. A bill may be split across
// several.
type PosOrderPayment struct {
Method string `json:"method"`
Amount float64 `json:"amount"`
Reference string `json:"reference"`
}
// PosOrderPromo records a campaign that fired, as an amount rather than a rule.
// A bill read back years later must show what was actually given, not what
// today's rules would give.
type PosOrderPromo struct {
Id string `json:"id"`
Name string `json:"name"`
Type string `json:"type"`
Amount float64 `json:"amount"`
}
// PosOrder is one completed sale.
//
// Id is a UUID minted at the till and is the only thing that identifies this
// bill. It is what deduplication keys on, because at-least-once delivery means
// the same bill legitimately arrives more than once.
type PosOrder struct {
Id string `json:"id"`
Invoicenumber string `json:"invoice_number"`
Createdat string `json:"created_at"`
Terminalid string `json:"terminal_id"`
Cashier string `json:"cashier"`
Customer *PosOrderCustomer `json:"customer"`
Subtotal float64 `json:"subtotal"`
Discount float64 `json:"discount"`
Promos []PosOrderPromo `json:"promos"`
Tax float64 `json:"tax"`
Roundoff float64 `json:"round_off"`
Total float64 `json:"total"`
Pointsearned int `json:"points_earned"`
Pointsredeemed int `json:"points_redeemed"`
Payments []PosOrderPayment `json:"payments"`
Items []PosOrderItem `json:"items"`
// GST per slab, as printed on the tax invoice: {"0.05": 12.30, "0.18": 4.50}.
// Absent from terminals built before this field existed, which is why every
// consumer of it has to tolerate an empty map.
Taxbreakdown map[string]float64 `json:"tax_breakdown"`
}
// PosOrderBatch is the envelope a terminal publishes.
//
// Storeid carries the numeric tenantlocations.locationid as a string. The
// tenant is resolved from it server-side and never taken from the terminal — a
// till must not be able to name the tenant it posts into.
type PosOrderBatch struct {
Schema int `json:"schema"`
Batchid string `json:"batch_id"`
Storeid string `json:"store_id"`
Terminalid string `json:"terminal_id"`
Sentat string `json:"sent_at"`
Orders []PosOrder `json:"orders"`
}
// PosCustomer is a shopper registered at a till.
//
// No loyalty figures. Points, lifetime spend and visit counts are derived from
// the bill stream, which is idempotent and sees every counter; accepting a
// terminal's local balance would make the last till to sync win.
type PosCustomer struct {
Id string `json:"id"`
Mobile string `json:"mobile"`
Name string `json:"name"`
Email string `json:"email"`
Gender string `json:"gender"`
Dateofbirth string `json:"date_of_birth"`
Registeredat string `json:"registered_at"`
Registeredbyterminal string `json:"registered_by_terminal"`
}
type PosCustomerBatch struct {
Schema int `json:"schema"`
Batchid string `json:"batch_id"`
Storeid string `json:"store_id"`
Terminalid string `json:"terminal_id"`
Sentat string `json:"sent_at"`
Customers []PosCustomer `json:"customers"`
}
// PosAck is the only thing that retires a bill on the terminal.
//
// The rule the whole design rests on: a till marks a record synced if and only
// if its id appears in Accepted. Silence is not acceptance — an empty ack, a
// dropped connection or a 200 with no body all leave the record pending and it
// is sent again.
//
// Naming an id in Rejected is a decision, not a fault: the terminal stops
// retrying that record and waits for a person. Use it for "this bill is
// malformed", never for "the database is having a bad minute" — for the latter,
// do not ack at all and let the till back off and retry.
type PosAck struct {
Batchid string `json:"batch_id"`
Accepted []string `json:"accepted"`
Rejected map[string]string `json:"rejected,omitempty"`
}
// NewPosAck returns an ack with non-nil members, so it serialises as `[]` and
// `{}` rather than `null`. A terminal reading null for accepted would treat the
// whole batch as unconfirmed.
func NewPosAck(batchID string) *PosAck {
return &PosAck{
Batchid: batchID,
Accepted: make([]string, 0),
Rejected: make(map[string]string),
}
}
func (a *PosAck) Accept(id string) {
a.Accepted = append(a.Accepted, id)
}
func (a *PosAck) Reject(id, reason string) {
a.Rejected[id] = reason
}
// PosCatalogueProduct is one product as the till stores it.
type PosCatalogueProduct struct {
Id string `json:"id"`
Name string `json:"name"`
Barcode string `json:"barcode"`
Sku string `json:"sku"`
Category string `json:"category"`
Price float64 `json:"price"`
Mrp float64 `json:"mrp,omitempty"`
Stock float64 `json:"stock"`
Unit string `json:"unit"`
Gstrate float64 `json:"gst_rate"`
Hsncode string `json:"hsn_code,omitempty"`
Brand string `json:"brand,omitempty"`
Isactive bool `json:"is_active"`
}
// PosCatalogueCustomer is a shopper travelling *down* to a terminal.
//
// The mirror of PosCustomer, and the difference is the point: the uplink
// carries no loyalty figures because a till's local balance is only its own
// view, while the downlink carries them because the back office has seen every
// counter and is the only thing that can total them.
type PosCatalogueCustomer struct {
Id string `json:"id"`
Name string `json:"name"`
Mobile string `json:"mobile"`
Email string `json:"email,omitempty"`
Gender string `json:"gender,omitempty"`
Dateofbirth string `json:"date_of_birth,omitempty"`
Loyaltypoints int `json:"loyalty_points"`
Lifetimespend float64 `json:"lifetime_spend"`
Visitcount int `json:"visit_count"`
Createdat string `json:"created_at,omitempty"`
Lastvisitat string `json:"last_visit_at,omitempty"`
}
// PosCatalogueResponse answers a terminal's catalogue pull.
//
// Isdelta is load-bearing. A response marked false is treated as a full
// snapshot and the terminal withdraws every product it does not mention — so
// answering a change set with false empties the shelf.
type PosCatalogueResponse struct {
Revision string `json:"revision"`
Isdelta bool `json:"is_delta"`
Hasmore bool `json:"has_more"`
Products []PosCatalogueProduct `json:"products"`
Customers []PosCatalogueCustomer `json:"customers"`
Retiredids []string `json:"retired_product_ids"`
}

58
models/poshealth.go Normal file
View File

@@ -0,0 +1,58 @@
package models
// PosHealth is what a till reports about itself every 30 seconds.
//
// This is a liveness signal, not a record. It lives in Redis under a TTL and is
// never written to Postgres: a terminal that dies simply stops refreshing and
// disappears from the board on its own, with no reaper job and no row left
// claiming "online" three days after the shop closed.
//
// The fields exist to answer questions a person actually asks when a shop
// phones in: is the till on, is it reaching us, is it selling anything, and is
// the hardware in the way.
type PosHealth struct {
// Identity. Terminalid is the short code printed on invoices — the thing a
// support call starts with.
Terminalid string `json:"terminal_id"`
Locationid string `json:"location_id"`
Storename string `json:"store_name"`
Appversion string `json:"app_version"`
// "online" while the till is refreshing this. The broker's Last Will
// overwrites it with "offline" if the terminal loses power mid-shift, which
// is the only way to tell *closed for the night* from *unplugged*.
Status string `json:"status"`
// Queue depth — the number that matters most. A shop quietly accumulating
// unsynced takings looks completely normal from the shop floor, and this is
// the only thing that makes it visible before someone reconciles a till and
// finds a day missing.
Pendingbills int `json:"pending_bills"`
Pendingregistrations int `json:"pending_registrations"`
Oldestpendingat string `json:"oldest_pending_at"`
// Today's trading. A till that is connected but has rung nothing in three
// hours usually means a jammed printer or an absent cashier, and neither
// shows up in a plain online/offline board.
Todaybills int `json:"today_bills"`
Todayamount float64 `json:"today_amount"`
Lastbillat string `json:"last_bill_at"`
// Device state, for pre-emptive support.
//
// Pointers so that *not reported* is distinguishable from *reported as
// zero*. Not every build collects these — battery and free storage need
// platform packages a desktop till has no use for — and writing an
// uncollected reading as 0 would show a board full of terminals on a flat
// battery with an unreachable printer. A nil field is skipped entirely.
Batterylevel *int `json:"battery_level,omitempty"`
Batterycharging *bool `json:"battery_charging,omitempty"`
Storagefreemb *int `json:"storage_free_mb,omitempty"`
Printerreachable *bool `json:"printer_reachable,omitempty"`
Drawerstatus *string `json:"drawer_status,omitempty"`
// Stamped by the till. The consumer also stamps its own arrival time, and
// the two disagreeing is itself a signal — a till whose clock is wrong
// writes bills under the wrong business date.
Reportedat string `json:"reported_at"`
}

138
models/posorder.go Normal file
View File

@@ -0,0 +1,138 @@
package models
import "time"
// Counter sales, stored at the fidelity the till actually rang them.
//
// Separate from `orders` on purpose. An app order and a counter bill are
// different documents: a bill carries a cashier, a terminal, a rounding
// adjustment, promo campaigns, loyalty movement and a payment split across
// several tenders, none of which `orders` has anywhere to put. Forcing one into
// the other's shape loses whichever fields do not fit, and the loss is silent.
//
// The cost of the split is that existing revenue queries do not see these rows
// until they are extended to union them in — done in orderRepository's summary
// queries, and the thing to remember when adding a new report.
//
// Stock is *not* separate: a counter sale writes the same productstocks "out"
// rows an app order does, through the same helper. Two stock ledgers would mean
// the catalogue pull sends a till figures that ignore its own sales.
// PosOrders is one counter bill.
type PosOrders struct {
Posorderid int `json:"posorderid" gorm:"primaryKey;autoIncrement;column:posorderid"`
// The UUID minted at the till. Globally unique by construction and the only
// thing that identifies this bill, so it carries a unique index: delivery is
// at-least-once and the same bill legitimately arrives more than once.
Terminalorderid string `json:"terminalorderid" gorm:"column:terminalorderid;uniqueIndex;not null"`
// Human-facing, and unique only per terminal — a till that was replaced
// restarts its own series, so gaps are normal and duplicates across
// terminals are expected.
Invoicenumber string `json:"invoicenumber" gorm:"column:invoicenumber;index"`
Tenantid int `json:"tenantid" gorm:"column:tenantid;index"`
Locationid int `json:"locationid" gorm:"column:locationid;index"`
// Which physical till, e.g. "T4A9". Free text: nothing keys on it, but a
// support call starts with it.
Terminalid string `json:"terminalid" gorm:"column:terminalid;index"`
Cashiername string `json:"cashiername" gorm:"column:cashiername"`
// Resolved against the customers table. Zero for a walk-in.
Customerid int `json:"customerid" gorm:"column:customerid;index"`
Customermobile string `json:"customermobile" gorm:"column:customermobile"`
Customername string `json:"customername" gorm:"column:customername"`
// When the sale was rung, not when it reached us — a till that was offline
// for a day uploads bills whose Billedat is yesterday, and every daily
// figure must use this rather than Receivedat.
Billedat time.Time `json:"billedat" gorm:"column:billedat;index"`
// YYYY-MM-DD of Billedat, denormalised so a day's takings are one indexed
// equality match rather than a range scan with timezone arithmetic.
Businessdate string `json:"businessdate" gorm:"column:businessdate;index"`
Subtotal float64 `json:"subtotal" gorm:"column:subtotal"`
Discount float64 `json:"discount" gorm:"column:discount"`
Taxamount float64 `json:"taxamount" gorm:"column:taxamount"`
// The paise adjustment printed on the bill. Kept because total is not
// derivable from the other columns without it.
Roundoff float64 `json:"roundoff" gorm:"column:roundoff"`
// What the shopper actually paid. The figure every revenue report sums.
Total float64 `json:"total" gorm:"column:total"`
Pointsearned int `json:"pointsearned" gorm:"column:pointsearned"`
Pointsredeemed int `json:"pointsredeemed" gorm:"column:pointsredeemed"`
Itemcount int `json:"itemcount" gorm:"column:itemcount"`
// The largest tender, for the common "how did they pay" grouping.
Paymentmode string `json:"paymentmode" gorm:"column:paymentmode;index"`
// The full split, verbatim. A bill can be part cash, part card, part
// loyalty, and collapsing that to one mode would lose the reconciliation a
// cashier settles their drawer against.
Paymentsjson string `json:"paymentsjson" gorm:"column:paymentsjson;type:jsonb"`
// Campaigns that fired, stored as amounts rather than rules — a bill read
// back years later must show what was given, not what today's rules give.
Promosjson string `json:"promosjson" gorm:"column:promosjson;type:jsonb"`
// GST per slab, as printed on the tax invoice.
Taxbreakdownjson string `json:"taxbreakdownjson" gorm:"column:taxbreakdownjson;type:jsonb"`
// Which upload carried this bill, and when it landed. Kept for tracing a
// terminal's complaint back to a specific batch.
Batchid string `json:"batchid" gorm:"column:batchid;index"`
Receivedat time.Time `json:"receivedat" gorm:"column:receivedat"`
Created time.Time `json:"created" gorm:"column:created;autoCreateTime"`
Updated time.Time `json:"updated" gorm:"column:updated;autoUpdateTime"`
Items []PosOrderItems `json:"items" gorm:"-"`
}
func (PosOrders) TableName() string {
return "pos_orders"
}
// PosOrderItems is one line of a counter bill.
type PosOrderItems struct {
Posorderitemid int `json:"posorderitemid" gorm:"primaryKey;autoIncrement;column:posorderitemid"`
Posorderid int `json:"posorderid" gorm:"column:posorderid;index"`
Tenantid int `json:"tenantid" gorm:"column:tenantid;index"`
Locationid int `json:"locationid" gorm:"column:locationid;index"`
Productid int `json:"productid" gorm:"column:productid;index"`
// Snapshotted rather than joined. A product renamed or withdrawn next month
// must not change what a bill from today says it sold.
Productname string `json:"productname" gorm:"column:productname"`
Barcode string `json:"barcode" gorm:"column:barcode"`
Unitname string `json:"unitname" gorm:"column:unitname"`
// Fractional: a counter sells 1.5 kg of onions. Note that productstocks
// cannot represent that — see roundStockQty.
Quantity float64 `json:"quantity" gorm:"column:quantity"`
Unitprice float64 `json:"unitprice" gorm:"column:unitprice"`
Discountamount float64 `json:"discountamount" gorm:"column:discountamount"`
// Stored as a fraction (0.18), matching how the till holds it.
Gstrate float64 `json:"gstrate" gorm:"column:gstrate"`
Taxamount float64 `json:"taxamount" gorm:"column:taxamount"`
// What this line contributed to the bill total, after its share of every
// discount. The lines sum to the bill's Total less Roundoff.
Linetotal float64 `json:"linetotal" gorm:"column:linetotal"`
Created time.Time `json:"created" gorm:"column:created;autoCreateTime"`
}
func (PosOrderItems) TableName() string {
return "pos_order_items"
}