nearle_daily/nearle_pos
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Drain bills to the back office automatically, over MQTT or HTTP

Browse Source 
Turns the orders table into a queue that empties itself. Bills were only
uploaded when a cashier pressed Sync at end of day; a till that was never
pressed held a day's takings indefinitely.

Drain engine (lib/data/sync/sync_engine.dart)
- Triggers on sale committed, network regained, 5-minute poll, head-office
  request, and the manual button.
- Single flight: a busy till firing a trigger per sale would otherwise have
  several passes reading the same pending rows and send every bill twice.
  A trigger arriving mid-drain is queued and replayed, so nothing is dropped.
- Exponential backoff with +/-20% jitter to a 5-minute ceiling. The jitter
  matters: a store's terminals all fail at the same instant when the line
  drops, and would retry in lockstep without it.
- Halts rather than loops on a failure retrying cannot fix (bad credential,
  refused batch). Pressing Sync clears the halt.

Transports (lib/data/remote/)
- OrderTransport interface; MQTT, HTTP and simulated implementations. The
  repository does not know which is in use.
- MQTT: QoS 1 uplink, application-level ACK correlated by batch_id on a return
  topic, retained Last Will for terminal-offline detection, downlink for
  catalogue pushes and remote sync requests.
- A broker PUBACK is never treated as acceptance. It means the broker holds
  the bytes, not that the ledger took the sale. Only ids the back office names
  are marked synced; silence leaves a bill pending.
- HTTP carries a stable idempotency key across retries of the same bills.

Retention
- Accepted bills are kept 7 days instead of deleted, so a batch the back
  office later loses can be re-sent in full. Purged after that; archived
  totals stay forever.
- forBusinessDate now reads pending rows only. A retained bill exists in both
  the orders table and day_archive, and summing both would overstate the day.

Fixes found while building this
- SyncEngine._refreshPending wrote state.copyWith(pending: await ...). Dart
  evaluates the receiver before the awaited argument, so a connectivity drop
  during the wait was silently overwritten by the stale snapshot. Caught by
  the first run of the new engine tests.
- PrinterSettingsController wrote state after four awaits with no mounted
  check, throwing "used after dispose" when Settings was left mid-load. This
  was pre-existing and reached the cashier as a red screen.

Also
- Header pill now reports real sync state: LIVE / n QUEUED / SYNCING /
  SYNC HALTED, with an explanation of where the bills are.
- Settings shows the route, last upload, next retry and retention window.
- docs/sync-contract.md states what the back office must implement, including
  the idempotency requirement that at-least-once delivery makes mandatory.

Tests: 90 -> 129 passing. New coverage for backoff shape and jitter band,
single flight, halting, ACK correlation and partial acceptance, at-least-once
duplicate handling, retention and purge, and no double-counting after a sync.
Suite run six times clean.

Not addressed: bills already synced by an older build went up overstated and
still need server-side reconciliation. Broker credentials have no Settings
editor yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Suriya
2026-08-01 10:57:29 +05:30
parent af3933092f
commit 0a49323858
29 changed files with 2855 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files

116
lib/data/repositories/sync_repository_impl.dart
View File

@@ -10,13 +10,24 @@ import '../../domain/repositories/sync_repository.dart';
import '../datasources/local_store.dart';
import '../datasources/remote_catalogue_source.dart';
import '../local/order_dao.dart';
import '../remote/order_transport.dart';
class SyncRepositoryImpl implements SyncRepository {
SyncRepositoryImpl(this._store, this._catalogue, this._orderSink);
SyncRepositoryImpl(
this._store,
this._catalogue,
this._transport, {
this.batchSize = 50,
});
final LocalStore _store;
final RemoteCatalogueSource _catalogue;
final RemoteOrderSink _orderSink;
final OrderTransport _transport;
/// Bills per publish. Kept modest because an MQTT broker will refuse an
/// oversized message outright, and a terminal that has been offline for a
/// day can easily hold hundreds of bills.
final int batchSize;
static const _uuid = Uuid();
@@ -176,13 +187,14 @@ class SyncRepositoryImpl implements SyncRepository {
var attempted = 0;
var uploaded = 0;
var refused = 0;
final syncedInvoices = <String>[];
// `unsynced()` returns a bounded page. Draining it in a loop means a day
// with more bills than one page still uploads completely, instead of
// reporting success with the remainder silently left behind.
while (true) {
final batch = await _store.orders.unsynced();
final batch = await _store.orders.unsynced(limit: batchSize);
if (batch.isEmpty) break;
final ids = batch.map((o) => o.id).toList();
@@ -193,36 +205,19 @@ class SyncRepositoryImpl implements SyncRepository {
'Uploading $attempted of $total bills…',
);
PushReceipt receipt;
try {
final accepted = await _orderSink.pushOrders(
receipt = await _transport.pushOrders(
batch.map(_orderToPayload).toList(),
);
// Only what the server confirmed is archived and removed. Anything it
// did not acknowledge stays on disk.
final acceptedOrders =
batch.where((o) => accepted.contains(o.id)).toList();
await _store.orders.archiveAndDelete(acceptedOrders);
await _store.refreshUnsyncedCount();
uploaded += acceptedOrders.length;
syncedInvoices.addAll(acceptedOrders.map((o) => o.invoiceNumber));
final rejected = ids.where((id) => !accepted.contains(id)).toList();
if (rejected.isNotEmpty) {
await _store.orders.markFailed(rejected, 'Rejected by server');
// Rejected rows stay pending, so the next page would return the same
// bills forever. Stop and let the cashier retry.
break;
}
} catch (e) {
// Transport failed: record the attempt but leave every row at 0.
} on Object catch (e) {
// The outcome is unknown, so nothing is marked sent. The attempt is
// recorded against the rows and every one of them stays at 0.
await _store.orders.markFailed(ids, e.toString());
await _store.refreshUnsyncedCount();
final remaining = await _store.orders.unsyncedCount();
final pendingValue =
batch.fold<double>(0, (s, o) => s + o.total);
final pendingValue = batch.fold<double>(0, (s, o) => s + o.total);
await _log(SyncEvent(
id: _uuid.v4(),
@@ -238,29 +233,90 @@ class SyncRepositoryImpl implements SyncRepository {
return SyncOutcome(
attempted: attempted,
uploaded: uploaded,
rejected: refused,
error: e.toString(),
isRetryable: e is! TransportException || e.retryable,
);
}
// Only what the back office named is archived and marked synced.
// Anything it stayed silent about is left pending — silence is not
// acceptance, whatever the transport reported at its own layer.
final acceptedIds = receipt.accepted.toSet();
final acceptedOrders =
batch.where((o) => acceptedIds.contains(o.id)).toList();
await _store.orders.archiveAccepted(acceptedOrders);
await _store.refreshUnsyncedCount();
uploaded += acceptedOrders.length;
syncedInvoices.addAll(acceptedOrders.map((o) => o.invoiceNumber));
final unconfirmed = ids.where((id) => !acceptedIds.contains(id)).toList();
if (unconfirmed.isNotEmpty) {
for (final id in unconfirmed) {
await _store.orders.markFailed(
[id],
receipt.rejected[id] ?? 'Not confirmed by the back office',
);
}
refused += unconfirmed.length;
// These rows are still pending, so the next page would hand back the
// same bills forever. Stop, and let a person look at why.
final reasons = receipt.rejected.values.toSet().join('; ');
await _log(SyncEvent(
id: _uuid.v4(),
type: SyncEventType.shiftReport,
status: SyncStatus.failed,
createdAt: started,
summary: '$uploaded uploaded, ${unconfirmed.length} refused',
error: reasons.isEmpty ? 'Not confirmed by the back office' : reasons,
attempts: 1,
),);
return SyncOutcome(
attempted: attempted,
uploaded: uploaded,
rejected: refused,
error: '${unconfirmed.length} bill(s) were not accepted'
'${reasons.isEmpty ? '' : ': $reasons'}',
// A refusal is a decision, not a fault. Retrying the same bytes gets
// the same answer, so the engine halts instead of looping.
isRetryable: false,
);
}
}
onProgress?.call(0.95, 'Tidying up…');
await purgeExpired();
onProgress?.call(1, 'Done');
// Synced bills are deleted from the terminal, so this log line is the only
// remaining record on the device that they went up.
// Once the retention window closes this log line is the only remaining
// record on the device that these bills went up.
await _log(SyncEvent(
id: _uuid.v4(),
type: SyncEventType.shiftReport,
status: SyncStatus.synced,
createdAt: started,
syncedAt: DateTime.now(),
summary: '$uploaded of $attempted bills uploaded',
summary: '$uploaded of $attempted bills uploaded '
'via ${_transport.label}',
payload: {'invoices': syncedInvoices},
attempts: 1,
),);
return SyncOutcome(attempted: attempted, uploaded: uploaded);
return SyncOutcome(
attempted: attempted,
uploaded: uploaded,
rejected: refused,
);
}
@override
Future<int> purgeExpired() => _store.orders.purgeSyncedBefore(
DateTime.now().subtract(OrderDao.retentionWindow),
);
/// The JSON body sent per order.
Map<String, Object?> _orderToPayload(SaleTransaction t) => {
'id': t.id,