# Doormile Backend — Changelog & Summary of Recent Changes This document provides a comprehensive log of the major features, architectural upgrades, schema modifications, and API changes recently implemented in the `doormile_backend`. --- ## 0. Customer sign-in fix, config hardening & admin ordering (2026-09-11) Six defects were reported against customer bookings. Five were real, one was not. Full detail in `CLAUDE.md` §8.6. **Fixed** - **`POST /customer/auth/otp/verify` now accepts `otp` as well as `code`.** The handler only ever read `code`; the app sent `otp`, because this repo's own quick-reference documented `otp` while `openapi-customer.yaml` said `code`. Every sign-in failed with a `400` — a correct code failed exactly like a wrong one. `code` remains the contract and wins when both are sent; `otp` is a deprecated alias kept so builds already installed keep working. - **A failed OTP send no longer leaves a live code behind.** `issueCxOtp` now rolls back the stored code, the resend cooldown and the rate-limit slot when SMS or email delivery fails, instead of charging the customer for the gateway's failure. - **`GET /admin/bookings` is ordered `bookingid DESC`.** It had no `ORDER BY`, so row order was unspecified — in practice oldest first, putting the newest booking on the last page and outside any client that reads a bounded number of pages. `OFFSET` paging over an unordered result was also unstable. - **Production hosts and credentials removed from `config.Load()` defaults.** `JWT_SECRET_KEY`, `NATS_URL`/`NATS_USER`/`NATS_PASSWORD`, `AI_LAYER_BASE_URL`, `ROUTE_OPTIMIZER_URL` and `DB_PASSWORD` all defaulted to real values, so a clone of this repo could mint a valid token for any account and any local run joined the live NATS stream. A second hardcoded production URL in `internal/assignment/ai_layer.go` was removed too. **Behaviour change to be aware of when deploying** `cfg.Validate()` now runs at startup and **the service refuses to boot when `JWT_SECRET_KEY` is unset and `ENV=production`**. Outside production an ephemeral per-process key is generated with a warning, so local development needs no configuration — but tokens no longer survive a restart unless you set the variable. Make sure `JWT_SECRET_KEY` is present in the production environment before the next deploy. Unset `NATS_URL` now means "no NATS" rather than "production NATS": publishes are dropped and no consumer starts. Set it explicitly wherever NATS is wanted. **Investigated and rejected** A reported +5:30 timestamp drift (`DBNow()` vs `timestamp with time zone` columns) does **not** exist on production — the columns there are `timestamp without time zone`, which is what `DBNow()` assumes, confirmed by a round-trip with zero drift. Changing `DBNow()` would introduce the bug. The real risk is that GORM's `AutoMigrate` produces `timestamptz`, so a freshly built schema does not match production and every new dev environment shows a drift that production does not. **Docs corrected** `customer-app-api-crisp.md` carried three request shapes that did not match their parsers (`auth/otp/verify`, `fare/estimate`, `bookings`) plus a wrong booking response shape; `express-console-api.md` documented pagination as "default 500, cap 1000" when the code enforces default 20, cap 100. Every one of these failed silently through `BodyParser` or a page budget, never as an error. **Still open** Email OTP returns 500 on production (`SMTP_*` unset); `.env` and a live GCP service-account key remain committed and need rotating; `GET /api/v1/ready` returns 503 with a body that says `"status":"ready"`. --- ## 1. Customer App v1 API Rebuild (`doormile_cx`) The customer-facing surface was completely rebuilt from the legacy single-destination / PIN-based flow to the production Customer App v1 contract. ### Key Additions & Refactorings: - **Authentication (`controllers/cxAuthController.go`)**: - Replaced legacy PIN authentication with 4-digit OTP verification via SMS/Email (`/customer/auth/send-otp`, `/customer/auth/verify-otp`). - Refresh token rotation and session management (`/customer/auth/refresh`, `/customer/auth/logout`, `/customer/auth/logout-all`). - Profile management and saved delivery locations (`/customer/profile`, `/customer/locations`). - **Multi-Destination Booking & Fanout (`controllers/cxBookingController.go`, `controllers/cxPickupFanout.go`)**: - Support for multi-stop pickups and multi-destination consignments under single parent bookings. - Pickup fanout algorithm ensuring correct route clustering and assignment creation. - **Dynamic Fare Engine (`controllers/cxFareController.go`)**: - Automated distance-based, weight-tiered, and peak-hour pricing calculations (`/customer/fare/estimate`). - **Stage Rollup & Tracking Lifecycle (`internal/cxstage/stage.go`, `controllers/cxBookingView.go`)**: - Unified lifecycle status machine mapping complex internal consignment and hub states to simplified customer stages (`created`, `assigned`, `arrived`, `picked_up`, `at_hub`, `out_for_delivery`, `delivered`, `cancelled`). - Strict 5-minute cancellation window enforced server-side. - **Public Reference Obfuscation (`controllers/cxIdentifiers.go`, `controllers/cxIdentifierScramble.go`)**: - Replaced sequential database ID leaks in public endpoints with collision-resistant Sqids/hash tokens. - **Catalogue, Serviceability & Places (`controllers/cxCatalogueController.go`, `controllers/cxPlacesController.go`)**: - Dynamic serviceability limits, slot schedules, and parcel category catalog queries (`/customer/catalogue`, `/customer/serviceability/limits`, `/customer/serviceability/slots`). - Places autocomplete proxy and reverse geocoding (`/customer/places/autocomplete`, `/customer/places/reverse-geocode`). - **Push Device Token Registry (`controllers/cxDeviceController.go`)**: - FCM/APNS device token registration for push notifications (`/customer/devices/register`, `/customer/devices/deregister`). - **Ops Staging Overrides (`controllers/cxOpsController.go`)**: - Development and QA testing endpoint for simulating order stage transitions in non-production environments (`POST /ops/bookings/:ref/stage`). --- ## 2. Logistics Base Handover & Hub Routing - **Logistics Handover (`controllers/logisticsHandoverController.go`)**: - Handover workflows between milers and logistics bases / hubs. - Audit logging of parcel check-ins and handoffs. - **Hub Inbound Processing (`controllers/hubInboundController.go`)**: - Bag scanning, parcel inwarding, and multi-hub dispatch reconciliation. - **Leg Optimizer & Routing (`internal/legs/legs.go`, `internal/routing/optimizer.go`)**: - Multi-hop inter-hub routing and transit leg calculations. --- ## 3. Miler App & Assignment Enhancements - **Arrival Confirmation Facts**: - Added support for `reachedat` / `arrivedat` timestamps on booking assignments and miler action payloads. - **Miler POD S3/Spaces Upload (`internal/storage/spaces.go`)**: - Presigned upload URL generation (`/miler/uploads/sign`) allowing milers to upload Proof of Delivery photos directly to object storage. - **Tenant Context**: - Exposed `tenantname` in `verify-pin` and miler profile responses. - **Consignment Status Check Expansion**: - Updated database checks to permit `Collected_By_Miler` and `Cancelled` statuses. --- ## 4. Middleware, Observability & Core Utilities - **Request ID & Structured Logging (`middlewares/requestid.go`, `middlewares/logger.go`)**: - Correlation IDs attached to incoming requests and structured log entries. - **Epoch Timestamp Conversions (`utils/epoch.go`)**: - Standardized millisecond/second epoch converters for unified JSON responses. - **Database Migrations & Models (`models/customer_app.go`, `migrations/migrate.go`)**: - Database schema migrations for customer auth tokens, devices, OTP logs, and extended booking columns. --- ## 5. Comprehensive Test Suite Added automated test suites covering all new and modified packages: - `controllers/cxCustomerApp_test.go` — Customer auth, booking creation, and validation tests. - `controllers/cxHttp_test.go` — End-to-end HTTP endpoint tests. - `routes/routes_customer_test.go` — Route registration and regression guards. - `internal/cxstage/stage_test.go` — Lifecycle state rollup logic and cancellation-window enforcement. - `utils/epoch_test.go` — Epoch timestamp validation. - `controllers/logisticsHandover_test.go` & `controllers/logisticsRouting_test.go` — Handover and routing tests.