doormilxpress_astryx/docs/reverse-logistics-plan.md

Reverse Logistics — Analysis & Implementation Plan

Status: Phases 1–3 built 2026-10-05 (uncommitted, not deployed) — see §10. Drafted 2026-10-05. Scope: doormile_backend (rules + endpoints), krow_talent_app (ops console), and the rider app (Flutter, separate team). The customer app is touched only where noted.

Defaults below are proposals; every one marked ⚑ is an open decision listed in §9.

---

1. What exists today (verified in code)

Area	Finding	Where
Return statuses	RTO_Initiated, Returned_to_Sender are defined and allowed by the DB constraint — but nothing in the code ever sets them	doormile_backend/constants/constants.go:91-92, migrations/migrate.go:124
Return fields on the parcel	Consignment already has Returnreason, Returninitiatedat, Returndeliveredat, Parentconsignmentid — all unused	doormile_backend/models/audit.go:70-73
Failed delivery	Rider taps skip → MilerSkipDelivery adds 1 to Attemptcount, logs Delivery_Skipped history. After 3 attempts it opens an Undeliverable exception — and stops there. The parcel stays Out_for_Delivery with the rider; no return leg, no owner, client not told	doormile_backend/controllers/milerAppController.go:990-1070
Exception types	Receiver_Refused, Undeliverable, Damaged, Lost, Misrouted, Missing_Contents exist	constants.go:186-195
Rider "what next"	nextActionForConsignment maps status → rider action. RTO/Returned fall into none ("past this rider's leg") — so a rider is never asked to bring a parcel back	controllers/logisticsHandoverController.go:233-251
Admin status change	PUT /admin/consignments/:id/status accepts any status string with no transition rules (a separate bug: it can set Delivered on a cancelled parcel)	controllers/adminController.go:3133-3190
Console	Only traces: an rto badge tone (components/ds/StatusBadge.jsx:47), rto → skipped mapping (api/doormile/queries.js:110), a comment in lib/orderFlow.js:108. Status update offers only Out_for_Delivery / Delivered / Cancelled	krow_talent_app/src/...
Pricing	Pricing has base/per-km/per-kg/handling — no return charge	models Pricing
Customer returns (reverse pickup)	Not present anywhere	—

Conclusion: the data model is mostly ready; the lifecycle, the rules, the endpoints, the console screens and the rider task are all missing.

---

2. Scope

Flow	Description	Phase
A. RTO — Return to Origin	Delivery fails → parcel goes back to the sender's pickup point (⚑ or a hub)	Phase 1–3 (this plan)
B. Customer return	Receiver sends a delivered item back; rider picks up from receiver, returns to sender	Phase 5 (later)
C. Exchange	Deliver new + collect old in one visit	Out of scope for now

---

3. RTO lifecycle (proposed)

Collected_By_Miler / Out_for_Delivery
        │  failed attempt (rider skip)  → Attemptcount++  (as today)
        │
        ├─ attempts ≥ N (⚑ default 3)  ─┐
        ├─ ops "Initiate RTO" (console) ─┼─▶ RTO_Initiated
        └─ (⚑ later) client request  ───┘      returnreason, returninitiatedat set
                                                history: RTO_Initiated
                                                rider next action: return_to_sender
                         │
          ┌──────────────┼───────────────────────────┐
          ▼              ▼                           ▼
  rider returns    ops "Re-attempt delivery"   ops "Mark returned"
  to sender        (cancels the RTO)           (manual close, e.g. hub drop)
          │              │                           │
          ▼              ▼                           ▼
 Returned_to_Sender  Out_for_Delivery       Returned_to_Sender
 returndeliveredat   (attempts kept)        returndeliveredat

Rules:

• RTO only from Collected_By_Miler, Out_for_Delivery (or Inwarded_at_Hub ⚑).
• Returned_to_Sender is terminal.
• Every transition writes ConsignmentHistory with actor + reason.
• Opening RTO resolves the linked Undeliverable exception (if any) with resolution "RTO initiated".
• COD: a returned parcel collects nothing; Codcollected stays 0.

---

4. Backend — doormile_backend

#	Change	Detail
B1	Transition guard	One function canTransition(from, to) used by every consignment status write; PUT /admin/consignments/:id/status refuses illegal moves (also fixes the any-status bug).
B2	POST /admin/consignments/:id/rto {reason, note}	Staff only. Sets RTO_Initiated, returnreason, returninitiatedat; history; resolves the Undeliverable exception; publishes NATS consignment.rto_initiated; notifies the rider. Idempotent.
B3	POST /admin/consignments/:id/rto/cancel {note}	Back to Out_for_Delivery (re-attempt).
B4	POST /admin/consignments/:id/rto/complete {note}	Ops closes it manually → Returned_to_Sender, returndeliveredat.
B5	GET /admin/returns?status&from&to&tenantid&hubid&pageno	List for the Returns page: tracking no, client, sender, reason, attempts, initiated/returned times, rider, age. Tenant-scoped like other admin lists.
B6	Auto-RTO in MilerSkipDelivery	When Attemptcount ≥ RTO_AUTO_AFTER_ATTEMPTS (env, ⚑ default 3; 0 = off) → call the same B2 logic instead of only opening an exception.
B7	Rider task	New next action return_to_sender; nextActionForConsignment(RTO_Initiated) returns it; rider queue includes the return stop (sender's pickup coords). New POST /miler/consignments/:id/return-complete {lat, lon, photourl?, receivedby} → Returned_to_Sender. Behind flag MILER_RTO_FLOW_ENABLED (default off) — the deployed rider app doesn't know return_to_sender, same pattern as MILER_HUB_HANDOVER_ENABLED.
B8	Booking/customer stage	For customer-app bookings, record the return in cxstage. ⚠ The customer app renders unknown stage keys as booked, so a new "returning" stage needs a client release — until then do not add a new key.
B9	Tests	Transition table, each endpoint's gates (staff/tenant/owner), auto-RTO at N attempts, flag off/on rider queue, idempotency. Postgres-gated tests for the SQL.

Schema: none for Phase 1–3 (columns exist). ⚑ Return-to-hub needs one additive nullable column (returnhubid) — Phase 4.

---

5. Console — krow_talent_app

#	Change	Where
C1	API + hooks: initiateRto, cancelRto, completeRto, getReturns + useInitiateRto … with invalidation of deliveries/returns keys	src/api/doormile/endpoints.js, src/lib/doormileHooks.js
C2	Deliveries page actions: on failed / out-for-delivery rows — Initiate RTO (reason picker: Receiver refused, Address not found, Customer unavailable, Attempts exhausted, Other + note), Re-attempt, Mark returned. Show attempt count	src/pages/doormile/deliveries/Deliveries.jsx
C3	Status tabs: add RTO and Returned tabs/counts (today rto collapses into skipped)	queries.js status map, StatusBadge
C4	Returns page /doormile/returns: tabs Initiated · In return · Returned; filters (date, client, hub, reason); age/SLA column; XLSX export; nav entry under Fleet Ops (staff)	new src/pages/doormile/returns/Returns.jsx, App.jsx, AdminLayout.jsx
C5	Exceptions page: an Undeliverable / Receiver_Refused exception gets a Start RTO button	src/pages/doormile/exceptions/Exceptions.jsx
C6	Order / consignment timeline shows attempts, RTO start, return	booking detail drawer
C7	Reports: return rate per client and per reason on Orders Summary	src/pages/doormile/reports/
C8	Client (tenant) logins: read-only view of their own returns (no actions)	Returns page + role check
C9	Tests: actions call the right endpoints, reason required, tabs/counts, Returns filters, client read-only	tests/integration/

---

6. Rider app (Flutter — separate team)

• Handle next action return_to_sender: show the return stop, navigate to the sender, "Returned" button → POST /miler/consignments/:id/return-complete with location (+ optional photo / receiver name).
• Release before turning on MILER_RTO_FLOW_ENABLED. Until then, ops close returns from the console (B4) and riders are told by push.

---

7. Phases & order

Phase	Content	Repos	Depends on
1	B1 guard, B2–B5 endpoints, B9 tests	backend	—
2	C1–C5, C9	console	Phase 1 deployed
3	B6 auto-RTO, B7 rider task (flag off) + rider app release, then flag on	backend + rider app	Phase 1
4	Return-to-hub option (returnhubid), C6 timeline, C7 reports, C8 client view, return charges	all	decisions ⚑
5	Flow B — customer returns (reverse pickup booking linked by Parentconsignmentid)	all + customer app	product spec

Rough effort: Phase 1 ≈ 1.5–2 days · Phase 2 ≈ 2 days · Phase 3 ≈ 1 day backend (+ rider app team) · Phase 4/5 to estimate after decisions.

---

8. Risks

• Rider app compatibility — a new next action unknown to the deployed app must stay behind a flag (B7).
• Customer app stage keys — unknown keys render as booked (B8).
• Ops discipline — until the rider task ships, returns depend on ops closing them in the console.
• Existing any-status endpoint — B1 changes its behaviour: callers that relied on setting arbitrary statuses will now get 400. The console only sends Out_for_Delivery / Delivered / Cancelled, which stay allowed.
• Deploy — no migration in Phase 1–3; Phase 4 adds one nullable column (needs approval).

---

9. Open decisions ⚑

1. Flows first: A (RTO) only, then B? (proposed: yes)
2. Return destination: sender's pickup point, nearest hub, or per client? (proposed: sender for Phase 1; hub option in Phase 4)
3. Who starts RTO: auto after N attempts (N = ?), ops, client, or all? (proposed: ops + auto after 3; client later)
4. Can RTO start from a hub (Inwarded_at_Hub)? (proposed: yes)
5. Return charges: billed to the client? Same rate, fixed fee, or free? (proposed: decide before Phase 4; Phase 1–3 record only)
6. Rider app: is the Flutter team available for B7, and when?
7. Client visibility: may tenants see their own returns? (proposed: read-only)

---

10. Implementation status (2026-10-05)

Built with the proposed defaults: return to the sender, started by ops or automatically after 3 failed attempts, rider task behind a flag (off). Nothing is committed or pushed.

Backend — doormile_backend

Item	Where
RTO core: startRTO, completeRTO, cancel → previous status (recorded as [from:<status>] in the history remark), exception auto-resolve, rider push	controllers/consignmentReturn.go
POST /admin/consignments/:id/rto · /rto/cancel · /rto/complete (Doormile staff only)	routes/routes.go
GET /admin/returns?status=initiated|returned|all&from&to&pageno&pagesize (tenant-scoped; client logins read their own)	same
POST /miler/consignments/:id/return-complete — 403 RTO_FLOW_DISABLED unless MILER_RTO_FLOW_ENABLED=true	same
Status guard on PUT /admin/consignments/:id/status (unknown status, leaving Delivered/Cancelled/Returned, RTO statuses → 400)	controllers/adminController.go
Auto-RTO after RTO_AUTO_AFTER_ATTEMPTS (default 3, 0 = off)	MilerSkipDelivery in controllers/milerAppController.go
MilerSkipDelivery ownership now multi-destination safe (milerConsignmentForRider) — fixes skip on orders 2..N	same
Next action return_to_sender (flagged); rider consignment read adds returning, can_return, return_reason, return_to	constants, logisticsHandoverController.go, milerAppController.go
Tests: 7 unit + 5 route-gate tests	controllers/consignmentReturn_test.go, routes/routes_rto_test.go

New env vars: RTO_AUTO_AFTER_ATTEMPTS (default 3), MILER_RTO_FLOW_ENABLED (default off). No schema change. No customer-app stage key added (B8).

Console — krow_talent_app

Item	Where
API + hooks: initiateRto, cancelRto, completeRto, getReturns, RTO_REASONS; useReturns, useInitiateRto, useCancelRto, useCompleteRto	src/api/doormile/endpoints.js, src/lib/doormileHooks.js
Status mapping: RTO_Initiated → rto, Returned_to_Sender → returned (legacy rto/returned keys unchanged) + counts	src/api/doormile/queries.js
Shared dialogs: Start return (reason + note, pre-select), Re-attempt / Mark returned	src/components/doormile/RtoDialogs.jsx
Deliveries: In return + Returned tabs; Return to sender / Re-attempt / Mark returned (staff only); status-update and cancel hidden on returning rows	src/pages/doormile/deliveries/Deliveries.jsx
Returns page /doormile/returns (tabs, date range, search, export, pagination; read-only for clients); nav under Fleet Ops	src/pages/doormile/returns/Returns.jsx, App.jsx, AdminLayout.jsx
Exceptions: Start return on open Undeliverable / Receiver_Refused	src/pages/doormile/exceptions/Exceptions.jsx
Tests: 13 new; deliveries counts test updated	tests/integration/returns.test.jsx, tests/api/deliveries.test.js

Verified / not verified

• ✅ go build, go vet, all 17 backend packages; console suites pass except the 13 Agent Studio tests already failing since the 2026-09-30 redesign; console production build OK.
• ✅ End-to-end on a real database (2026-10-05). This ran on a throwaway local stack: PostgreSQL 17.10 on 127.0.0.1, the backend built from this tree with MILER_RTO_FLOW_ENABLED=true, and the console on a local port pointed at it. Production was not touched.
  • Browser, as staff:
    • Return to sender (reason + note), then Re-attempt (back to Out_for_Delivery), then Start again, then Mark returned. Result: Returned_to_Sender, the rider's assignment Completed, and the full status history.
    • Exceptions → Start return pre-selects the reason, and the exception leaves the list.
    • The Returns page lists rows with correct India times.
  • API:
    • 3 skips start RTO automatically and resolve the Undeliverable exception.
    • The rider read gives next_action=return_to_sender with return_to set to the sender's coordinates.
    • Rider return-complete closes the assignment.
    • Every guard answers 400 / INVALID_STATE as designed.
  • Client login: no return buttons on Deliveries; Returns is read-only; rto/cancel and rto/complete return 403.
• Bugs the browser test found, all fixed:
  1. Notify rider showed on returned and delivered rows. It is now hidden.
  2. Exceptions didn't refresh after Start return. RETURN_KEYS now includes the exceptions key.
  3. The dialog subtitle from Exceptions read "Order consignment #…". It now uses a label prop.
• Backend review (2026-10-05), fixed:
  1. Race conditions. Start, complete and cancel used to read the parcel and then save the whole row. A rider delivering at the same moment, or a double click, could be overwritten. Each one is now a compare-and-set (moveConsignment: update only if the status is still the one read). A lost race either becomes the same no-op as a repeat, or returns "changed, refresh".
  2. Re-attempt clears the return. returnreason and returninitiatedat are cleared, so a parcel delivered afterwards doesn't carry a stale reason. The history keeps the reason.
  3. "OTHER" check. A reason sent as "OTHER" or "Other " skipped the note-required check. The check now uses the normalised reason (rtoReasonText).
  4. Note length. The 500 limit is counted in characters, not bytes. Before, 200 Tamil characters were refused even though the console allows 500.
  5. Wrong rider notified. Starting a return on a parcel already handed over at a base pushed "do not attempt delivery" to its old pickup rider. Now only a rider holding the parcel is notified (riderHoldsParcel: Created, Collected_By_Miler, Out_for_Delivery).
• Impact on existing users (checked 2026-10-05). Decide these before deploying:
  • Rider app (deployed build): it shows a parcel from the booking status, which a return doesn't change. So a parcel in return still looks deliverable; tapping deliver or skip returns a 400. The rider also can't go off duty until ops press Mark returned, because the assignment stays open. Automatic return is on by default (3 attempts), so this starts on deploy. Either set RTO_AUTO_AFTER_ATTEMPTS=0 until the rider app supports returns, or make sure ops close returns daily.
  • Console "Update status": Delivered and Cancelled parcels can no longer be changed (400). Before, ops could undo a wrong "Delivered" or reopen a cancelled parcel. That is now refused by design.
  • Customer app: a returned parcel keeps showing "Out for delivery" (there is no customer stage for returns).
  • Rider pay: closing a return marks the assignment Completed with 0 km and 0 charges. The return trip isn't paid.
  • Not affected: rider delivered counts and reports (Delivered only), the hub console (a parcel in return simply drops out of its status-based lists), the B2C booking flow, the existing status values, and the database schema (no change).
• Tests added: TestRiderHoldsParcel, TestRTOReasonText, plus controllers/consignmentReturn_pg_test.go (5 tests on real Postgres: lifecycle, refusals, no overwrite of a concurrent delivery, 10 simultaneous starts → one return, cancel). These skip unless REGISTRY_TEST_DSN is set, and passed on PostgreSQL 17.
• Timestamps stay time.Now(), not utils.DBNow(). AutoMigrate creates timestamptz columns, where DBNow() would store a time 5 h 30 m ahead. time.Now() is correct for both column types because the Dockerfile sets TZ=Asia/Kolkata.
• Seen, not fixed (pre-existing, outside RTO): the Exceptions page shows "Raised" times about 6 h ahead of IST.
• Not built: Phase 4 (return-to-hub, timeline, reports, charges), Phase 5 (customer returns), the rider-app UI (Flutter team).

11. Phase 4, first part (2026-10-07)

Built the two Phase 4 items that need no product decision. Uncommitted.

Item	Where
GET /admin/consignments/:id/history: every event of one parcel, oldest first, with who did it and at which base. The [from:<status>] bookkeeping tag is removed from the remark and returned as fromstatus. Client logins read their own parcels only.	doormile_backend/controllers/returnInsights.go, routes/routes.go
GET /admin/returns/summary?from&to: of the parcels created in the period (cancelled ones excluded), the return rate overall, per client and per reason, plus the average days a completed return took. Defaults to the last 30 days. Client logins get their own figures only.	same
C6 timeline: shared ConsignmentTimeline, shown in the Deliveries order drawer and behind a History button on every Returns row (clients included).	src/components/doormile/ConsignmentTimeline.jsx, Deliveries.jsx, Returns.jsx
C7 report: a summary on the Returns page (KPIs, return rate by client, reasons), driven by the page's date range. Clients don't get the per-client table. Placed on Returns rather than Orders Summary so that clients see it too.	src/pages/doormile/returns/Returns.jsx
Tests: 2 Postgres route tests (timeline order, actor, hub, scoping; summary rates, reasons, window, scoping) and 4 console tests.	routes/routes_return_insights_pg_test.go, tests/integration/returns.test.jsx

Verified in the browser against a throwaway local database: summary figures, per-client and per-reason tables, and the timeline drawer.

Decision 2026-10-07 on return charges (open decision 5): none for now. Returns are free for every client. Revisit after 2–3 months using the return rates the Returns summary now shows. If a charge is introduced later, it should be per client, only for receiver- or client-caused returns, and never for damage or Doormile errors.

Still open in Phase 4 (needs a decision ⚑): return-to-hub (the returnhubid column).

Decision 2026-10-07 on Phase 5 (customer returns after delivery): deferred. Current clients are mainly food and medicine businesses, where returns after delivery are rare, and refunds are handled by the client. Build it when a client asks for it or when parcel/e-commerce clients are onboarded. Until then, ops handle the rare case by booking a normal order with the customer's address as pickup and the client as drop. The return button appears only on active orders (RTO, before delivery); Delivered stays terminal.