Merge pull request #544 from Oloodi/codex/feat-architecture-lead-bootstrap

M4 architecture lead bootstrap: docs, core API guidance, schema planning
This commit is contained in:
Wielfried Zouantcha
2026-02-26 10:09:33 -05:00
committed by GitHub
57 changed files with 10391 additions and 314 deletions

View File

@@ -0,0 +1,64 @@
name: Backend Foundation
on:
pull_request:
branches:
- dev
- main
push:
branches:
- dev
- main
jobs:
backend-foundation-makefile:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Validate backend make targets
run: |
make backend-help
make help | grep "backend-"
- name: Dry-run backend automation targets
run: |
make -n backend-enable-apis ENV=dev
make -n backend-bootstrap-dev ENV=dev
make -n backend-deploy-core ENV=dev
make -n backend-deploy-commands ENV=dev
make -n backend-deploy-workers ENV=dev
make -n backend-smoke-core ENV=dev
make -n backend-smoke-commands ENV=dev
make -n backend-logs-core ENV=dev
backend-services-tests:
runs-on: ubuntu-latest
strategy:
matrix:
service:
- backend/core-api
- backend/command-api
defaults:
run:
working-directory: ${{ matrix.service }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: ${{ matrix.service }}/package-lock.json
- name: Install dependencies
run: npm ci
- name: Run tests
env:
AUTH_BYPASS: "true"
LLM_MOCK: "true"
run: npm test

6
.gitignore vendored
View File

@@ -187,3 +187,9 @@ krow-workforce-export-latest/
# Data Connect Generated SDKs (Explicit)
apps/mobile/packages/data_connect/lib/src/dataconnect_generated/
apps/web/src/dataconnect-generated/
AGENTS.md
CLAUDE.md
GEMINI.md
TASKS.md

30
CHANGELOG.md Normal file
View File

@@ -0,0 +1,30 @@
# KROW Workforce Change Log
| Date | Version | Change |
|---|---|---|
| 2026-02-24 | 0.1.0 | Confirmed dev owner access and current runtime baseline in `krow-workforce-dev`. |
| 2026-02-24 | 0.1.1 | Added backend foundation implementation plan document. |
| 2026-02-24 | 0.1.2 | Added API implementation contract and transition route aliases. |
| 2026-02-24 | 0.1.3 | Added auth-first security policy with deferred role-map integration hooks. |
| 2026-02-24 | 0.1.4 | Locked defaults for idempotency, validation, bucket split, model provider, and p95 objectives. |
| 2026-02-24 | 0.1.5 | Added backend makefile module and CI workflow for backend target validation. |
| 2026-02-24 | 0.1.6 | Added Cloud SQL-backed idempotency storage, migration script, and command API test coverage. |
| 2026-02-24 | 0.1.7 | Added `/health` endpoints and switched smoke checks to `/health` for Cloud Run compatibility. |
| 2026-02-24 | 0.1.8 | Enabled dev frontend reachability and made deploy auth mode environment-aware (`dev` public, `staging` private). |
| 2026-02-24 | 0.1.9 | Switched core API from mock behavior to real GCS upload/signed URLs and real Vertex model calls in dev deployment. |
| 2026-02-24 | 0.1.10 | Hardened core APIs with signed URL ownership/expiry checks, object existence checks, and per-user LLM rate limiting. |
| 2026-02-24 | 0.1.11 | Added frontend-ready core API guide and linked M4 API catalog to it as source of truth for consumption. |
| 2026-02-24 | 0.1.12 | Reduced M4 API docs to core-only scope and removed command-route references until command implementation is complete. |
| 2026-02-24 | 0.1.13 | Added verification architecture contract with endpoint design and workflow split for attire, government ID, and certification. |
| 2026-02-24 | 0.1.14 | Implemented core verification endpoints in dev and updated frontend/API docs with live verification route contracts. |
| 2026-02-24 | 0.1.15 | Added live Vertex Flash Lite attire verification path and third-party adapter scaffolding for government ID and certification checks. |
| 2026-02-24 | 0.1.16 | Added M4 target schema blueprint doc with first-principles modular model, constraints, and migration phases. |
| 2026-02-24 | 0.1.17 | Added full current-schema mermaid model relationship map to the M4 target schema blueprint. |
| 2026-02-24 | 0.1.18 | Updated schema blueprint with explicit multi-tenant stakeholder model and phased RBAC rollout with shadow mode before enforcement. |
| 2026-02-24 | 0.1.19 | Added customer stakeholder-wheel mapping and future stakeholder extension model to the M4 schema blueprint. |
| 2026-02-25 | 0.1.20 | Added roadmap CSV schema-reconciliation document with stakeholder capability matrix and concrete schema gap analysis. |
| 2026-02-25 | 0.1.21 | Updated target schema blueprint with roadmap-evidence section plus attendance/offense, stakeholder-network, and settlement-table coverage. |
| 2026-02-25 | 0.1.22 | Updated core actor scenarios with explicit business and vendor user partitioning via membership tables. |
| 2026-02-25 | 0.1.23 | Updated schema blueprint and reconciliation docs to add `business_memberships` and `vendor_memberships` as first-class data actors. |
| 2026-02-25 | 0.1.24 | Removed stale `m4-discrepencies.md` document from M4 planning docs cleanup. |
| 2026-02-25 | 0.1.25 | Added target schema model catalog with keys and domain relationship diagrams for slide/workshop use. |

129
CLAUDE.md
View File

@@ -1,129 +0,0 @@
# CLAUDE.md - Project Context for AI Assistants
This file provides context for Claude Code and other AI assistants working on this codebase.
## Project Overview
**KROW Workforce** is a workforce management platform connecting businesses with temporary/on-demand workers. It consists of:
- **Client App**: For businesses to create orders, manage hubs, handle billing
- **Staff App**: For workers to manage availability, clock in/out, view earnings
- **Web Dashboard**: Admin portal (React/Vite - WIP)
- **Backend**: Firebase Data Connect + PostgreSQL on Cloud SQL
## Monorepo Structure
```
/apps
/mobile # Flutter apps (managed by Melos)
/apps
/client # krowwithus_client - Business app
/staff # krowwithus_staff - Worker app
/design_system_viewer
/packages
/core # Base utilities
/domain # Business entities, repository interfaces
/data_connect # Data layer, Firebase Data Connect SDK
/design_system # Shared UI components
/core_localization # i18n (Slang)
/features
/client/* # Client-specific features
/staff/* # Staff-specific features
/web-dashboard # React web app (WIP)
/backend
/dataconnect # GraphQL schemas, Firebase Data Connect config
/cloud-functions # Serverless functions (placeholder)
/internal
/launchpad # Internal DevOps portal
/api-harness # API testing tool
/makefiles # Modular Make targets
/docs # Project documentation
```
## Key Commands
### Mobile Development
```bash
# Install dependencies
make mobile-install
# Run client app (specify your device ID)
make mobile-client-dev-android DEVICE=<device_id>
# Run staff app
make mobile-staff-dev-android DEVICE=<device_id>
# Find your device ID
flutter devices
# Build APK
make mobile-client-build PLATFORM=apk
make mobile-staff-build PLATFORM=apk
# Code generation (localization + build_runner)
cd apps/mobile && melos run gen:all
```
### Web Development
```bash
make install # Install web dependencies
make dev # Run web dev server
```
### Data Connect
```bash
make dataconnect-sync # Deploy schemas, migrate, regenerate SDK
```
## Architecture Patterns
- **State Management**: BLoC pattern (flutter_bloc)
- **Navigation**: Flutter Modular
- **Architecture**: Clean Architecture (domain/data/presentation layers)
- **Feature Organization**: Each feature is a separate package
- **Value Objects**: Equatable for entity equality
## Code Conventions
- Features go in `/apps/mobile/packages/features/{client|staff}/`
- Shared code goes in `/apps/mobile/packages/{core|domain|data_connect}/`
- UI components go in `/apps/mobile/packages/design_system/`
- GraphQL schemas go in `/backend/dataconnect/schema/`
- Documentation language: **English**
## Important Files
- `apps/mobile/melos.yaml` - Melos workspace config
- `makefiles/mobile.mk` - Mobile Make targets
- `backend/dataconnect/dataconnect.yaml` - Data Connect config
- `firebase.json` - Firebase hosting/emulator config
- `BLOCKERS.md` - Known blockers and deviations
## Branch Protection
- `main` and `dev` branches are protected
- Always create feature branches: `feature/`, `fix/`, `chore/`
- PRs required for merging
## Testing Mobile Apps
1. Connect your Android device or start emulator
2. Run `flutter devices` to get device ID
3. Run `make mobile-client-dev-android DEVICE=<your_device_id>`
## Common Issues
### "No devices found with name 'android'"
The Makefile defaults to device ID `android`. Override with your actual device:
```bash
make mobile-client-dev-android DEVICE=3fb285a7
```
### Dependency resolution issues
```bash
cd apps/mobile && melos clean && melos bootstrap
```
### Code generation out of sync
```bash
cd apps/mobile && melos run gen:all
```

138
GEMINI.md
View File

@@ -1,138 +0,0 @@
# GEMINI.md - Project Context for AI Assistants
This file provides context for Gemini and other AI assistants working on this codebase.
## Project Overview
**KROW Workforce** is a workforce management platform connecting businesses with temporary/on-demand workers. It consists of:
- **Client App**: For businesses to create orders, manage hubs, handle billing
- **Staff App**: For workers to manage availability, clock in/out, view earnings
- **Web Dashboard**: Admin portal (React/Vite - WIP)
- **Backend**: Firebase Data Connect + PostgreSQL on Cloud SQL
## Monorepo Structure
```
/apps
/mobile # Flutter apps (managed by Melos)
/apps
/client # krowwithus_client - Business app
/staff # krowwithus_staff - Worker app
/design_system_viewer
/packages
/core # Base utilities
/domain # Business entities, repository interfaces
/data_connect # Data layer, Firebase Data Connect SDK
/design_system # Shared UI components
/core_localization # i18n (Slang)
/features
/client/* # Client-specific features
/staff/* # Staff-specific features
/web-dashboard # React web app (WIP)
/backend
/dataconnect # GraphQL schemas, Firebase Data Connect config
/cloud-functions # Serverless functions (placeholder)
/internal
/launchpad # Internal DevOps portal
/api-harness # API testing tool
/makefiles # Modular Make targets
/docs # Project documentation
/bugs # Bug reports and screenshots
```
## Key Commands
### Mobile Development
```bash
# Install dependencies
make mobile-install
# Run client app (specify your device ID)
make mobile-client-dev-android DEVICE=<device_id>
# Run staff app
make mobile-staff-dev-android DEVICE=<device_id>
# Find your device ID
flutter devices
# Build APK
make mobile-client-build PLATFORM=apk
make mobile-staff-build PLATFORM=apk
# Code generation (localization + build_runner)
cd apps/mobile && melos run gen:all
```
### Web Development
```bash
make install # Install web dependencies
make dev # Run web dev server
```
### Data Connect
```bash
make dataconnect-sync # Deploy schemas, migrate, regenerate SDK
```
## Architecture Patterns
- **State Management**: BLoC pattern (flutter_bloc)
- **Navigation**: Flutter Modular
- **Architecture**: Clean Architecture (domain/data/presentation layers)
- **Feature Organization**: Each feature is a separate package
- **Value Objects**: Equatable for entity equality
## Code Conventions
- Features go in `/apps/mobile/packages/features/{client|staff}/`
- Shared code goes in `/apps/mobile/packages/{core|domain|data_connect}/`
- UI components go in `/apps/mobile/packages/design_system/`
- GraphQL schemas go in `/backend/dataconnect/schema/`
- Documentation language: **English**
## Important Files
- `apps/mobile/melos.yaml` - Melos workspace config
- `makefiles/mobile.mk` - Mobile Make targets
- `backend/dataconnect/dataconnect.yaml` - Data Connect config
- `firebase.json` - Firebase hosting/emulator config
- `BLOCKERS.md` - Known blockers and deviations
- `bugs/BUG-REPORT-*.md` - Bug reports with analysis
## Branch Protection
- `main` and `dev` branches are protected
- Always create feature branches: `feature/`, `fix/`, `chore/`
- PRs required for merging
## Testing Mobile Apps
1. Connect your Android device or start emulator
2. Run `flutter devices` to get device ID
3. Run `make mobile-client-dev-android DEVICE=<your_device_id>`
## Common Issues
### "No devices found with name 'android'"
The Makefile defaults to device ID `android`. Override with your actual device:
```bash
make mobile-client-dev-android DEVICE=3fb285a7
```
### Dependency resolution issues
```bash
cd apps/mobile && melos clean && melos bootstrap
```
### Code generation out of sync
```bash
cd apps/mobile && melos run gen:all
```
## Known Technical Debt
See `bugs/BUG-REPORT-*.md` for detailed analysis of:
- Authentication/User sync issues
- Error handling architecture (needs AppException pattern)
- BLoC state management patterns (copyWith null handling)

View File

@@ -11,6 +11,7 @@ include makefiles/web.mk
include makefiles/launchpad.mk
include makefiles/mobile.mk
include makefiles/dataconnect.mk
include makefiles/backend.mk
include makefiles/tools.mk
# --- Main Help Command ---
@@ -71,6 +72,19 @@ help:
@echo " make dataconnect-bootstrap-validation-database ONE-TIME: Setup validation database"
@echo " make dataconnect-backup-dev-to-validation Backup dev database to validation"
@echo ""
@echo " ☁️ BACKEND FOUNDATION (Cloud Run + Workers)"
@echo " ────────────────────────────────────────────────────────────────────"
@echo " make backend-help Show backend foundation commands"
@echo " make backend-enable-apis [ENV=dev] Enable backend GCP APIs"
@echo " make backend-bootstrap-dev Bootstrap backend foundation resources (dev)"
@echo " make backend-migrate-idempotency Create/upgrade command idempotency table"
@echo " make backend-deploy-core [ENV=dev] Build and deploy core API service"
@echo " make backend-deploy-commands [ENV=dev] Build and deploy command API service"
@echo " make backend-deploy-workers [ENV=dev] Deploy async worker functions scaffold"
@echo " make backend-smoke-core [ENV=dev] Run health smoke test for core service (/health)"
@echo " make backend-smoke-commands [ENV=dev] Run health smoke test for command service (/health)"
@echo " make backend-logs-core [ENV=dev] Tail/read logs for core service"
@echo ""
@echo " 🛠️ DEVELOPMENT TOOLS"
@echo " ────────────────────────────────────────────────────────────────────"
@echo " make install-melos Install Melos globally (for mobile dev)"

View File

@@ -0,0 +1,13 @@
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY src ./src
ENV PORT=8080
EXPOSE 8080
CMD ["node", "src/server.js"]

3035
backend/command-api/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,25 @@
{
"name": "@krow/command-api",
"version": "0.1.0",
"private": true,
"type": "module",
"engines": {
"node": ">=20"
},
"scripts": {
"start": "node src/server.js",
"test": "node --test",
"migrate:idempotency": "node scripts/migrate-idempotency.mjs"
},
"dependencies": {
"express": "^4.21.2",
"firebase-admin": "^13.0.2",
"pg": "^8.16.3",
"pino": "^9.6.0",
"pino-http": "^10.3.0",
"zod": "^3.24.2"
},
"devDependencies": {
"supertest": "^7.0.0"
}
}

View File

@@ -0,0 +1,29 @@
import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { Pool } from 'pg';
const databaseUrl = process.env.IDEMPOTENCY_DATABASE_URL;
if (!databaseUrl) {
// eslint-disable-next-line no-console
console.error('IDEMPOTENCY_DATABASE_URL is required');
process.exit(1);
}
const scriptDir = resolve(fileURLToPath(new URL('.', import.meta.url)));
const migrationPath = resolve(scriptDir, '../sql/001_command_idempotency.sql');
const sql = readFileSync(migrationPath, 'utf8');
const pool = new Pool({
connectionString: databaseUrl,
max: Number.parseInt(process.env.IDEMPOTENCY_DB_POOL_MAX || '5', 10),
});
try {
await pool.query(sql);
// eslint-disable-next-line no-console
console.log('Idempotency migration applied successfully');
} finally {
await pool.end();
}

View File

@@ -0,0 +1,13 @@
CREATE TABLE IF NOT EXISTS command_idempotency (
composite_key TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
route TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
status_code INTEGER NOT NULL,
response_payload JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
expires_at TIMESTAMPTZ NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_command_idempotency_expires_at
ON command_idempotency (expires_at);

View File

@@ -0,0 +1,30 @@
import express from 'express';
import pino from 'pino';
import pinoHttp from 'pino-http';
import { requestContext } from './middleware/request-context.js';
import { errorHandler, notFoundHandler } from './middleware/error-handler.js';
import { healthRouter } from './routes/health.js';
import { createCommandsRouter } from './routes/commands.js';
const logger = pino({ level: process.env.LOG_LEVEL || 'info' });
export function createApp() {
const app = express();
app.use(requestContext);
app.use(
pinoHttp({
logger,
customProps: (req) => ({ requestId: req.requestId }),
})
);
app.use(express.json({ limit: '2mb' }));
app.use(healthRouter);
app.use('/commands', createCommandsRouter());
app.use(notFoundHandler);
app.use(errorHandler);
return app;
}

View File

@@ -0,0 +1,6 @@
import { z } from 'zod';
export const commandBaseSchema = z.object({
payload: z.record(z.any()).optional(),
metadata: z.record(z.any()).optional(),
});

View File

@@ -0,0 +1,26 @@
export class AppError extends Error {
constructor(code, message, status = 400, details = {}) {
super(message);
this.name = 'AppError';
this.code = code;
this.status = status;
this.details = details;
}
}
export function toErrorEnvelope(error, requestId) {
const status = error?.status && Number.isInteger(error.status) ? error.status : 500;
const code = error?.code || 'INTERNAL_ERROR';
const message = error?.message || 'Unexpected error';
const details = error?.details || {};
return {
status,
body: {
code,
message,
details,
requestId,
},
};
}

View File

@@ -0,0 +1,45 @@
import { AppError } from '../lib/errors.js';
import { can } from '../services/policy.js';
import { verifyFirebaseToken } from '../services/firebase-auth.js';
function getBearerToken(header) {
if (!header) return null;
const [scheme, token] = header.split(' ');
if (!scheme || scheme.toLowerCase() !== 'bearer' || !token) return null;
return token;
}
export async function requireAuth(req, _res, next) {
try {
const token = getBearerToken(req.get('Authorization'));
if (!token) {
throw new AppError('UNAUTHENTICATED', 'Missing bearer token', 401);
}
if (process.env.AUTH_BYPASS === 'true') {
req.actor = { uid: 'test-user', email: 'test@krow.local', role: 'TEST' };
return next();
}
const decoded = await verifyFirebaseToken(token);
req.actor = {
uid: decoded.uid,
email: decoded.email || null,
role: decoded.role || null,
};
return next();
} catch (error) {
if (error instanceof AppError) return next(error);
return next(new AppError('UNAUTHENTICATED', 'Token verification failed', 401));
}
}
export function requirePolicy(action, resource) {
return (req, _res, next) => {
if (!can(action, resource, req.actor)) {
return next(new AppError('FORBIDDEN', 'Not allowed to perform this action', 403));
}
return next();
};
}

View File

@@ -0,0 +1,25 @@
import { toErrorEnvelope } from '../lib/errors.js';
export function notFoundHandler(req, res) {
res.status(404).json({
code: 'NOT_FOUND',
message: `Route not found: ${req.method} ${req.path}`,
details: {},
requestId: req.requestId,
});
}
export function errorHandler(error, req, res, _next) {
const envelope = toErrorEnvelope(error, req.requestId);
if (req.log) {
req.log.error(
{
errCode: envelope.body.code,
status: envelope.status,
details: envelope.body.details,
},
envelope.body.message
);
}
res.status(envelope.status).json(envelope.body);
}

View File

@@ -0,0 +1,10 @@
import { AppError } from '../lib/errors.js';
export function requireIdempotencyKey(req, _res, next) {
const idempotencyKey = req.get('Idempotency-Key');
if (!idempotencyKey) {
return next(new AppError('MISSING_IDEMPOTENCY_KEY', 'Missing Idempotency-Key header', 400));
}
req.idempotencyKey = idempotencyKey;
return next();
}

View File

@@ -0,0 +1,9 @@
import { randomUUID } from 'node:crypto';
export function requestContext(req, res, next) {
const incoming = req.get('X-Request-Id');
req.requestId = incoming || randomUUID();
res.setHeader('X-Request-Id', req.requestId);
res.locals.startedAt = Date.now();
next();
}

View File

@@ -0,0 +1,113 @@
import { Router } from 'express';
import { AppError } from '../lib/errors.js';
import { requireAuth, requirePolicy } from '../middleware/auth.js';
import { requireIdempotencyKey } from '../middleware/idempotency.js';
import { buildIdempotencyKey, readIdempotentResult, writeIdempotentResult } from '../services/idempotency-store.js';
import { commandBaseSchema } from '../contracts/commands/command-base.js';
function parseBody(body) {
const parsed = commandBaseSchema.safeParse(body || {});
if (!parsed.success) {
throw new AppError('VALIDATION_ERROR', 'Invalid command payload', 400, {
issues: parsed.error.issues,
});
}
return parsed.data;
}
function createCommandResponse(route, requestId, idempotencyKey) {
return {
accepted: true,
route,
commandId: `${route}:${Date.now()}`,
idempotencyKey,
requestId,
};
}
function buildCommandHandler(policyAction, policyResource) {
return async (req, res, next) => {
try {
parseBody(req.body);
const route = `${req.baseUrl}${req.route.path}`;
const compositeKey = buildIdempotencyKey({
userId: req.actor.uid,
route,
idempotencyKey: req.idempotencyKey,
});
const existing = await readIdempotentResult(compositeKey);
if (existing) {
return res.status(existing.statusCode).json(existing.payload);
}
const payload = createCommandResponse(route, req.requestId, req.idempotencyKey);
const persisted = await writeIdempotentResult({
compositeKey,
userId: req.actor.uid,
route,
idempotencyKey: req.idempotencyKey,
payload,
statusCode: 200,
});
return res.status(persisted.statusCode).json(persisted.payload);
} catch (error) {
return next(error);
}
};
}
export function createCommandsRouter() {
const router = Router();
router.post(
'/orders/create',
requireAuth,
requireIdempotencyKey,
requirePolicy('orders.create', 'order'),
buildCommandHandler('orders.create', 'order')
);
router.post(
'/orders/:orderId/update',
requireAuth,
requireIdempotencyKey,
requirePolicy('orders.update', 'order'),
buildCommandHandler('orders.update', 'order')
);
router.post(
'/orders/:orderId/cancel',
requireAuth,
requireIdempotencyKey,
requirePolicy('orders.cancel', 'order'),
buildCommandHandler('orders.cancel', 'order')
);
router.post(
'/shifts/:shiftId/change-status',
requireAuth,
requireIdempotencyKey,
requirePolicy('shifts.change-status', 'shift'),
buildCommandHandler('shifts.change-status', 'shift')
);
router.post(
'/shifts/:shiftId/assign-staff',
requireAuth,
requireIdempotencyKey,
requirePolicy('shifts.assign-staff', 'shift'),
buildCommandHandler('shifts.assign-staff', 'shift')
);
router.post(
'/shifts/:shiftId/accept',
requireAuth,
requireIdempotencyKey,
requirePolicy('shifts.accept', 'shift'),
buildCommandHandler('shifts.accept', 'shift')
);
return router;
}

View File

@@ -0,0 +1,15 @@
import { Router } from 'express';
export const healthRouter = Router();
function healthHandler(req, res) {
res.status(200).json({
ok: true,
service: 'krow-command-api',
version: process.env.SERVICE_VERSION || 'dev',
requestId: req.requestId,
});
}
healthRouter.get('/health', healthHandler);
healthRouter.get('/healthz', healthHandler);

View File

@@ -0,0 +1,9 @@
import { createApp } from './app.js';
const port = Number(process.env.PORT || 8080);
const app = createApp();
app.listen(port, () => {
// eslint-disable-next-line no-console
console.log(`krow-command-api listening on port ${port}`);
});

View File

@@ -0,0 +1,13 @@
import { applicationDefault, getApps, initializeApp } from 'firebase-admin/app';
import { getAuth } from 'firebase-admin/auth';
function ensureAdminApp() {
if (getApps().length === 0) {
initializeApp({ credential: applicationDefault() });
}
}
export async function verifyFirebaseToken(token) {
ensureAdminApp();
return getAuth().verifyIdToken(token);
}

View File

@@ -0,0 +1,208 @@
import { Pool } from 'pg';
const DEFAULT_TTL_SECONDS = Number.parseInt(process.env.IDEMPOTENCY_TTL_SECONDS || '86400', 10);
const CLEANUP_EVERY_OPS = Number.parseInt(process.env.IDEMPOTENCY_CLEANUP_EVERY_OPS || '100', 10);
const memoryStore = new Map();
let adapterPromise = null;
function shouldUseSqlStore() {
const mode = (process.env.IDEMPOTENCY_STORE || '').toLowerCase();
if (mode === 'memory') {
return false;
}
if (mode === 'sql') {
return true;
}
return Boolean(process.env.IDEMPOTENCY_DATABASE_URL);
}
function gcExpiredMemoryRecords(now = Date.now()) {
for (const [key, value] of memoryStore.entries()) {
if (value.expiresAt <= now) {
memoryStore.delete(key);
}
}
}
function createMemoryAdapter() {
return {
async read(compositeKey) {
gcExpiredMemoryRecords();
return memoryStore.get(compositeKey) || null;
},
async write({
compositeKey,
payload,
statusCode = 200,
}) {
const now = Date.now();
const existing = memoryStore.get(compositeKey);
if (existing && existing.expiresAt > now) {
return existing;
}
const record = {
payload,
statusCode,
createdAt: now,
expiresAt: now + (DEFAULT_TTL_SECONDS * 1000),
};
memoryStore.set(compositeKey, record);
return record;
},
};
}
async function createSqlAdapter() {
const connectionString = process.env.IDEMPOTENCY_DATABASE_URL;
if (!connectionString) {
throw new Error('IDEMPOTENCY_DATABASE_URL is required for sql idempotency store');
}
const pool = new Pool({
connectionString,
max: Number.parseInt(process.env.IDEMPOTENCY_DB_POOL_MAX || '5', 10),
});
await pool.query(`
CREATE TABLE IF NOT EXISTS command_idempotency (
composite_key TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
route TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
status_code INTEGER NOT NULL,
response_payload JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
expires_at TIMESTAMPTZ NOT NULL
);
`);
await pool.query(`
CREATE INDEX IF NOT EXISTS idx_command_idempotency_expires_at
ON command_idempotency (expires_at);
`);
let opCount = 0;
async function maybeCleanupExpiredRows() {
opCount += 1;
if (CLEANUP_EVERY_OPS <= 0 || opCount % CLEANUP_EVERY_OPS !== 0) {
return;
}
await pool.query('DELETE FROM command_idempotency WHERE expires_at <= NOW()');
}
function mapRow(row) {
return {
statusCode: row.status_code,
payload: row.response_payload,
};
}
return {
async read(compositeKey) {
await maybeCleanupExpiredRows();
const result = await pool.query(
`
SELECT status_code, response_payload
FROM command_idempotency
WHERE composite_key = $1
AND expires_at > NOW()
`,
[compositeKey]
);
if (result.rowCount === 0) {
return null;
}
return mapRow(result.rows[0]);
},
async write({
compositeKey,
userId,
route,
idempotencyKey,
payload,
statusCode = 200,
}) {
await maybeCleanupExpiredRows();
const expiresAt = new Date(Date.now() + (DEFAULT_TTL_SECONDS * 1000));
const payloadJson = JSON.stringify(payload);
await pool.query(
`
INSERT INTO command_idempotency (
composite_key,
user_id,
route,
idempotency_key,
status_code,
response_payload,
expires_at
)
VALUES ($1, $2, $3, $4, $5, $6::jsonb, $7)
ON CONFLICT (composite_key) DO NOTHING
`,
[compositeKey, userId, route, idempotencyKey, statusCode, payloadJson, expiresAt]
);
const existingResult = await pool.query(
`
SELECT status_code, response_payload
FROM command_idempotency
WHERE composite_key = $1
AND expires_at > NOW()
`,
[compositeKey]
);
if (existingResult.rowCount === 0) {
throw new Error('Idempotency write failed to persist or recover existing record');
}
return mapRow(existingResult.rows[0]);
},
};
}
async function getAdapter() {
if (!adapterPromise) {
adapterPromise = shouldUseSqlStore()
? createSqlAdapter()
: Promise.resolve(createMemoryAdapter());
}
return adapterPromise;
}
export function buildIdempotencyKey({ userId, route, idempotencyKey }) {
return `${userId}:${route}:${idempotencyKey}`;
}
export async function readIdempotentResult(compositeKey) {
const adapter = await getAdapter();
return adapter.read(compositeKey);
}
export async function writeIdempotentResult({
compositeKey,
userId,
route,
idempotencyKey,
payload,
statusCode = 200,
}) {
const adapter = await getAdapter();
return adapter.write({
compositeKey,
userId,
route,
idempotencyKey,
payload,
statusCode,
});
}
export function __resetIdempotencyStoreForTests() {
memoryStore.clear();
adapterPromise = null;
}

View File

@@ -0,0 +1,5 @@
export function can(action, resource, actor) {
void action;
void resource;
return Boolean(actor?.uid);
}

View File

@@ -0,0 +1,54 @@
import test, { beforeEach } from 'node:test';
import assert from 'node:assert/strict';
import request from 'supertest';
import { createApp } from '../src/app.js';
import { __resetIdempotencyStoreForTests } from '../src/services/idempotency-store.js';
process.env.AUTH_BYPASS = 'true';
beforeEach(() => {
process.env.IDEMPOTENCY_STORE = 'memory';
delete process.env.IDEMPOTENCY_DATABASE_URL;
__resetIdempotencyStoreForTests();
});
test('GET /healthz returns healthy response', async () => {
const app = createApp();
const res = await request(app).get('/healthz');
assert.equal(res.status, 200);
assert.equal(res.body.ok, true);
assert.equal(typeof res.body.requestId, 'string');
});
test('command route requires idempotency key', async () => {
const app = createApp();
const res = await request(app)
.post('/commands/orders/create')
.set('Authorization', 'Bearer test-token')
.send({ payload: {} });
assert.equal(res.status, 400);
assert.equal(res.body.code, 'MISSING_IDEMPOTENCY_KEY');
});
test('command route is idempotent by key', async () => {
const app = createApp();
const first = await request(app)
.post('/commands/orders/create')
.set('Authorization', 'Bearer test-token')
.set('Idempotency-Key', 'abc-123')
.send({ payload: { order: 'x' } });
const second = await request(app)
.post('/commands/orders/create')
.set('Authorization', 'Bearer test-token')
.set('Idempotency-Key', 'abc-123')
.send({ payload: { order: 'x' } });
assert.equal(first.status, 200);
assert.equal(second.status, 200);
assert.equal(first.body.commandId, second.body.commandId);
assert.equal(first.body.idempotencyKey, 'abc-123');
});

View File

@@ -0,0 +1,56 @@
import test, { beforeEach } from 'node:test';
import assert from 'node:assert/strict';
import {
__resetIdempotencyStoreForTests,
buildIdempotencyKey,
readIdempotentResult,
writeIdempotentResult,
} from '../src/services/idempotency-store.js';
beforeEach(() => {
process.env.IDEMPOTENCY_STORE = 'memory';
delete process.env.IDEMPOTENCY_DATABASE_URL;
__resetIdempotencyStoreForTests();
});
test('buildIdempotencyKey composes user route and client key', () => {
const key = buildIdempotencyKey({
userId: 'user-1',
route: '/commands/orders/create',
idempotencyKey: 'req-abc',
});
assert.equal(key, 'user-1:/commands/orders/create:req-abc');
});
test('memory idempotency store returns existing payload for duplicate key', async () => {
const compositeKey = buildIdempotencyKey({
userId: 'user-1',
route: '/commands/orders/create',
idempotencyKey: 'req-abc',
});
const first = await writeIdempotentResult({
compositeKey,
userId: 'user-1',
route: '/commands/orders/create',
idempotencyKey: 'req-abc',
payload: { accepted: true, commandId: 'c-1' },
statusCode: 200,
});
const second = await writeIdempotentResult({
compositeKey,
userId: 'user-1',
route: '/commands/orders/create',
idempotencyKey: 'req-abc',
payload: { accepted: true, commandId: 'c-2' },
statusCode: 200,
});
const read = await readIdempotentResult(compositeKey);
assert.equal(first.payload.commandId, 'c-1');
assert.equal(second.payload.commandId, 'c-1');
assert.equal(read.payload.commandId, 'c-1');
});

View File

@@ -0,0 +1,13 @@
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY src ./src
ENV PORT=8080
EXPOSE 8080
CMD ["node", "src/server.js"]

2962
backend/core-api/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,26 @@
{
"name": "@krow/core-api",
"version": "0.1.0",
"private": true,
"type": "module",
"engines": {
"node": ">=20"
},
"scripts": {
"start": "node src/server.js",
"test": "node --test"
},
"dependencies": {
"@google-cloud/storage": "^7.16.0",
"express": "^4.21.2",
"firebase-admin": "^13.0.2",
"google-auth-library": "^9.15.1",
"multer": "^2.0.2",
"pino": "^9.6.0",
"pino-http": "^10.3.0",
"zod": "^3.24.2"
},
"devDependencies": {
"supertest": "^7.0.0"
}
}

View File

@@ -0,0 +1,31 @@
import express from 'express';
import pino from 'pino';
import pinoHttp from 'pino-http';
import { requestContext } from './middleware/request-context.js';
import { errorHandler, notFoundHandler } from './middleware/error-handler.js';
import { healthRouter } from './routes/health.js';
import { createCoreRouter, createLegacyCoreRouter } from './routes/core.js';
const logger = pino({ level: process.env.LOG_LEVEL || 'info' });
export function createApp() {
const app = express();
app.use(requestContext);
app.use(
pinoHttp({
logger,
customProps: (req) => ({ requestId: req.requestId }),
})
);
app.use(express.json({ limit: '2mb' }));
app.use(healthRouter);
app.use('/core', createCoreRouter());
app.use('/', createLegacyCoreRouter());
app.use(notFoundHandler);
app.use(errorHandler);
return app;
}

View File

@@ -0,0 +1,6 @@
import { z } from 'zod';
export const createSignedUrlSchema = z.object({
fileUri: z.string().startsWith('gs://', 'fileUri must start with gs://'),
expiresInSeconds: z.number().int().min(60).max(3600).optional(),
});

View File

@@ -0,0 +1,10 @@
import { z } from 'zod';
export const createVerificationSchema = z.object({
type: z.enum(['attire', 'government_id', 'certification']),
subjectType: z.string().min(1).max(80).optional(),
subjectId: z.string().min(1).max(120).optional(),
fileUri: z.string().startsWith('gs://', 'fileUri must start with gs://'),
rules: z.record(z.any()).optional().default({}),
metadata: z.record(z.any()).optional().default({}),
});

View File

@@ -0,0 +1,7 @@
import { z } from 'zod';
export const invokeLlmSchema = z.object({
prompt: z.string().min(1).max(12000),
responseJsonSchema: z.record(z.any()),
fileUrls: z.array(z.string().url()).optional(),
});

View File

@@ -0,0 +1,7 @@
import { z } from 'zod';
export const reviewVerificationSchema = z.object({
decision: z.enum(['APPROVED', 'REJECTED']),
note: z.string().max(1000).optional().default(''),
reasonCode: z.string().max(100).optional().default('MANUAL_REVIEW'),
});

View File

@@ -0,0 +1,26 @@
export class AppError extends Error {
constructor(code, message, status = 400, details = {}) {
super(message);
this.name = 'AppError';
this.code = code;
this.status = status;
this.details = details;
}
}
export function toErrorEnvelope(error, requestId) {
const status = error?.status && Number.isInteger(error.status) ? error.status : 500;
const code = error?.code || 'INTERNAL_ERROR';
const message = error?.message || 'Unexpected error';
const details = error?.details || {};
return {
status,
body: {
code,
message,
details,
requestId,
},
};
}

View File

@@ -0,0 +1,45 @@
import { AppError } from '../lib/errors.js';
import { can } from '../services/policy.js';
import { verifyFirebaseToken } from '../services/firebase-auth.js';
function getBearerToken(header) {
if (!header) return null;
const [scheme, token] = header.split(' ');
if (!scheme || scheme.toLowerCase() !== 'bearer' || !token) return null;
return token;
}
export async function requireAuth(req, _res, next) {
try {
const token = getBearerToken(req.get('Authorization'));
if (!token) {
throw new AppError('UNAUTHENTICATED', 'Missing bearer token', 401);
}
if (process.env.AUTH_BYPASS === 'true') {
req.actor = { uid: 'test-user', email: 'test@krow.local', role: 'TEST' };
return next();
}
const decoded = await verifyFirebaseToken(token);
req.actor = {
uid: decoded.uid,
email: decoded.email || null,
role: decoded.role || null,
};
return next();
} catch (error) {
if (error instanceof AppError) return next(error);
return next(new AppError('UNAUTHENTICATED', 'Token verification failed', 401));
}
}
export function requirePolicy(action, resource) {
return (req, _res, next) => {
if (!can(action, resource, req.actor)) {
return next(new AppError('FORBIDDEN', 'Not allowed to perform this action', 403));
}
return next();
};
}

View File

@@ -0,0 +1,28 @@
import { toErrorEnvelope } from '../lib/errors.js';
export function notFoundHandler(req, res) {
res.status(404).json({
code: 'NOT_FOUND',
message: `Route not found: ${req.method} ${req.path}`,
details: {},
requestId: req.requestId,
});
}
export function errorHandler(error, req, res, _next) {
const envelope = toErrorEnvelope(error, req.requestId);
if (envelope.status === 429 && envelope.body.details?.retryAfterSeconds) {
res.set('Retry-After', String(envelope.body.details.retryAfterSeconds));
}
if (req.log) {
req.log.error(
{
errCode: envelope.body.code,
status: envelope.status,
details: envelope.body.details,
},
envelope.body.message
);
}
res.status(envelope.status).json(envelope.body);
}

View File

@@ -0,0 +1,9 @@
import { randomUUID } from 'node:crypto';
export function requestContext(req, res, next) {
const incoming = req.get('X-Request-Id');
req.requestId = incoming || randomUUID();
res.setHeader('X-Request-Id', req.requestId);
res.locals.startedAt = Date.now();
next();
}

View File

@@ -0,0 +1,287 @@
import { Router } from 'express';
import multer from 'multer';
import { z } from 'zod';
import { AppError } from '../lib/errors.js';
import { requireAuth, requirePolicy } from '../middleware/auth.js';
import { createSignedUrlSchema } from '../contracts/core/create-signed-url.js';
import { createVerificationSchema } from '../contracts/core/create-verification.js';
import { invokeLlmSchema } from '../contracts/core/invoke-llm.js';
import { reviewVerificationSchema } from '../contracts/core/review-verification.js';
import { invokeVertexModel } from '../services/llm.js';
import { checkLlmRateLimit } from '../services/llm-rate-limit.js';
import {
ensureFileExistsForActor,
generateReadSignedUrl,
uploadToGcs,
validateFileUriAccess,
} from '../services/storage.js';
import {
createVerificationJob,
getVerificationJob,
retryVerificationJob,
reviewVerificationJob,
} from '../services/verification-jobs.js';
const DEFAULT_MAX_FILE_BYTES = 10 * 1024 * 1024;
const DEFAULT_MAX_SIGNED_URL_SECONDS = 900;
const ALLOWED_FILE_TYPES = new Set(['application/pdf', 'image/jpeg', 'image/jpg', 'image/png']);
const upload = multer({
storage: multer.memoryStorage(),
limits: {
fileSize: Number(process.env.MAX_UPLOAD_BYTES || DEFAULT_MAX_FILE_BYTES),
},
});
const uploadMetaSchema = z.object({
category: z.string().max(80).optional(),
visibility: z.enum(['public', 'private']).optional(),
});
function mockSignedUrl(fileUri, expiresInSeconds) {
const encoded = encodeURIComponent(fileUri);
const expiresAt = new Date(Date.now() + expiresInSeconds * 1000).toISOString();
return {
signedUrl: `https://storage.googleapis.com/mock-signed-url/${encoded}?expires=${expiresInSeconds}`,
expiresAt,
};
}
function useMockSignedUrl() {
return process.env.SIGNED_URL_MOCK !== 'false';
}
function useMockUpload() {
return process.env.UPLOAD_MOCK !== 'false';
}
function requireVerificationFileExists() {
return process.env.VERIFICATION_REQUIRE_FILE_EXISTS !== 'false';
}
function parseBody(schema, body) {
const parsed = schema.safeParse(body);
if (!parsed.success) {
throw new AppError('VALIDATION_ERROR', 'Invalid request payload', 400, {
issues: parsed.error.issues,
});
}
return parsed.data;
}
async function handleUploadFile(req, res, next) {
try {
const file = req.file;
if (!file) {
throw new AppError('INVALID_FILE', 'Missing file in multipart form data', 400);
}
if (!ALLOWED_FILE_TYPES.has(file.mimetype)) {
throw new AppError('INVALID_FILE_TYPE', `Unsupported file type: ${file.mimetype}`, 400);
}
const maxFileSize = Number(process.env.MAX_UPLOAD_BYTES || DEFAULT_MAX_FILE_BYTES);
if (file.size > maxFileSize) {
throw new AppError('FILE_TOO_LARGE', `File exceeds ${maxFileSize} bytes`, 400);
}
const meta = parseBody(uploadMetaSchema, req.body || {});
const visibility = meta.visibility || 'private';
const bucket = visibility === 'public'
? process.env.PUBLIC_BUCKET || 'krow-workforce-dev-public'
: process.env.PRIVATE_BUCKET || 'krow-workforce-dev-private';
const safeName = file.originalname.replace(/[^a-zA-Z0-9._-]/g, '_');
const objectPath = `uploads/${req.actor.uid}/${Date.now()}_${safeName}`;
const fileUri = `gs://${bucket}/${objectPath}`;
if (!useMockUpload()) {
await uploadToGcs({
bucket,
objectPath,
contentType: file.mimetype,
buffer: file.buffer,
});
}
res.status(200).json({
fileUri,
contentType: file.mimetype,
size: file.size,
bucket,
path: objectPath,
requestId: req.requestId,
});
} catch (error) {
if (error?.code === 'LIMIT_FILE_SIZE') {
return next(new AppError('FILE_TOO_LARGE', 'File exceeds upload limit', 400));
}
return next(error);
}
}
async function handleCreateSignedUrl(req, res, next) {
try {
const payload = parseBody(createSignedUrlSchema, req.body || {});
const expiresInSeconds = payload.expiresInSeconds || 300;
const maxSignedUrlSeconds = Number.parseInt(
process.env.MAX_SIGNED_URL_SECONDS || `${DEFAULT_MAX_SIGNED_URL_SECONDS}`,
10
);
if (expiresInSeconds > maxSignedUrlSeconds) {
throw new AppError('VALIDATION_ERROR', `expiresInSeconds must be <= ${maxSignedUrlSeconds}`, 400);
}
const signed = useMockSignedUrl()
? (() => {
validateFileUriAccess({
fileUri: payload.fileUri,
actorUid: req.actor.uid,
});
return mockSignedUrl(payload.fileUri, expiresInSeconds);
})()
: await generateReadSignedUrl({
fileUri: payload.fileUri,
actorUid: req.actor.uid,
expiresInSeconds,
});
res.status(200).json({
...signed,
requestId: req.requestId,
});
} catch (error) {
return next(error);
}
}
async function handleInvokeLlm(req, res, next) {
try {
const payload = parseBody(invokeLlmSchema, req.body || {});
const rateLimit = checkLlmRateLimit({ uid: req.actor.uid });
if (!rateLimit.allowed) {
throw new AppError('RATE_LIMITED', 'Too many model requests. Please retry shortly.', 429, {
retryAfterSeconds: rateLimit.retryAfterSeconds,
});
}
const startedAt = Date.now();
if (process.env.LLM_MOCK === 'false') {
const llmResult = await invokeVertexModel({
prompt: payload.prompt,
responseJsonSchema: payload.responseJsonSchema,
fileUrls: payload.fileUrls,
});
return res.status(200).json({
result: llmResult.result,
model: llmResult.model,
latencyMs: Date.now() - startedAt,
requestId: req.requestId,
});
}
return res.status(200).json({
result: {
summary: 'Mock model response. Replace with Vertex AI integration.',
inputPromptSize: payload.prompt.length,
},
model: process.env.LLM_MODEL || 'vertexai/gemini-mock',
latencyMs: Date.now() - startedAt,
requestId: req.requestId,
});
} catch (error) {
return next(error);
}
}
async function handleCreateVerification(req, res, next) {
try {
const payload = parseBody(createVerificationSchema, req.body || {});
validateFileUriAccess({
fileUri: payload.fileUri,
actorUid: req.actor.uid,
});
if (requireVerificationFileExists() && !useMockUpload()) {
await ensureFileExistsForActor({
fileUri: payload.fileUri,
actorUid: req.actor.uid,
});
}
const created = createVerificationJob({
actorUid: req.actor.uid,
payload,
});
return res.status(202).json({
...created,
requestId: req.requestId,
});
} catch (error) {
return next(error);
}
}
async function handleGetVerification(req, res, next) {
try {
const verificationId = req.params.verificationId;
const job = getVerificationJob(verificationId, req.actor.uid);
return res.status(200).json({
...job,
requestId: req.requestId,
});
} catch (error) {
return next(error);
}
}
async function handleReviewVerification(req, res, next) {
try {
const verificationId = req.params.verificationId;
const payload = parseBody(reviewVerificationSchema, req.body || {});
const updated = reviewVerificationJob(verificationId, req.actor.uid, payload);
return res.status(200).json({
...updated,
requestId: req.requestId,
});
} catch (error) {
return next(error);
}
}
async function handleRetryVerification(req, res, next) {
try {
const verificationId = req.params.verificationId;
const updated = retryVerificationJob(verificationId, req.actor.uid);
return res.status(202).json({
...updated,
requestId: req.requestId,
});
} catch (error) {
return next(error);
}
}
export function createCoreRouter() {
const router = Router();
router.post('/upload-file', requireAuth, requirePolicy('core.upload', 'file'), upload.single('file'), handleUploadFile);
router.post('/create-signed-url', requireAuth, requirePolicy('core.sign-url', 'file'), handleCreateSignedUrl);
router.post('/invoke-llm', requireAuth, requirePolicy('core.invoke-llm', 'model'), handleInvokeLlm);
router.post('/verifications', requireAuth, requirePolicy('core.verification.create', 'verification'), handleCreateVerification);
router.get('/verifications/:verificationId', requireAuth, requirePolicy('core.verification.read', 'verification'), handleGetVerification);
router.post('/verifications/:verificationId/review', requireAuth, requirePolicy('core.verification.review', 'verification'), handleReviewVerification);
router.post('/verifications/:verificationId/retry', requireAuth, requirePolicy('core.verification.retry', 'verification'), handleRetryVerification);
return router;
}
export function createLegacyCoreRouter() {
const router = Router();
router.post('/uploadFile', requireAuth, requirePolicy('core.upload', 'file'), upload.single('file'), handleUploadFile);
router.post('/createSignedUrl', requireAuth, requirePolicy('core.sign-url', 'file'), handleCreateSignedUrl);
router.post('/invokeLLM', requireAuth, requirePolicy('core.invoke-llm', 'model'), handleInvokeLlm);
return router;
}

View File

@@ -0,0 +1,15 @@
import { Router } from 'express';
export const healthRouter = Router();
function healthHandler(req, res) {
res.status(200).json({
ok: true,
service: 'krow-core-api',
version: process.env.SERVICE_VERSION || 'dev',
requestId: req.requestId,
});
}
healthRouter.get('/health', healthHandler);
healthRouter.get('/healthz', healthHandler);

View File

@@ -0,0 +1,9 @@
import { createApp } from './app.js';
const port = Number(process.env.PORT || 8080);
const app = createApp();
app.listen(port, () => {
// eslint-disable-next-line no-console
console.log(`krow-core-api listening on port ${port}`);
});

View File

@@ -0,0 +1,13 @@
import { applicationDefault, getApps, initializeApp } from 'firebase-admin/app';
import { getAuth } from 'firebase-admin/auth';
function ensureAdminApp() {
if (getApps().length === 0) {
initializeApp({ credential: applicationDefault() });
}
}
export async function verifyFirebaseToken(token) {
ensureAdminApp();
return getAuth().verifyIdToken(token);
}

View File

@@ -0,0 +1,41 @@
const counters = new Map();
function currentWindowKey(now = Date.now()) {
return Math.floor(now / 60000);
}
function perMinuteLimit() {
return Number.parseInt(process.env.LLM_RATE_LIMIT_PER_MINUTE || '20', 10);
}
export function checkLlmRateLimit({ uid, now = Date.now() }) {
const limit = perMinuteLimit();
if (!Number.isFinite(limit) || limit <= 0) {
return { allowed: true, remaining: null, retryAfterSeconds: 0 };
}
const windowKey = currentWindowKey(now);
const record = counters.get(uid);
if (!record || record.windowKey !== windowKey) {
counters.set(uid, { windowKey, count: 1 });
return { allowed: true, remaining: limit - 1, retryAfterSeconds: 0 };
}
if (record.count >= limit) {
const retryAfterSeconds = (windowKey + 1) * 60 - Math.floor(now / 1000);
return {
allowed: false,
remaining: 0,
retryAfterSeconds: Math.max(1, retryAfterSeconds),
};
}
record.count += 1;
counters.set(uid, record);
return { allowed: true, remaining: limit - record.count, retryAfterSeconds: 0 };
}
export function __resetLlmRateLimitForTests() {
counters.clear();
}

View File

@@ -0,0 +1,145 @@
import { GoogleAuth } from 'google-auth-library';
import { AppError } from '../lib/errors.js';
function buildVertexConfig() {
const project = process.env.GCP_PROJECT_ID || process.env.GOOGLE_CLOUD_PROJECT;
const location = process.env.LLM_LOCATION || process.env.BACKEND_REGION || 'us-central1';
if (!project) {
throw new AppError('MODEL_FAILED', 'GCP project is required for model invocation', 500);
}
return {
project,
location,
};
}
function withTimeout(promise, timeoutMs) {
return Promise.race([
promise,
new Promise((_, reject) => {
setTimeout(() => {
reject(new AppError('MODEL_TIMEOUT', `Model request exceeded ${timeoutMs}ms`, 504));
}, timeoutMs);
}),
]);
}
function toTextFromCandidate(candidate) {
if (!candidate?.content?.parts) {
return '';
}
return candidate.content.parts
.map((part) => part?.text || '')
.join('')
.trim();
}
function withJsonSchemaInstruction(prompt, responseJsonSchema) {
const schemaText = JSON.stringify(responseJsonSchema);
return `${prompt}\n\nRespond with strict JSON only. Follow this schema exactly:\n${schemaText}`;
}
function guessMimeTypeFromUri(fileUri) {
const path = fileUri.split('?')[0].toLowerCase();
if (path.endsWith('.jpg') || path.endsWith('.jpeg')) return 'image/jpeg';
if (path.endsWith('.png')) return 'image/png';
if (path.endsWith('.pdf')) return 'application/pdf';
return 'application/octet-stream';
}
function buildMultimodalParts(prompt, fileUris = []) {
const parts = [{ text: prompt }];
for (const fileUri of fileUris) {
parts.push({
fileData: {
fileUri,
mimeType: guessMimeTypeFromUri(fileUri),
},
});
}
return parts;
}
async function callVertexJsonModel({ model, timeoutMs, parts }) {
const { project, location } = buildVertexConfig();
const url = `https://${location}-aiplatform.googleapis.com/v1/projects/${project}/locations/${location}/publishers/google/models/${model}:generateContent`;
const auth = new GoogleAuth({
scopes: ['https://www.googleapis.com/auth/cloud-platform'],
});
let response;
try {
const client = await auth.getClient();
response = await withTimeout(
client.request({
url,
method: 'POST',
data: {
contents: [{ role: 'user', parts }],
generationConfig: {
temperature: 0.2,
responseMimeType: 'application/json',
},
},
}),
timeoutMs
);
} catch (error) {
if (error instanceof AppError) {
throw error;
}
throw new AppError('MODEL_FAILED', 'Model invocation failed', 502);
}
const text = toTextFromCandidate(response?.data?.candidates?.[0]);
if (!text) {
throw new AppError('MODEL_FAILED', 'Model returned empty response', 502);
}
try {
return {
model,
result: JSON.parse(text),
};
} catch {
return {
model,
result: {
raw: text,
},
};
}
}
export async function invokeVertexModel({ prompt, responseJsonSchema, fileUrls = [] }) {
const model = process.env.LLM_MODEL || 'gemini-2.0-flash-001';
const timeoutMs = Number.parseInt(process.env.LLM_TIMEOUT_MS || '20000', 10);
const promptWithSchema = withJsonSchemaInstruction(prompt, responseJsonSchema);
const fileContext = fileUrls.length > 0 ? `\nFiles:\n${fileUrls.join('\n')}` : '';
return callVertexJsonModel({
model,
timeoutMs,
parts: [{ text: `${promptWithSchema}${fileContext}` }],
});
}
export async function invokeVertexMultimodalModel({
prompt,
responseJsonSchema,
fileUris = [],
model,
timeoutMs,
}) {
const resolvedModel = model || process.env.LLM_MODEL || 'gemini-2.0-flash-001';
const resolvedTimeoutMs = Number.parseInt(
`${timeoutMs || process.env.LLM_TIMEOUT_MS || '20000'}`,
10
);
const promptWithSchema = withJsonSchemaInstruction(prompt, responseJsonSchema);
return callVertexJsonModel({
model: resolvedModel,
timeoutMs: resolvedTimeoutMs,
parts: buildMultimodalParts(promptWithSchema, fileUris),
});
}

View File

@@ -0,0 +1,5 @@
export function can(action, resource, actor) {
void action;
void resource;
return Boolean(actor?.uid);
}

View File

@@ -0,0 +1,83 @@
import { Storage } from '@google-cloud/storage';
import { AppError } from '../lib/errors.js';
const storage = new Storage();
export function parseGsUri(fileUri) {
if (!fileUri.startsWith('gs://')) {
throw new AppError('VALIDATION_ERROR', 'fileUri must start with gs://', 400);
}
const raw = fileUri.replace('gs://', '');
const slashIndex = raw.indexOf('/');
if (slashIndex <= 0 || slashIndex >= raw.length - 1) {
throw new AppError('VALIDATION_ERROR', 'fileUri must include bucket and object path', 400);
}
return {
bucket: raw.slice(0, slashIndex),
path: raw.slice(slashIndex + 1),
};
}
function allowedBuckets() {
return new Set([
process.env.PUBLIC_BUCKET || 'krow-workforce-dev-public',
process.env.PRIVATE_BUCKET || 'krow-workforce-dev-private',
]);
}
export function validateFileUriAccess({ fileUri, actorUid }) {
const { bucket, path } = parseGsUri(fileUri);
if (!allowedBuckets().has(bucket)) {
throw new AppError('FORBIDDEN', `Bucket not allowed for signing: ${bucket}`, 403);
}
const ownedPrefix = `uploads/${actorUid}/`;
if (!path.startsWith(ownedPrefix)) {
throw new AppError('FORBIDDEN', 'Cannot sign URL for another user path', 403);
}
return { bucket, path };
}
export async function uploadToGcs({ bucket, objectPath, contentType, buffer }) {
const file = storage.bucket(bucket).file(objectPath);
await file.save(buffer, {
resumable: false,
contentType,
metadata: {
cacheControl: 'private, max-age=0',
},
});
}
export async function generateReadSignedUrl({ fileUri, actorUid, expiresInSeconds }) {
const { bucket, path } = validateFileUriAccess({ fileUri, actorUid });
const file = storage.bucket(bucket).file(path);
const [exists] = await file.exists();
if (!exists) {
throw new AppError('NOT_FOUND', 'File not found for signed URL', 404, { fileUri });
}
const expiresAtMs = Date.now() + expiresInSeconds * 1000;
const [signedUrl] = await file.getSignedUrl({
version: 'v4',
action: 'read',
expires: expiresAtMs,
});
return {
signedUrl,
expiresAt: new Date(expiresAtMs).toISOString(),
};
}
export async function ensureFileExistsForActor({ fileUri, actorUid }) {
const { bucket, path } = validateFileUriAccess({ fileUri, actorUid });
const file = storage.bucket(bucket).file(path);
const [exists] = await file.exists();
if (!exists) {
throw new AppError('NOT_FOUND', 'Evidence file not found', 404, { fileUri });
}
}

View File

@@ -0,0 +1,510 @@
import crypto from 'node:crypto';
import { AppError } from '../lib/errors.js';
import { invokeVertexMultimodalModel } from './llm.js';
const jobs = new Map();
export const VerificationStatus = Object.freeze({
PENDING: 'PENDING',
PROCESSING: 'PROCESSING',
AUTO_PASS: 'AUTO_PASS',
AUTO_FAIL: 'AUTO_FAIL',
NEEDS_REVIEW: 'NEEDS_REVIEW',
APPROVED: 'APPROVED',
REJECTED: 'REJECTED',
ERROR: 'ERROR',
});
const MACHINE_TERMINAL_STATUSES = new Set([
VerificationStatus.AUTO_PASS,
VerificationStatus.AUTO_FAIL,
VerificationStatus.NEEDS_REVIEW,
VerificationStatus.ERROR,
]);
const HUMAN_TERMINAL_STATUSES = new Set([
VerificationStatus.APPROVED,
VerificationStatus.REJECTED,
]);
function nowIso() {
return new Date().toISOString();
}
function accessMode() {
return process.env.VERIFICATION_ACCESS_MODE || 'authenticated';
}
function eventRecord({ fromStatus, toStatus, actorType, actorId, details = {} }) {
return {
id: crypto.randomUUID(),
fromStatus,
toStatus,
actorType,
actorId,
details,
createdAt: nowIso(),
};
}
function toPublicJob(job) {
return {
verificationId: job.id,
type: job.type,
subjectType: job.subjectType,
subjectId: job.subjectId,
fileUri: job.fileUri,
status: job.status,
confidence: job.confidence,
reasons: job.reasons,
extracted: job.extracted,
provider: job.provider,
review: job.review,
createdAt: job.createdAt,
updatedAt: job.updatedAt,
};
}
function assertAccess(job, actorUid) {
if (accessMode() === 'authenticated') {
return;
}
if (job.ownerUid !== actorUid) {
throw new AppError('FORBIDDEN', 'Not allowed to access this verification', 403);
}
}
function requireJob(id) {
const job = jobs.get(id);
if (!job) {
throw new AppError('NOT_FOUND', 'Verification not found', 404, { verificationId: id });
}
return job;
}
function normalizeMachineStatus(status) {
if (
status === VerificationStatus.AUTO_PASS
|| status === VerificationStatus.AUTO_FAIL
|| status === VerificationStatus.NEEDS_REVIEW
) {
return status;
}
return VerificationStatus.NEEDS_REVIEW;
}
function clampConfidence(value, fallback = 0.5) {
const parsed = Number(value);
if (!Number.isFinite(parsed)) return fallback;
if (parsed < 0) return 0;
if (parsed > 1) return 1;
return parsed;
}
function asReasonList(reasons, fallback) {
if (Array.isArray(reasons) && reasons.length > 0) {
return reasons.map((item) => `${item}`);
}
return [fallback];
}
function providerTimeoutMs() {
return Number.parseInt(process.env.VERIFICATION_PROVIDER_TIMEOUT_MS || '8000', 10);
}
function attireModel() {
return process.env.VERIFICATION_ATTIRE_MODEL || 'gemini-2.0-flash-lite-001';
}
async function runAttireChecks(job) {
if (process.env.VERIFICATION_ATTIRE_AUTOPASS === 'true') {
return {
status: VerificationStatus.AUTO_PASS,
confidence: 0.8,
reasons: ['Auto-pass mode enabled for attire in dev'],
extracted: {
expected: job.rules,
},
provider: {
name: 'attire-auto-pass',
reference: null,
},
};
}
const attireProvider = process.env.VERIFICATION_ATTIRE_PROVIDER || 'vertex';
if (attireProvider !== 'vertex') {
return {
status: VerificationStatus.NEEDS_REVIEW,
confidence: 0.45,
reasons: [`Attire provider '${attireProvider}' is not supported`],
extracted: {
expected: job.rules,
},
provider: {
name: attireProvider,
reference: null,
},
};
}
try {
const prompt = [
'You are validating worker attire evidence.',
`Rules: ${JSON.stringify(job.rules || {})}`,
'Return AUTO_PASS only when the image clearly matches required attire.',
'Return AUTO_FAIL when the image clearly violates required attire.',
'Return NEEDS_REVIEW when uncertain.',
].join('\n');
const schema = {
type: 'object',
properties: {
status: { type: 'string' },
confidence: { type: 'number' },
reasons: {
type: 'array',
items: { type: 'string' },
},
extracted: {
type: 'object',
additionalProperties: true,
},
},
required: ['status', 'confidence', 'reasons'],
};
const modelOutput = await invokeVertexMultimodalModel({
prompt,
responseJsonSchema: schema,
fileUris: [job.fileUri],
model: attireModel(),
timeoutMs: providerTimeoutMs(),
});
const result = modelOutput?.result || {};
return {
status: normalizeMachineStatus(result.status),
confidence: clampConfidence(result.confidence, 0.6),
reasons: asReasonList(result.reasons, 'Attire check completed'),
extracted: result.extracted || {},
provider: {
name: 'vertex-attire',
reference: modelOutput?.model || attireModel(),
},
};
} catch (error) {
return {
status: VerificationStatus.NEEDS_REVIEW,
confidence: 0.35,
reasons: ['Automatic attire check unavailable, manual review required'],
extracted: {},
provider: {
name: 'vertex-attire',
reference: `error:${error?.code || 'unknown'}`,
},
};
}
}
function getProviderConfig(type) {
if (type === 'government_id') {
return {
name: 'government-id-provider',
url: process.env.VERIFICATION_GOV_ID_PROVIDER_URL,
token: process.env.VERIFICATION_GOV_ID_PROVIDER_TOKEN,
};
}
return {
name: 'certification-provider',
url: process.env.VERIFICATION_CERT_PROVIDER_URL,
token: process.env.VERIFICATION_CERT_PROVIDER_TOKEN,
};
}
async function runThirdPartyChecks(job, type) {
const provider = getProviderConfig(type);
if (!provider.url) {
return {
status: VerificationStatus.NEEDS_REVIEW,
confidence: 0.4,
reasons: [`${provider.name} is not configured`],
extracted: {},
provider: {
name: provider.name,
reference: null,
},
};
}
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), providerTimeoutMs());
try {
const response = await fetch(provider.url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
...(provider.token ? { Authorization: `Bearer ${provider.token}` } : {}),
},
body: JSON.stringify({
type,
subjectType: job.subjectType,
subjectId: job.subjectId,
fileUri: job.fileUri,
rules: job.rules,
metadata: job.metadata,
}),
signal: controller.signal,
});
const bodyText = await response.text();
let body = {};
try {
body = bodyText ? JSON.parse(bodyText) : {};
} catch {
body = {};
}
if (!response.ok) {
return {
status: VerificationStatus.NEEDS_REVIEW,
confidence: 0.35,
reasons: [`${provider.name} returned ${response.status}`],
extracted: {},
provider: {
name: provider.name,
reference: body?.reference || null,
},
};
}
return {
status: normalizeMachineStatus(body.status),
confidence: clampConfidence(body.confidence, 0.6),
reasons: asReasonList(body.reasons, `${provider.name} completed check`),
extracted: body.extracted || {},
provider: {
name: provider.name,
reference: body.reference || null,
},
};
} catch (error) {
const isAbort = error?.name === 'AbortError';
return {
status: VerificationStatus.NEEDS_REVIEW,
confidence: 0.3,
reasons: [
isAbort
? `${provider.name} timeout, manual review required`
: `${provider.name} unavailable, manual review required`,
],
extracted: {},
provider: {
name: provider.name,
reference: null,
},
};
} finally {
clearTimeout(timeout);
}
}
async function runMachineChecks(job) {
if (job.type === 'attire') {
return runAttireChecks(job);
}
if (job.type === 'government_id') {
return runThirdPartyChecks(job, 'government_id');
}
return runThirdPartyChecks(job, 'certification');
}
async function processVerificationJob(id) {
const job = requireJob(id);
if (job.status !== VerificationStatus.PENDING) {
return;
}
const beforeProcessing = job.status;
job.status = VerificationStatus.PROCESSING;
job.updatedAt = nowIso();
job.events.push(
eventRecord({
fromStatus: beforeProcessing,
toStatus: VerificationStatus.PROCESSING,
actorType: 'system',
actorId: 'verification-worker',
})
);
try {
const outcome = await runMachineChecks(job);
if (!MACHINE_TERMINAL_STATUSES.has(outcome.status)) {
throw new Error(`Invalid machine outcome status: ${outcome.status}`);
}
const fromStatus = job.status;
job.status = outcome.status;
job.confidence = outcome.confidence;
job.reasons = outcome.reasons;
job.extracted = outcome.extracted;
job.provider = outcome.provider;
job.updatedAt = nowIso();
job.events.push(
eventRecord({
fromStatus,
toStatus: job.status,
actorType: 'system',
actorId: 'verification-worker',
details: {
confidence: job.confidence,
reasons: job.reasons,
provider: job.provider,
},
})
);
} catch (error) {
const fromStatus = job.status;
job.status = VerificationStatus.ERROR;
job.confidence = null;
job.reasons = [error?.message || 'Verification processing failed'];
job.extracted = {};
job.provider = {
name: 'verification-worker',
reference: null,
};
job.updatedAt = nowIso();
job.events.push(
eventRecord({
fromStatus,
toStatus: VerificationStatus.ERROR,
actorType: 'system',
actorId: 'verification-worker',
details: {
error: error?.message || 'Verification processing failed',
},
})
);
}
}
function queueVerificationProcessing(id) {
setTimeout(() => {
processVerificationJob(id).catch(() => {});
}, 0);
}
export function createVerificationJob({ actorUid, payload }) {
const now = nowIso();
const id = `ver_${crypto.randomUUID()}`;
const job = {
id,
type: payload.type,
subjectType: payload.subjectType || null,
subjectId: payload.subjectId || null,
ownerUid: actorUid,
fileUri: payload.fileUri,
rules: payload.rules || {},
metadata: payload.metadata || {},
status: VerificationStatus.PENDING,
confidence: null,
reasons: [],
extracted: {},
provider: null,
review: null,
createdAt: now,
updatedAt: now,
events: [
eventRecord({
fromStatus: null,
toStatus: VerificationStatus.PENDING,
actorType: 'system',
actorId: actorUid,
}),
],
};
jobs.set(id, job);
queueVerificationProcessing(id);
return toPublicJob(job);
}
export function getVerificationJob(verificationId, actorUid) {
const job = requireJob(verificationId);
assertAccess(job, actorUid);
return toPublicJob(job);
}
export function reviewVerificationJob(verificationId, actorUid, review) {
const job = requireJob(verificationId);
assertAccess(job, actorUid);
if (HUMAN_TERMINAL_STATUSES.has(job.status)) {
throw new AppError('CONFLICT', 'Verification already finalized', 409, {
verificationId,
status: job.status,
});
}
const fromStatus = job.status;
job.status = review.decision;
job.review = {
decision: review.decision,
reviewedBy: actorUid,
reviewedAt: nowIso(),
note: review.note || '',
reasonCode: review.reasonCode || 'MANUAL_REVIEW',
};
job.updatedAt = nowIso();
job.events.push(
eventRecord({
fromStatus,
toStatus: job.status,
actorType: 'reviewer',
actorId: actorUid,
details: {
reasonCode: job.review.reasonCode,
},
})
);
return toPublicJob(job);
}
export function retryVerificationJob(verificationId, actorUid) {
const job = requireJob(verificationId);
assertAccess(job, actorUid);
if (job.status === VerificationStatus.PROCESSING) {
throw new AppError('CONFLICT', 'Cannot retry while verification is processing', 409, {
verificationId,
});
}
const fromStatus = job.status;
job.status = VerificationStatus.PENDING;
job.confidence = null;
job.reasons = [];
job.extracted = {};
job.provider = null;
job.review = null;
job.updatedAt = nowIso();
job.events.push(
eventRecord({
fromStatus,
toStatus: VerificationStatus.PENDING,
actorType: 'reviewer',
actorId: actorUid,
details: {
retried: true,
},
})
);
queueVerificationProcessing(verificationId);
return toPublicJob(job);
}
export function __resetVerificationJobsForTests() {
jobs.clear();
}

View File

@@ -0,0 +1,246 @@
import test, { beforeEach } from 'node:test';
import assert from 'node:assert/strict';
import request from 'supertest';
import { createApp } from '../src/app.js';
import { __resetLlmRateLimitForTests } from '../src/services/llm-rate-limit.js';
import { __resetVerificationJobsForTests } from '../src/services/verification-jobs.js';
beforeEach(() => {
process.env.AUTH_BYPASS = 'true';
process.env.LLM_MOCK = 'true';
process.env.SIGNED_URL_MOCK = 'true';
process.env.UPLOAD_MOCK = 'true';
process.env.MAX_SIGNED_URL_SECONDS = '900';
process.env.LLM_RATE_LIMIT_PER_MINUTE = '20';
process.env.VERIFICATION_REQUIRE_FILE_EXISTS = 'false';
process.env.VERIFICATION_ACCESS_MODE = 'authenticated';
process.env.VERIFICATION_ATTIRE_PROVIDER = 'mock';
__resetLlmRateLimitForTests();
__resetVerificationJobsForTests();
});
async function waitForMachineStatus(app, verificationId, maxAttempts = 30) {
let last;
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
last = await request(app)
.get(`/core/verifications/${verificationId}`)
.set('Authorization', 'Bearer test-token');
if (
last.body?.status === 'AUTO_PASS'
|| last.body?.status === 'AUTO_FAIL'
|| last.body?.status === 'NEEDS_REVIEW'
|| last.body?.status === 'ERROR'
) {
return last;
}
// eslint-disable-next-line no-await-in-loop
await new Promise((resolve) => setTimeout(resolve, 10));
}
return last;
}
test('GET /healthz returns healthy response', async () => {
const app = createApp();
const res = await request(app).get('/healthz');
assert.equal(res.status, 200);
assert.equal(res.body.ok, true);
assert.equal(typeof res.body.requestId, 'string');
assert.equal(typeof res.headers['x-request-id'], 'string');
});
test('POST /core/create-signed-url requires auth', async () => {
process.env.AUTH_BYPASS = 'false';
const app = createApp();
const res = await request(app).post('/core/create-signed-url').send({
fileUri: 'gs://krow-workforce-dev-private/foo.pdf',
});
assert.equal(res.status, 401);
assert.equal(res.body.code, 'UNAUTHENTICATED');
process.env.AUTH_BYPASS = 'true';
});
test('POST /core/create-signed-url returns signed URL', async () => {
const app = createApp();
const res = await request(app)
.post('/core/create-signed-url')
.set('Authorization', 'Bearer test-token')
.send({
fileUri: 'gs://krow-workforce-dev-private/uploads/test-user/foo.pdf',
expiresInSeconds: 300,
});
assert.equal(res.status, 200);
assert.equal(typeof res.body.signedUrl, 'string');
assert.equal(typeof res.body.expiresAt, 'string');
assert.equal(typeof res.body.requestId, 'string');
});
test('POST /core/create-signed-url rejects non-owned path', async () => {
const app = createApp();
const res = await request(app)
.post('/core/create-signed-url')
.set('Authorization', 'Bearer test-token')
.send({
fileUri: 'gs://krow-workforce-dev-private/uploads/other-user/foo.pdf',
expiresInSeconds: 300,
});
assert.equal(res.status, 403);
assert.equal(res.body.code, 'FORBIDDEN');
});
test('POST /core/create-signed-url enforces expiry cap', async () => {
process.env.MAX_SIGNED_URL_SECONDS = '300';
const app = createApp();
const res = await request(app)
.post('/core/create-signed-url')
.set('Authorization', 'Bearer test-token')
.send({
fileUri: 'gs://krow-workforce-dev-private/uploads/test-user/foo.pdf',
expiresInSeconds: 301,
});
assert.equal(res.status, 400);
assert.equal(res.body.code, 'VALIDATION_ERROR');
});
test('POST /invokeLLM legacy alias works', async () => {
const app = createApp();
const res = await request(app)
.post('/invokeLLM')
.set('Authorization', 'Bearer test-token')
.send({
prompt: 'hello',
responseJsonSchema: { type: 'object' },
fileUrls: [],
});
assert.equal(res.status, 200);
assert.equal(typeof res.body.result, 'object');
assert.equal(typeof res.body.model, 'string');
});
test('POST /core/invoke-llm enforces per-user rate limit', async () => {
process.env.LLM_RATE_LIMIT_PER_MINUTE = '1';
const app = createApp();
const first = await request(app)
.post('/core/invoke-llm')
.set('Authorization', 'Bearer test-token')
.send({
prompt: 'hello',
responseJsonSchema: { type: 'object' },
fileUrls: [],
});
const second = await request(app)
.post('/core/invoke-llm')
.set('Authorization', 'Bearer test-token')
.send({
prompt: 'hello again',
responseJsonSchema: { type: 'object' },
fileUrls: [],
});
assert.equal(first.status, 200);
assert.equal(second.status, 429);
assert.equal(second.body.code, 'RATE_LIMITED');
assert.equal(typeof second.headers['retry-after'], 'string');
});
test('POST /core/verifications creates async job and GET returns status', async () => {
const app = createApp();
const created = await request(app)
.post('/core/verifications')
.set('Authorization', 'Bearer test-token')
.send({
type: 'attire',
subjectType: 'staff',
subjectId: 'staff_1',
fileUri: 'gs://krow-workforce-dev-private/uploads/test-user/attire.jpg',
rules: { attireType: 'shoes', expectedColor: 'black' },
});
assert.equal(created.status, 202);
assert.equal(created.body.type, 'attire');
assert.equal(created.body.status, 'PENDING');
assert.equal(typeof created.body.verificationId, 'string');
const status = await waitForMachineStatus(app, created.body.verificationId);
assert.equal(status.status, 200);
assert.equal(status.body.verificationId, created.body.verificationId);
assert.equal(status.body.type, 'attire');
assert.ok(['NEEDS_REVIEW', 'AUTO_PASS', 'AUTO_FAIL', 'ERROR'].includes(status.body.status));
});
test('POST /core/verifications rejects file paths not owned by actor', async () => {
const app = createApp();
const res = await request(app)
.post('/core/verifications')
.set('Authorization', 'Bearer test-token')
.send({
type: 'attire',
fileUri: 'gs://krow-workforce-dev-private/uploads/other-user/not-allowed.jpg',
rules: { attireType: 'shoes' },
});
assert.equal(res.status, 403);
assert.equal(res.body.code, 'FORBIDDEN');
});
test('POST /core/verifications/:id/review finalizes verification', async () => {
const app = createApp();
const created = await request(app)
.post('/core/verifications')
.set('Authorization', 'Bearer test-token')
.send({
type: 'certification',
subjectType: 'staff',
subjectId: 'staff_1',
fileUri: 'gs://krow-workforce-dev-private/uploads/test-user/cert.pdf',
rules: { certType: 'food_safety' },
});
const status = await waitForMachineStatus(app, created.body.verificationId);
assert.equal(status.status, 200);
const reviewed = await request(app)
.post(`/core/verifications/${created.body.verificationId}/review`)
.set('Authorization', 'Bearer test-token')
.send({
decision: 'APPROVED',
note: 'Looks good',
reasonCode: 'MANUAL_REVIEW',
});
assert.equal(reviewed.status, 200);
assert.equal(reviewed.body.status, 'APPROVED');
assert.equal(reviewed.body.review.decision, 'APPROVED');
});
test('POST /core/verifications/:id/retry requeues verification', async () => {
const app = createApp();
const created = await request(app)
.post('/core/verifications')
.set('Authorization', 'Bearer test-token')
.send({
type: 'government_id',
subjectType: 'staff',
subjectId: 'staff_1',
fileUri: 'gs://krow-workforce-dev-private/uploads/test-user/id-front.jpg',
rules: {},
});
const status = await waitForMachineStatus(app, created.body.verificationId);
assert.equal(status.status, 200);
const retried = await request(app)
.post(`/core/verifications/${created.body.verificationId}/retry`)
.set('Authorization', 'Bearer test-token')
.send({});
assert.equal(retried.status, 202);
assert.equal(retried.body.status, 'PENDING');
});

View File

@@ -0,0 +1,232 @@
# M4 API Catalog (Core Only)
Status: Active
Date: 2026-02-24
Owner: Technical Lead
Environment: dev
## Frontend source of truth
Use this file and `docs/MILESTONES/M4/planning/m4-core-api-frontend-guide.md` for core endpoint consumption.
## Related next-slice contract
Verification pipeline design (attire, government ID, certification):
- `docs/MILESTONES/M4/planning/m4-verification-architecture-contract.md`
## 1) Scope and purpose
This catalog defines the currently implemented core backend contract for M4.
## 2) Global API rules
1. Route group in scope: `/core/*`.
2. Compatibility aliases in scope:
- `POST /uploadFile` -> `POST /core/upload-file`
- `POST /createSignedUrl` -> `POST /core/create-signed-url`
- `POST /invokeLLM` -> `POST /core/invoke-llm`
3. Auth model:
- `GET /health` is public in dev
- all other routes require `Authorization: Bearer <firebase-id-token>`
4. Standard error envelope:
```json
{
"code": "STRING_CODE",
"message": "Human readable message",
"details": {},
"requestId": "optional-request-id"
}
```
5. Response header:
- `X-Request-Id`
## 3) Core routes
## 3.1 Upload file
1. Method and route: `POST /core/upload-file`
2. Request format: `multipart/form-data`
3. Fields:
- `file` (required)
- `visibility` (`public` or `private`, optional)
- `category` (optional)
4. Accepted types:
- `application/pdf`
- `image/jpeg`
- `image/jpg`
- `image/png`
5. Max size: `10 MB` (default)
6. Behavior: real upload to Cloud Storage.
7. Success `200`:
```json
{
"fileUri": "gs://krow-workforce-dev-private/uploads/<uid>/...",
"contentType": "application/pdf",
"size": 12345,
"bucket": "krow-workforce-dev-private",
"path": "uploads/<uid>/...",
"requestId": "uuid"
}
```
8. Errors:
- `UNAUTHENTICATED`
- `INVALID_FILE_TYPE`
- `FILE_TOO_LARGE`
## 3.2 Create signed URL
1. Method and route: `POST /core/create-signed-url`
2. Request:
```json
{
"fileUri": "gs://krow-workforce-dev-private/uploads/<uid>/file.pdf",
"expiresInSeconds": 300
}
```
3. Security checks:
- bucket must be allowed
- path must be owned by caller (`uploads/<caller_uid>/...`)
- object must exist
- `expiresInSeconds <= 900`
4. Success `200`:
```json
{
"signedUrl": "https://storage.googleapis.com/...",
"expiresAt": "2026-02-24T15:22:28.105Z",
"requestId": "uuid"
}
```
5. Errors:
- `VALIDATION_ERROR`
- `FORBIDDEN`
- `NOT_FOUND`
## 3.3 Invoke model
1. Method and route: `POST /core/invoke-llm`
2. Request:
```json
{
"prompt": "...",
"responseJsonSchema": {},
"fileUrls": []
}
```
3. Behavior:
- real Vertex AI call
- model default: `gemini-2.0-flash-001`
- timeout default: `20 seconds`
4. Rate limit:
- `20 requests/minute` per user (default)
- when exceeded: `429 RATE_LIMITED` and `Retry-After` header
5. Success `200`:
```json
{
"result": {},
"model": "gemini-2.0-flash-001",
"latencyMs": 367,
"requestId": "uuid"
}
```
6. Errors:
- `UNAUTHENTICATED`
- `VALIDATION_ERROR`
- `MODEL_TIMEOUT`
- `MODEL_FAILED`
- `RATE_LIMITED`
## 3.4 Create verification job
1. Method and route: `POST /core/verifications`
2. Auth: required
3. Request:
```json
{
"type": "attire",
"subjectType": "worker",
"subjectId": "worker_123",
"fileUri": "gs://krow-workforce-dev-private/uploads/<uid>/file.pdf",
"rules": {}
}
```
4. Behavior:
- validates `fileUri` ownership
- requires file existence when `UPLOAD_MOCK=false` and `VERIFICATION_REQUIRE_FILE_EXISTS=true`
- enqueues async verification
5. Success `202`:
```json
{
"verificationId": "ver_123",
"status": "PENDING",
"type": "attire",
"requestId": "uuid"
}
```
6. Errors:
- `UNAUTHENTICATED`
- `VALIDATION_ERROR`
- `FORBIDDEN`
- `NOT_FOUND`
## 3.5 Get verification status
1. Method and route: `GET /core/verifications/{verificationId}`
2. Auth: required
3. Success `200`:
```json
{
"verificationId": "ver_123",
"status": "NEEDS_REVIEW",
"type": "attire",
"requestId": "uuid"
}
```
4. Errors:
- `UNAUTHENTICATED`
- `FORBIDDEN`
- `NOT_FOUND`
## 3.6 Review verification
1. Method and route: `POST /core/verifications/{verificationId}/review`
2. Auth: required
3. Request:
```json
{
"decision": "APPROVED",
"note": "Manual review passed",
"reasonCode": "MANUAL_REVIEW"
}
```
4. Success `200`: status becomes `APPROVED` or `REJECTED`.
5. Errors:
- `UNAUTHENTICATED`
- `VALIDATION_ERROR`
- `FORBIDDEN`
- `NOT_FOUND`
## 3.7 Retry verification
1. Method and route: `POST /core/verifications/{verificationId}/retry`
2. Auth: required
3. Success `202`: status resets to `PENDING`.
4. Errors:
- `UNAUTHENTICATED`
- `FORBIDDEN`
- `NOT_FOUND`
## 3.8 Health
1. Method and route: `GET /health`
2. Success `200`:
```json
{
"ok": true,
"service": "krow-core-api",
"version": "dev",
"requestId": "uuid"
}
```
## 4) Locked defaults
1. Validation library: `zod`.
2. Validation schema location: `backend/core-api/src/contracts/`.
3. Buckets:
- `krow-workforce-dev-public`
- `krow-workforce-dev-private`
4. Model provider: Vertex AI Gemini.
5. Max signed URL expiry: `900` seconds.
6. LLM timeout: `20000` ms.
7. LLM rate limit: `20` requests/minute/user.
8. Verification access mode default: `authenticated`.
9. Verification file existence check default: enabled (`VERIFICATION_REQUIRE_FILE_EXISTS=true`).
10. Verification attire provider default in dev: `vertex` with model `gemini-2.0-flash-lite-001`.
11. Verification government/certification providers: external adapters via configured provider URL/token.

View File

@@ -0,0 +1,269 @@
# M4 Backend Foundation Implementation Plan (Dev First)
Date: 2026-02-24
Owner: Wilfred (Technical Lead)
Primary environment: `krow-workforce-dev`
## 1) Objective
Build a secure, modular, and scalable backend foundation in `dev` without breaking the current frontend while we migrate high-risk writes from direct Data Connect mutations to backend command endpoints.
## 2) First-principles architecture rules
1. Client apps are untrusted for business-critical writes.
2. Backend is the enforcement layer for validation, permissions, and write orchestration.
3. Multi-entity writes must be atomic, idempotent, and observable.
4. Configuration and deployment must be reproducible by automation.
5. Migration must be backward-compatible until each frontend flow is cut over.
## 3) Pre-coding gates (must be true before implementation starts)
## Gate A: Security boundary
1. Frontend sends Firebase token only. No database credentials in client code.
2. Every new backend endpoint validates Firebase token.
3. Data Connect write access strategy is defined:
- keep simple reads available to client
- route high-risk writes through backend command endpoints
4. Upload and signed URL paths are server-controlled.
## Gate B: Contract standards
1. Standard error envelope is frozen:
```json
{
"code": "STRING_CODE",
"message": "Human readable message",
"details": {},
"requestId": "optional-request-id"
}
```
2. Request validation layer is chosen and centralized.
3. Route naming strategy is frozen:
- canonical routes under `/core` and `/commands`
- compatibility aliases preserved during migration (`/uploadFile`, `/createSignedUrl`, `/invokeLLM`)
4. Validation standard is locked:
- library: `zod`
- schema location: `backend/<service>/src/contracts/` with `core/` and `commands/` subfolders
## Gate C: Atomicity and reliability
1. Command endpoints support idempotency keys for retry-safe writes.
2. Multi-step write flows are wrapped in single backend transaction boundaries.
3. Domain conflict codes are defined for expected business failures.
4. Idempotency storage is locked:
- store in Cloud SQL table
- key scope: `userId + route + idempotencyKey`
- retain records for 24 hours
- repeated key returns original response
## Gate D: Automation and operability
1. Makefile is source of truth for backend setup and deploy in dev.
2. Core deploy and smoke test commands exist before feature migration.
3. Logging format and request tracing fields are standardized.
## 4) Security baseline for foundation phase
## 4.1 Authentication and authorization
1. Foundation phase is authentication-first.
2. Role-based access control is intentionally deferred.
3. All handlers include a policy hook for future role checks (`can(action, resource, actor)`).
## 4.2 Data access control model
1. Client retains Data Connect reads required for existing screens.
2. High-risk writes move behind `/commands/*` endpoints.
3. Backend mediates write interactions with Data Connect and Cloud SQL.
## 4.3 File and URL security
1. Validate file type and size server-side.
2. Separate public and private storage behavior.
3. Signed URL creation checks ownership/prefix scope and expiry limits.
4. Bucket policy split is locked:
- `krow-workforce-dev-public`
- `krow-workforce-dev-private`
- private bucket access only through signed URL
## 4.4 Model invocation safety
1. Enforce schema-constrained output.
2. Apply per-user rate limits and request timeout.
3. Log model failures with safe redaction (no sensitive prompt leakage in logs).
4. Model provider and timeout defaults are locked:
- provider: Vertex AI Gemini
- max route timeout: 20 seconds
- timeout error code: `MODEL_TIMEOUT`
## 4.5 Secrets and credentials
1. Runtime secrets come from Secret Manager only.
2. Service accounts use least-privilege roles.
3. No secrets committed in repository files.
## 5) Modularity baseline
## 5.1 Backend module boundaries
1. `core` module: upload, signed URL, model invocation, health.
2. `commands` module: business writes and state transitions.
3. `policy` module: validation and future role checks.
4. `data` module: Data Connect adapters and transaction wrappers.
5. `infra` module: logging, tracing, auth middleware, error mapping.
## 5.2 Contract separation
1. Keep API request/response schemas in one location.
2. Keep domain errors in one registry file.
3. Keep route declarations thin; business logic in services.
## 5.3 Cloud runtime roles
1. Cloud Run is the primary command and core API execution layer.
2. Cloud Functions v2 is worker-only in this phase:
- upload-related async handlers
- notification jobs
- model-related async helpers when needed
## 6) Automation baseline
## 6.1 Makefile requirements
Add `makefiles/backend.mk` and wire it into root `Makefile` with at least:
1. `make backend-enable-apis`
2. `make backend-bootstrap-dev`
3. `make backend-deploy-core`
4. `make backend-deploy-commands`
5. `make backend-deploy-workers`
6. `make backend-smoke-core`
7. `make backend-smoke-commands`
8. `make backend-logs-core`
## 6.2 CI requirements
1. Backend lint
2. Backend tests
3. Build/package
4. Smoke test against deployed dev route(s)
5. Block merge on failed checks
## 6.3 Session hygiene
1. Update `TASKS.md` and `CHANGELOG.md` each working session.
2. If a new service/API is added, Makefile target must be added in same change.
## 7) Migration safety contract (no frontend breakage)
1. Backend routes ship first.
2. Frontend migration is per-feature wave, not big bang.
3. Keep compatibility aliases until clients migrate.
4. Keep existing Data Connect reads during foundation.
5. For each migrated write flow:
- before/after behavior checklist
- rollback path
- smoke verification
## 8) Scope for foundation build
1. Backend runtime/deploy foundation in dev.
2. Core endpoints:
- `POST /core/upload-file`
- `POST /core/create-signed-url`
- `POST /core/invoke-llm`
- `GET /healthz`
3. Compatibility aliases:
- `POST /uploadFile`
- `POST /createSignedUrl`
- `POST /invokeLLM`
4. Command layer scaffold for first migration routes.
5. Initial migration of highest-risk write paths.
## 9) Implementation phases
## Phase 0: Baseline and contracts
Deliverables:
1. Freeze endpoint naming and compatibility aliases.
2. Freeze error envelope and error code registry.
3. Freeze auth middleware interface and policy hook interface.
4. Publish route inventory from web/mobile direct writes.
Exit criteria:
1. No unresolved contract ambiguity.
2. Team agrees on auth-first now and role-map-later approach.
## Phase 1: Backend infra and automation
Deliverables:
1. `makefiles/backend.mk` with bootstrap, deploy, smoke, logs targets.
2. Environment templates for backend runtime config.
3. Secret Manager and service account setup automation.
Exit criteria:
1. A fresh machine can deploy core backend to dev via Make commands.
## Phase 2: Core endpoint implementation
Deliverables:
1. `/core/upload-file`
2. `/core/create-signed-url`
3. `/core/invoke-llm`
4. `/healthz`
5. Compatibility aliases (`/uploadFile`, `/createSignedUrl`, `/invokeLLM`)
Exit criteria:
1. API harness passes for core routes.
2. Error, logging, and auth standards are enforced.
## Phase 3: Command layer scaffold
Deliverables:
1. `/commands/orders/create`
2. `/commands/orders/{orderId}/cancel`
3. `/commands/orders/{orderId}/update`
4. `/commands/shifts/{shiftId}/change-status`
5. `/commands/shifts/{shiftId}/assign-staff`
6. `/commands/shifts/{shiftId}/accept`
Exit criteria:
1. High-risk writes have backend command alternatives ready.
## Phase 4: Wave 1 frontend migration
Deliverables:
1. Replace direct writes in selected web/mobile flows.
2. Keep reads stable.
3. Verify no regressions in non-migrated screens.
Exit criteria:
1. Migrated flows run through backend commands only.
2. Rollback instructions validated.
## Phase 5: Hardening and handoff
Deliverables:
1. Runbook for deploy, rollback, and smoke.
2. Backend CI pipeline active.
3. Wave 2 and wave 3 migration task list defined.
Exit criteria:
1. Foundation is reusable for staging/prod with environment changes only.
## 10) Wave 1 migration inventory (real call sites)
Web:
1. `apps/web/src/features/operations/tasks/TaskBoard.tsx:100`
2. `apps/web/src/features/operations/orders/OrderDetail.tsx:145`
3. `apps/web/src/features/operations/orders/EditOrder.tsx:84`
4. `apps/web/src/features/operations/orders/components/CreateOrderDialog.tsx:31`
5. `apps/web/src/features/operations/orders/components/AssignStaffModal.tsx:60`
6. `apps/web/src/features/workforce/documents/DocumentVault.tsx:99`
Mobile:
1. `apps/mobile/packages/features/client/home/lib/src/presentation/widgets/shift_order_form_sheet.dart:232`
2. `apps/mobile/packages/features/client/view_orders/lib/src/presentation/widgets/view_order_card.dart:1195`
3. `apps/mobile/packages/features/client/create_order/lib/src/data/repositories_impl/client_create_order_repository_impl.dart:68`
4. `apps/mobile/packages/features/staff/shifts/lib/src/data/repositories_impl/shifts_repository_impl.dart:446`
5. `apps/mobile/packages/features/client/authentication/lib/src/data/repositories_impl/auth_repository_impl.dart:257`
6. `apps/mobile/packages/features/staff/profile_sections/onboarding/profile_info/lib/src/data/repositories/personal_info_repository_impl.dart:51`
## 11) Definition of done for foundation
1. Core endpoints deployed in dev and validated.
2. Command scaffolding in place for wave 1 writes.
3. Auth-first protection active on all new routes.
4. Idempotency + transaction model defined for command writes.
5. Makefile and CI automation cover bootstrap/deploy/smoke paths.
6. Frontend remains stable during migration.
7. Role-map integration points are documented for next phase.
## 12) Locked defaults (approved)
1. Idempotency key storage strategy:
- Cloud SQL table, 24-hour retention, keyed by `userId + route + idempotencyKey`.
2. Validation library and schema location:
- `zod` in `backend/<service>/src/contracts/` (`core/`, `commands/`).
3. Storage bucket naming and split:
- `krow-workforce-dev-public` and `krow-workforce-dev-private`.
4. Model provider and timeout:
- Vertex AI Gemini, 20-second max timeout.
5. Target response-time objectives (p95):
- `/healthz` under 200ms
- `/core/create-signed-url` under 500ms
- `/commands/*` under 1500ms
- `/core/invoke-llm` under 15000ms

View File

@@ -0,0 +1,166 @@
# M4 Core Data Actors and Example Scenarios
Status: Working draft
Date: 2026-02-25
Owner: Technical Lead
## 1) Core data actors
1. `Tenant`: staffing company boundary and data isolation root.
2. `User`: human identity that signs in.
3. `TenantMembership`: user role/context inside one tenant.
4. `Business`: client account served by the tenant.
5. `BusinessMembership`: maps users to a business with role/status (`owner`, `manager`, `approver`, `viewer`).
6. `Vendor`: supplier account that can fulfill staffing demand.
7. `VendorMembership`: maps users to a vendor with role/status (`owner`, `manager`, `scheduler`, `viewer`).
8. `Workforce/Staff`: worker profile used for assignment and attendance.
9. `StakeholderType`: typed category (`buyer`, `operator`, `vendor_partner`, `workforce`, `partner`, `procurement_partner`).
10. `StakeholderProfile`: typed actor record inside a tenant.
11. `StakeholderLink`: relationship between stakeholder profiles.
## 1.1 Current schema coverage (today)
Current Data Connect handles this only partially:
1. `Business.userId` supports one primary business user.
2. `Vendor.userId` supports one primary vendor user.
3. `TeamMember` can represent multiple users by team as a workaround.
This is why we need first-class membership tables:
1. `business_memberships`
2. `vendor_memberships`
Without those, client/vendor user partitioning is indirect and harder to enforce safely at scale.
## 2) Minimal actor map
```mermaid
flowchart LR
T["Tenant"] --> TM["TenantMembership (global tenant access)"]
U["User"] --> TM
T --> B["Business"]
T --> V["Vendor"]
U --> BM["BusinessMembership"]
BM --> B
U --> VM["VendorMembership"]
VM --> V
U --> S["Workforce/Staff"]
T --> SP["StakeholderProfile"]
ST["StakeholderType"] --> SP
SP --> SL["StakeholderLink"]
B --> O["Orders/Shifts"]
V --> O
S --> O
```
## 3) Scenario A: Legendary Event Staffing
Context:
1. Tenant is `Legendary Event Staffing and Entertainment`.
2. Business is `Google Mountain View Cafes`.
3. Legendary uses its own workforce, and can still route overflow to approved vendors.
Actor mapping (text):
1. Tenant: `Legendary Event Staffing and Entertainment` (the company using Krow).
2. User: `Wil` (ops lead), `Maria` (Google client manager), `Omar` (Google procurement approver), `Jose` (vendor scheduler), `Ana` (worker).
3. TenantMembership:
4. `Wil` is `admin` in Legendary tenant.
5. `Maria` is `member` in Legendary tenant.
6. `Omar` is `member` in Legendary tenant.
7. `Jose` is `member` in Legendary tenant.
8. `Ana` is `member` in Legendary tenant.
9. BusinessMembership:
10. `Maria` is `manager` in `Google Mountain View Cafes`.
11. `Omar` is `approver` in `Google Mountain View Cafes`.
12. VendorMembership:
13. `Jose` is `scheduler` in `Legendary Staffing Pool A`.
14. `Wil` is `owner` in `Legendary Staffing Pool A`.
15. Business: `Google Mountain View Cafes` (client account under the tenant).
16. Vendor: `Legendary Staffing Pool A` (or an external approved vendor).
17. Workforce/Staff: `Ana` is a staff profile in workforce, linked to certifications and assignments.
18. StakeholderType: `buyer`, `operator`, `vendor_partner`, `workforce`, `procurement_partner`.
19. StakeholderProfile:
20. `Google Procurement` = `buyer`.
21. `Legendary Ops` = `operator`.
22. `FoodBuy` = `procurement_partner`.
23. StakeholderLink:
24. `Google Procurement` `contracts_with` `Legendary Ops`.
25. `Legendary Ops` `sources_from` `Legendary Staffing Pool A`.
26. `Google Procurement` `reports_through` `FoodBuy`.
```mermaid
sequenceDiagram
participant Tenant as "Tenant (Legendary)"
participant BizUser as "Business user (Maria/Omar)"
participant Ops as "Ops user (Wil)"
participant VendorUser as "Vendor user (Jose)"
participant Staff as "Workforce/Staff (Barista Ana)"
participant StakeRel as "StakeholderLink"
BizUser->>Ops: "Create staffing request"
Ops->>Tenant: "Create order under tenant scope"
Ops->>StakeRel: "Resolve business-to-vendor/workforce relationships"
Ops->>VendorUser: "Dispatch role demand"
VendorUser->>Staff: "Confirm worker assignment"
Staff-->>Ops: "Clock in/out and complete shift"
Ops-->>BizUser: "Invoice/report generated with audit trail"
```
## 4) Scenario B: Another tenant (external vendor-heavy)
Context:
1. Tenant is `Peakline Events`.
2. Business is `NVIDIA Campus Dining`.
3. Peakline primarily fulfills demand through external approved vendors.
Actor mapping (text):
1. Tenant: `Peakline Events` (another staffing company using Krow).
2. User: `Chris` (operations coordinator), `Nina` (client manager), `Sam` (vendor manager), `Leo` (worker).
3. TenantMembership:
4. `Chris` is `admin` in Peakline tenant.
5. `Nina` is `member` in Peakline tenant.
6. `Sam` is `member` in Peakline tenant.
7. `Leo` is `member` in Peakline tenant.
8. BusinessMembership:
9. `Nina` is `manager` in `NVIDIA Campus Dining`.
10. VendorMembership:
11. `Sam` is `manager` in `Metro Staffing LLC`.
12. Business: `NVIDIA Campus Dining` (client account under the tenant).
13. Vendor: `Metro Staffing LLC` (approved external vendor for this tenant).
14. Workforce/Staff: `Leo` is a workforce profile fulfilled through Metro Staffing for assignment and attendance.
15. StakeholderType: `buyer`, `operator`, `vendor_partner`, `workforce`, `procurement_partner`.
16. StakeholderProfile:
17. `NVIDIA Procurement` = `buyer`.
18. `Peakline Ops` = `operator`.
19. `FoodBuy Regional` = `procurement_partner`.
20. StakeholderLink:
21. `NVIDIA Procurement` `contracts_with` `Peakline Ops`.
22. `Peakline Ops` `sources_from` `Metro Staffing LLC`.
23. `NVIDIA Procurement` `reports_through` `FoodBuy Regional`.
```mermaid
sequenceDiagram
participant Tenant as "Tenant (Peakline Events)"
participant BizUser as "Business user (Nina)"
participant Ops as "Ops user (Chris)"
participant VendorUser as "Vendor user (Sam)"
participant Staff as "Workforce/Staff (Vendor worker)"
BizUser->>Ops: "Request recurring event staffing"
Ops->>Tenant: "Create recurring order"
Ops->>VendorUser: "Dispatch required roles"
VendorUser->>Staff: "Provide available workers"
Staff-->>Ops: "Attendance and completion events"
Ops-->>BizUser: "Settlement + performance reports"
```
## 5) Why this model scales
1. New stakeholders can be added through `StakeholderType` and `StakeholderProfile` without changing core order/shift tables.
2. Business and vendor user partitioning is explicit through membership tables, not hidden in one owner field.
3. Multi-tenant isolation stays strict because all critical records resolve through `Tenant`.
4. Legendary and non-Legendary operating models both fit the same structure.
## 6) Industry alignment (primary references)
1. Google Cloud Identity Platform multi-tenancy:
- https://docs.cloud.google.com/identity-platform/docs/multi-tenancy
2. AWS SaaS tenant isolation fundamentals:
- https://docs.aws.amazon.com/whitepapers/latest/saas-architecture-fundamentals/tenant-isolation.html
3. B2B organization-aware login flows (Auth0 Organizations):
- https://auth0.com/docs/manage-users/organizations/login-flows-for-organizations
4. Supplier-side multi-user management patterns (Coupa Supplier Portal):
- https://compass.coupa.com/en-us/products/product-documentation/supplier-resources/for-suppliers/coupa-supplier-portal/set-up-the-csp/users/manage-users

View File

@@ -1,47 +0,0 @@
# M4 Planning Phase: Identified Discrepancies and Enhancements
## Feedback and Discrepancies (Based on M3 Review done by Iliana)
### Mobile Application (Client & Staff)
- **Flexible Shift Locations:** Feedback from the M3 review indicated a need for the ability to specify different locations for individual positions within an order on the mobile app. Currently, the web application handles this by requiring a separate shift for each location.
- **Order Visibility Management:** Currently, when an order is created with multiple positions, the "View Order" page displays them as separate orders. This is due to UI (prototype) cannot support multi-position views in the order. We should consider adopting the "legacy" app's approach to group these positions under a single order for better clarity.
- **Cost Center Clarification:** The purpose and functionality of the "Cost Center" field in the Hub creation process is not clear.
- **Tax Form Data Completeness:** Feedback noted that while tax forms are visible in the Staff mobile application, they appear to be missing critical information. This not clear.
### Web Dashboard
- **Role-Based Content Logic:** The current web dashboard prototype contains some logical inconsistencies regarding user roles:
- **Client Dashboard:** Currently includes Staff Availability, Staff Directory, and Staff Onboarding. Since workers (Staff) are managed by Vendors, these pages should be moved to the Vendor dashboard.
- **Vendor Dashboard:** Currently includes "Teams and Hubs." Since Hubs are client-specific locations where staff clock in/out, these management pages should be moved to the Client dashboard.
- **Admin Dashboard Filtering:** The Admin dashboard requires improved filtering capabilities. Admins should be able to select specific Clients or Vendors to filter related data, such as viewing only the orders associated with a chosen partner.
## Proposed Features and Enhancements (Post-M3 Identification)
- **Feature: Navigation to Hub Details from Coverage Screen (#321)**
- **Description:** Allow users to navigate directly to the Hub Details page by clicking on a hub within the Coverage Screen.
- **Feature: Dedicated Hub Details Screen with Order History (#320)**
- **Description:** Develop a comprehensive Hub Details view that aggregates all hub-specific data, including historical order records.
- **Benefit:** Centralizes information for better decision-making and easier access to historical data.
- **Feature: Dedicated Order Details Screen**
- **Description:** Transition from displaying all order information on the primary "View Order" page to a dedicated "Order Details" screen. This screen will support viewing multiple positions within a single order.
- **Benefit:**
- **Improved UX:** Reduces complexity by grouping associated positions together and presenting them in a structured way.
- **Performance:** Optimizes data loading by fetching detailed position information only when requested.
- **Feature: Optimized Clock-In Page (#350)**
- **Description:** Remove the calendar component from the Clock-In page. Since workers only clock in for current-day assignments, the calendar is unnecessary.
- **Benefit:** Simplifies the interface and reduces user confusion.
- **Feature: Contextual Shift Actions**
- **Description:** Restrict the Clock-In page to show only active or upcoming shifts (starting within 30 minutes). Shift-specific actions (Clock-In/Clock-Out) should be performed within the specific Shift Details page.
- **Reasoning:** This solves issues where staff cannot clock out of overnight shifts (shifts starting one day and ending the next) due to the current day-based UI.
- **Feature: Dedicated Emergency Contact Management (#356)**
- **Description:** Replace the inline form in the "View Emergency Contact" page with a dedicated "Create Emergency Contact" screen.
- **Benefit:** Standardizes the data entry process and improves UI organization within the Staff app.

View File

@@ -0,0 +1,168 @@
# M4 Roadmap CSV Schema Reconciliation
Status: Draft for implementation alignment
Date: 2026-02-25
Owner: Technical Lead
## 1) Why this exists
We reviewed the original product-roadmap exports to confirm that the target schema can support all stakeholder lanes as the platform grows.
This avoids two failure modes:
1. Building command APIs on top of a schema that cannot represent required workflows.
2. Hard-coding today's customer setup in a way that blocks future staffing companies.
## 2) Inputs reviewed
All 13 roadmap exports from `/Users/wiel/Downloads`:
1. `Krow App Roadmap - Business App_ Google, Nvidia.csv`
2. `Krow App Roadmap - Client_ Google, Nvidia.csv`
3. `Krow App Roadmap - Compass- The Operator.csv`
4. `Krow App Roadmap - Employee App.csv`
5. `Krow App Roadmap - Features.csv`
6. `Krow App Roadmap - FoodBuy- Procurement.csv`
7. `Krow App Roadmap - KROW Dashboard.csv`
8. `Krow App Roadmap - Offenses.csv`
9. `Krow App Roadmap - Partner.csv`
10. `Krow App Roadmap - Roadmap.csv`
11. `Krow App Roadmap - Sectors_ BA, Flik ( The executors).csv`
12. `Krow App Roadmap - The Workforce_ Employees.csv`
13. `Krow App Roadmap - Vendor_ Legendary (Staffing).csv`
Parsed signal:
1. 983 non-empty task lines.
2. 1,263 planning rows with task/status/priority/reference signals.
## 3) What the roadmap is clearly asking for
Cross-file recurring capabilities:
1. Multi-party org model: client, operator, vendor, procurement, workforce, partner, sector execution.
2. Orders and shift operations: recurring events, assignment, coverage, schedule management.
3. Attendance and policy enforcement: clock-in/out, no-show, tardiness, cancellation, offense ladders.
4. Compliance and document verification: certifications, insurance, legal docs, renewals, risk alerts.
5. Finance and settlement: invoice lifecycle, disputes, remittance, payment history, aging, payroll/earnings.
6. Reporting and prediction: dashboards, KPI, forecasting, scenario planning.
Repeated examples across many sheets:
1. `Vendor Onboarding`, `Service Locations`, `Compliance`, `Certifications`.
2. `All Invoices (Open/Pending/Overdue/Paid...)`, `Payment Summary`, `Remittance Advice Download`.
3. Offense progression rules in `Offenses.csv` and `Employee App.csv` (warning -> suspension -> disable/block).
## 4) Stakeholder capability matrix (from roadmap exports)
| Stakeholder lane | Org network | Orders and shifts | Attendance and offense | Compliance docs | Finance | Reporting |
|---|---|---|---|---|---|---|
| Client (Google, Nvidia) | Yes | Yes | Partial (visibility) | Yes | Yes | Yes |
| Vendor (Legendary) | Yes | Yes | Yes | Yes | Yes | Yes |
| Workforce (Employee app) | Limited | Yes | Yes | Yes | Earnings focus | Limited |
| Operator (Compass) | Yes | Yes | KPI visibility | Yes | Yes | Yes |
| Procurement (FoodBuy) | Yes | KPI/SLA focus | KPI/SLA focus | Yes | Yes | Yes |
| KROW Dashboard | Cross-entity | Cross-entity | Cross-entity risk | Cross-entity | Cross-entity | Heavy |
| Partner | Basic | Basic | Minimal | Yes | Basic | Basic |
Implication:
1. This is a multi-tenant, multi-party workflow platform, not a single-role CRUD app.
2. Schema must separate party identity, party relationships, and role-based permissions from workflow records.
## 5) Reconciliation against current Data Connect schema
What already exists and is useful:
1. Core scheduling entities: `orders`, `shifts`, `shift_roles`, `applications`, `assignments`.
2. Workforce entities: `staffs`, `workforce`, `staff_roles`.
3. Financial entities: `invoices`, `recent_payments`, `vendor_rates`.
4. Compliance entities: `documents`, `staff_documents`, `certificates`.
Current structural gaps for roadmap scale:
1. No tenant boundary key on core tables (`tenant_id` missing).
2. No first-class business user partitioning table (`business_memberships` missing).
3. No first-class vendor user partitioning table (`vendor_memberships` missing).
4. No first-class stakeholder profile/link model for buyer/operator/partner/sector relationships.
5. Attendance history is not first-class (check in/out only inside `applications`).
6. No offense policy, offense event, or enforcement action tables.
7. Finance is coarse (invoice + recent payment), missing line items, payment runs, remittance artifact model.
8. Sensitive bank fields are currently modeled directly in `accounts` (`accountNumber`, `routeNumber`).
9. Many core workflow fields are JSON (`orders.assignedStaff`, `orders.shifts`, `shift.managers`, `assignment.managers`).
10. Money still uses float in critical tables.
Connector boundary gap:
1. 147 Data Connect mutation operations exist.
2. 36 of those are high-risk core workflow mutations (`order`, `shift`, `shiftRole`, `application`, `assignment`, `invoice`, `vendor`, `business`, `workForce`, `teamMember`, `account`).
## 6) Target schema additions required before full command rollout
### 6.1 Tenant and stakeholder graph
1. `tenants`
2. `tenant_memberships`
3. `business_memberships`
4. `vendor_memberships`
5. `stakeholder_types`
6. `stakeholder_profiles`
7. `stakeholder_links`
8. `role_bindings` (scoped to tenant/team/hub/business/vendor/resource)
### 6.2 Attendance and timesheet reliability
1. `attendance_events` (append-only clock-in/out/NFC/manual-corrected)
2. `attendance_sessions` (derived per shift role assignment)
3. `timesheets` (approval state and pay-eligible snapshot)
4. `timesheet_adjustments` (manual corrections with audit reason)
### 6.3 Offense and policy enforcement
1. `offense_policies` (per tenant or per business)
2. `offense_rules` (threshold and consequence ladder)
3. `offense_events` (who, what, when, evidence)
4. `enforcement_actions` (warning, suspension, disable, block-from-client)
### 6.4 Compliance and verification
1. `verification_jobs`
2. `verification_reviews`
3. `verification_events`
4. `compliance_requirements` (per role, tenant, business, or client contract)
### 6.5 Finance completeness
1. `invoice_line_items`
2. `invoice_status_history`
3. `payment_runs`
4. `payment_allocations`
5. `remittance_documents`
6. `account_token_refs` (tokenized provider refs, no raw account/routing)
## 7) Minimal target relationship view
```mermaid
erDiagram
TENANT ||--o{ TENANT_MEMBERSHIP : has
BUSINESS ||--o{ BUSINESS_MEMBERSHIP : has
VENDOR ||--o{ VENDOR_MEMBERSHIP : has
USER ||--o{ BUSINESS_MEMBERSHIP : belongs_to
USER ||--o{ VENDOR_MEMBERSHIP : belongs_to
TENANT ||--o{ STAKEHOLDER_PROFILE : has
STAKEHOLDER_PROFILE ||--o{ STAKEHOLDER_LINK : links_to
TENANT ||--o{ BUSINESS : owns
TENANT ||--o{ VENDOR : owns
BUSINESS ||--o{ ORDER : requests
VENDOR ||--o{ ORDER : fulfills
ORDER ||--o{ SHIFT : expands_to
SHIFT ||--o{ SHIFT_ROLE : requires
SHIFT_ROLE ||--o{ APPLICATION : receives
APPLICATION ||--o{ ASSIGNMENT : becomes
ASSIGNMENT ||--o{ ATTENDANCE_EVENT : emits
ASSIGNMENT ||--o{ TIMESHEET : settles
OFFENSE_POLICY ||--o{ OFFENSE_RULE : defines
ASSIGNMENT ||--o{ OFFENSE_EVENT : may_trigger
OFFENSE_EVENT ||--o{ ENFORCEMENT_ACTION : causes
ORDER ||--o{ INVOICE : bills
INVOICE ||--o{ INVOICE_LINE_ITEM : contains
PAYMENT_RUN ||--o{ PAYMENT_ALLOCATION : allocates
INVOICE ||--o{ PAYMENT_ALLOCATION : receives
PAYMENT_RUN ||--o{ REMITTANCE_DOCUMENT : publishes
```
## 8) First-principles rules we should lock now
1. Every command-critical table includes `tenant_id`.
2. High-risk writes go through command APIs only.
3. Money uses exact numeric type (or integer cents), never float.
4. Core workflow state is relational and constrained, not JSON blobs.
5. Every irreversible state change has append-only audit event rows.
## 9) Decision
This roadmap evidence supports continuing with the target architecture direction (command boundary + multi-tenant schema), but we should add attendance/offense/settlement/stakeholder-graph tables before full command rollout on mission-critical flows.

View File

@@ -0,0 +1,490 @@
# M4 Target Schema Blueprint (Command-Ready)
Status: Draft for team alignment
Date: 2026-02-25
Owner: Technical Lead
## 1) Goal
Define the target database shape we want **before** command-backend implementation, so critical flows are atomic, secure, and scalable.
## 1.1 Stakeholder and tenancy model
This product should be designed as a **multi-tenant platform**.
1. Tenant:
- One staffing company account (example: Legendary Event Staffing and Entertainment).
2. Business:
- A customer/client account owned by a tenant.
3. User:
- A human identity (auth account) that can belong to one or more tenants.
4. Staff:
- A workforce profile linked to a user identity and tenant-scoped operations.
Practical meaning:
1. The same platform can serve multiple staffing companies safely.
2. Data isolation is by `tenant_id`, not only by business/vendor IDs.
3. Not every record starts as a full active user:
- invite-first or pending onboarding records are valid,
- then bound to `user_id` when activation is completed.
4. Business-side users and vendor-side users are partitioned with dedicated membership tables, not only one `userId` owner field.
```mermaid
flowchart LR
U["User identity"] --> M["Tenant membership"]
M --> T["Tenant staffing company"]
T --> B["Business client"]
T --> V["Vendor partner"]
B --> O["Orders and shifts"]
V --> O
```
## 1.2 Stakeholder wheel mapping (current baseline)
The stakeholder labels from the customer workshop map to schema as follows:
1. Buyer (Procurements):
- Buyer users inside a business/client account.
- Schema anchor: `users` + `tenant_memberships` + `business_memberships`.
2. Enterprises (Operator):
- Tenant operator/admin users running staffing operations.
- Schema anchor: `tenants`, `tenant_memberships`, `role_bindings`.
3. Sectors (Execution):
- Operational segments or business units executing events.
- Schema anchor: `teams`, `team_hubs`, `team_hud_departments`, `roles`.
4. Approved Vendor:
- Supplier companies approved to fulfill staffing demand.
- Schema anchor: `vendors`, `vendor_memberships`, `workforce`, `vendor_rates`, `vendor_benefit_plans`.
5. Workforce:
- Individual workers/staff and their assignments.
- Schema anchor: `staffs`, `staff_roles`, `applications`, `assignments`, `certificates`, `staff_documents`.
6. Partner:
- External integration or service partner (future).
- Schema anchor: `stakeholder_profiles` extension path + scoped role bindings.
Rule:
1. Start with baseline stakeholders above.
2. Add new stakeholders via extension tables and role bindings, not by changing core scheduling and finance tables.
## 1.3 Future stakeholder expansion model
To add stakeholders later without breaking core schema:
1. Add `stakeholder_types` (registry).
2. Add `stakeholder_profiles` (`tenant_id`, `type`, `status`, `metadata`).
3. Add `stakeholder_links` (relationship graph across stakeholders).
4. Bind permissions through `role_bindings` with scope (`tenant`, `team`, `hub`, `business`, or specific resource).
## 1.4 Roadmap CSV evidence snapshot
Evidence source:
1. `docs/MILESTONES/M4/planning/m4-roadmap-csv-schema-reconciliation.md`
What the exports confirmed:
1. The product is multi-party and multi-tenant by design (client, operator, vendor, workforce, procurement, partner, dashboard).
2. Attendance and offense enforcement are core business workflows, not side features.
3. Finance requires more than invoices (payment runs, remittance, status history, dispute/audit trace).
4. Compliance requires asynchronous verification and requirement templates by tenant/business/role.
## 2) First-principles rules
1. Every critical write must be server-mediated and transactional.
2. Tenant boundaries must be explicit in data and queries.
3. Money and rates must use exact numeric types, not floating point.
4. Data needed for constraints should be relational, not hidden in JSON blobs.
5. Every high-risk state transition must be auditable and replayable.
## 3) Current anti-patterns we are removing
1. Direct client mutation of core entities.
2. Broad `USER`-auth CRUD without strict tenant scoping.
3. Financial values as `Float`.
4. Core workflow state embedded in generic `Any/jsonb` fields.
5. Missing uniqueness/index constraints on high-traffic paths.
## 4) Target modular schema
## 4.1 Identity and Access
Tables:
1. `users` (source identity, profile, auth linkage)
2. `tenant_memberships` (new; membership + base access per tenant)
3. `business_memberships` (new; user access to business account scope)
4. `vendor_memberships` (new; user access to vendor account scope)
5. `team_members` (membership + scope per team)
6. `roles` (new)
7. `permissions` (new)
8. `role_bindings` (new; who has which role in which scope)
Rules:
1. Unique tenant membership: `(tenant_id, user_id)`.
2. Unique business membership: `(business_id, user_id)`.
3. Unique vendor membership: `(vendor_id, user_id)`.
4. Unique team membership: `(team_id, user_id)`.
5. Access checks resolve through tenant membership first, then business/vendor/team scope.
## 4.2 Organization and Tenant
Tables:
1. `tenants` (new canonical boundary: business/vendor ownership root)
2. `businesses`
3. `vendors`
4. `teams`
5. `team_hubs`
6. `hubs`
Rules:
1. Every command-critical row references `tenant_id`.
2. All list queries must include tenant predicate.
3. Business and vendor routes must enforce membership scope before data access.
## 4.8 RBAC rollout strategy (deferred enforcement)
RBAC should be introduced in phases and **not enforced everywhere immediately**.
Phase A: Auth-first (now)
1. Require valid auth token.
2. Resolve tenant context.
3. Allow current work to continue while logging actor + tenant + action.
Phase B: Shadow RBAC
1. Evaluate permissions (`allow`/`deny`) in backend.
2. Log decisions but do not block most requests yet.
3. Start with warnings and dashboards for denied actions.
Phase C: Enforced RBAC on command writes
1. Enforce RBAC on `/commands/*` only.
2. Keep low-risk read flows in transition mode.
Phase D: Enforced RBAC on high-risk reads
1. Enforce tenant and role checks on sensitive read connectors.
2. Remove remaining broad user-level access.
```mermaid
flowchart LR
A["Auth only"] --> B["Shadow RBAC logging"]
B --> C["Enforce RBAC on command writes"]
C --> D["Enforce RBAC on sensitive reads"]
```
## 4.3 Scheduling and Orders
Tables:
1. `orders`
2. `order_schedule_rules` (new; replaces schedule JSON fields)
3. `shifts`
4. `shift_roles`
5. `shift_role_requirements` (optional extension for policy rules)
6. `shift_managers` (new; replaces `managers: [Any!]`)
Rules:
1. No denormalized `assignedStaff` or `shifts` JSON in `orders`.
2. Time constraints: `start_time < end_time`.
3. Capacity constraints: `assigned <= count`, `filled <= workers_needed`.
4. Canonical status names (single spelling across schema).
## 4.4 Staffing and Matching
Tables:
1. `staffs`
2. `staff_roles`
3. `workforce`
4. `applications`
5. `assignments`
Rules:
1. One active workforce relation per `(vendor_id, staff_id)`.
2. One application per `(shift_id, role_id, staff_id)` unless versioned intentionally.
3. Assignment state transitions only through command APIs.
## 4.5 Compliance and Verification
Tables:
1. `documents`
2. `staff_documents`
3. `certificates`
4. `verification_jobs`
5. `verification_reviews`
6. `verification_events`
Rules:
1. Verification is asynchronous and append-only for events.
2. Manual review is explicit and tracked.
3. Government ID and certification provider references are persisted.
## 4.6 Financial and Payout
Tables:
1. `invoices`
2. `invoice_templates`
3. `recent_payments`
4. `accounts` (refactor to tokenized provider references)
Rules:
1. Replace monetary `Float` with exact numeric (`DECIMAL(12,2)` or integer cents).
2. Do not expose raw account/routing values in query connectors.
3. Add one-primary-account constraint per owner.
## 4.7 Audit and Reliability
Tables:
1. `domain_events` (new)
2. `idempotency_keys` (already started in command API SQL)
3. `activity_logs`
Rules:
1. Every command write emits a domain event.
2. Idempotency scope: `(actor_uid, route, idempotency_key)`.
## 4.9 Attendance, Timesheets, and Offense Governance
Tables:
1. `attendance_events` (append-only: clock-in/out, source, correction metadata)
2. `attendance_sessions` (derived work session per assignment)
3. `timesheets` (approval-ready payroll snapshot)
4. `timesheet_adjustments` (manual edits with reason and actor)
5. `offense_policies` (tenant/business scoped policy set)
6. `offense_rules` (threshold ladder and consequence)
7. `offense_events` (actual violation events)
8. `enforcement_actions` (warning, suspension, disable, block)
Rules:
1. Attendance corrections are additive events, not destructive overwrites.
2. Offense consequences are computed from policy + history and persisted as explicit actions.
3. Manual overrides require actor, reason, and timestamp in audit trail.
## 4.10 Stakeholder Network Extensibility
Tables:
1. `stakeholder_types` (buyer, operator, vendor, workforce, partner, future types)
2. `stakeholder_profiles` (tenant-scoped typed profile)
3. `stakeholder_links` (explicit relationship graph between profiles)
Rules:
1. New stakeholder categories are added by data, not by schema rewrites to core workflow tables.
2. Permission scope resolves through role bindings plus stakeholder links where needed.
3. Scheduling and finance records remain stable while stakeholder topology evolves.
## 5) Target core model (conceptual)
```mermaid
erDiagram
TENANT ||--o{ BUSINESS : owns
TENANT ||--o{ VENDOR : owns
TENANT ||--o{ TEAM : owns
TEAM ||--o{ TEAM_MEMBER : has
USER ||--o{ TEAM_MEMBER : belongs_to
USER ||--o{ BUSINESS_MEMBERSHIP : belongs_to
USER ||--o{ VENDOR_MEMBERSHIP : belongs_to
BUSINESS ||--o{ BUSINESS_MEMBERSHIP : has
VENDOR ||--o{ VENDOR_MEMBERSHIP : has
BUSINESS ||--o{ ORDER : requests
VENDOR ||--o{ ORDER : fulfills
ORDER ||--o{ ORDER_SCHEDULE_RULE : has
ORDER ||--o{ SHIFT : expands_to
SHIFT ||--o{ SHIFT_ROLE : requires
SHIFT ||--o{ SHIFT_MANAGER : has
USER ||--o{ STAFF : identity
STAFF ||--o{ STAFF_ROLE : skills
VENDOR ||--o{ WORKFORCE : contracts
STAFF ||--o{ WORKFORCE : linked
SHIFT_ROLE ||--o{ APPLICATION : receives
STAFF ||--o{ APPLICATION : applies
SHIFT_ROLE ||--o{ ASSIGNMENT : allocates
WORKFORCE ||--o{ ASSIGNMENT : executes
ASSIGNMENT ||--o{ ATTENDANCE_EVENT : emits
ASSIGNMENT ||--o{ TIMESHEET : settles
OFFENSE_POLICY ||--o{ OFFENSE_RULE : defines
ASSIGNMENT ||--o{ OFFENSE_EVENT : may_trigger
OFFENSE_EVENT ||--o{ ENFORCEMENT_ACTION : causes
STAFF ||--o{ CERTIFICATE : has
STAFF ||--o{ STAFF_DOCUMENT : uploads
DOCUMENT ||--o{ STAFF_DOCUMENT : references
STAFF ||--o{ VERIFICATION_JOB : subject
VERIFICATION_JOB ||--o{ VERIFICATION_REVIEW : reviewed_by
VERIFICATION_JOB ||--o{ VERIFICATION_EVENT : logs
ORDER ||--o{ INVOICE : billed_by
INVOICE ||--o{ RECENT_PAYMENT : settles
TENANT ||--o{ ACCOUNT_TOKEN_REF : payout_method
INVOICE ||--o{ INVOICE_LINE_ITEM : details
PAYMENT_RUN ||--o{ PAYMENT_ALLOCATION : allocates
INVOICE ||--o{ PAYMENT_ALLOCATION : receives
PAYMENT_RUN ||--o{ REMITTANCE_DOCUMENT : publishes
ORDER ||--o{ DOMAIN_EVENT : emits
SHIFT ||--o{ DOMAIN_EVENT : emits
ASSIGNMENT ||--o{ DOMAIN_EVENT : emits
STAKEHOLDER_TYPE ||--o{ STAKEHOLDER_PROFILE : classifies
STAKEHOLDER_PROFILE ||--o{ STAKEHOLDER_LINK : relates
```
## 6) Command write boundary on this schema
```mermaid
flowchart LR
A["Frontend app"] --> B["Command API"]
B --> C["Policy + validation"]
C --> D["Single database transaction"]
D --> E["orders, shifts, shift_roles, applications, assignments"]
D --> F["domain_events + idempotency_keys"]
E --> G["Read models and reports"]
```
## 7) Minimum constraints and indexes to add before command build
## 7.1 Constraints
1. `shift_roles`: check `assigned >= 0 AND assigned <= count`.
2. `shifts`: check `start_time < end_time`.
3. `applications`: unique `(shift_id, role_id, staff_id)`.
4. `workforce`: unique active `(vendor_id, staff_id)`.
5. `team_members`: unique `(team_id, user_id)`.
6. `accounts` (or token ref table): unique primary per owner.
7. `attendance_events`: unique idempotency tuple (for example `(assignment_id, source_event_id)`).
8. `offense_rules`: unique `(policy_id, trigger_type, threshold_count)`.
## 7.2 Indexes
1. `orders (tenant_id, status, date)`.
2. `shifts (order_id, date, status)`.
3. `shift_roles (shift_id, role_id, start_time)`.
4. `applications (shift_id, role_id, status, created_at)`.
5. `assignments (workforce_id, shift_id, role_id, status)`.
6. `verification_jobs (subject_id, type, status, created_at)`.
7. `invoices (business_id, vendor_id, status, due_date)`.
8. `attendance_events (assignment_id, event_time, event_type)`.
9. `offense_events (staff_id, occurred_at, offense_type, status)`.
10. `invoice_line_items (invoice_id, line_type, created_at)`.
## 8) Data type normalization
1. Monetary: `Float -> DECIMAL(12,2)` (or integer cents).
2. Generic JSON fields in core scheduling: split into relational tables.
3. Timestamps: store UTC and enforce server-generated creation/update fields.
## 9) Security boundary in schema/connectors
1. Remove broad list queries for sensitive entities unless tenant-scoped.
2. Strip sensitive fields from connector query payloads (bank/routing).
3. Keep high-risk mutations behind command API; Data Connect remains read-first for client.
## 10) Migration phases (schema-first)
```mermaid
flowchart TD
P0["Phase 0: Safety patch
- lock sensitive fields
- enforce tenant-scoped queries
- freeze new direct write connectors"] --> P1["Phase 1: Core constraints
- add unique/check constraints
- add indexes
- normalize money types"]
P1 --> P2["Phase 2: Tenant and RBAC base tables
- add tenants and tenant_memberships
- add roles permissions role_bindings
- run RBAC in shadow mode"]
P2 --> P3["Phase 3: Scheduling normalization
- remove order JSON workflow fields
- add order_schedule_rules and shift_managers
- add attendance and offense base tables"]
P3 --> P4["Phase 4: Command rollout
- command writes on hardened schema
- emit domain events + idempotency
- enforce RBAC for command routes
- add finance settlement tables for payment runs and remittance"]
P4 --> P5["Phase 5: Read migration + cleanup
- migrate frontend reads as needed
- enforce RBAC for sensitive reads
- retire deprecated connectors"]
```
## 11) Definition of ready for command backend
1. P0 and P1 complete in `dev`.
2. Tenant scoping verified in connector tests.
3. Sensitive field exposure removed.
4. Core transaction invariants enforced by schema constraints.
5. Command API contracts mapped to new normalized tables.
6. RBAC is in shadow mode with decision logs in place (not hard-blocking yet).
7. Attendance and offense tables are ready for policy-driven command routes.
8. Finance settlement tables (`invoice_line_items`, `payment_runs`, `payment_allocations`) are available.
## 12) Full current model relationship map (all models)
```mermaid
flowchart LR
Account["Account"]
ActivityLog["ActivityLog"]
Application["Application"]
Assignment["Assignment"]
AttireOption["AttireOption"]
BenefitsData["BenefitsData"]
Business["Business"]
Category["Category"]
Certificate["Certificate"]
ClientFeedback["ClientFeedback"]
Conversation["Conversation"]
Course["Course"]
CustomRateCard["CustomRateCard"]
Document["Document"]
EmergencyContact["EmergencyContact"]
FaqData["FaqData"]
Hub["Hub"]
Invoice["Invoice"]
InvoiceTemplate["InvoiceTemplate"]
Level["Level"]
MemberTask["MemberTask"]
Message["Message"]
Order["Order"]
RecentPayment["RecentPayment"]
Role["Role"]
RoleCategory["RoleCategory"]
Shift["Shift"]
ShiftRole["ShiftRole"]
Staff["Staff"]
StaffAvailability["StaffAvailability"]
StaffAvailabilityStats["StaffAvailabilityStats"]
StaffCourse["StaffCourse"]
StaffDocument["StaffDocument"]
StaffRole["StaffRole"]
Task["Task"]
TaskComment["TaskComment"]
TaxForm["TaxForm"]
Team["Team"]
TeamHub["TeamHub"]
TeamHudDepartment["TeamHudDepartment"]
TeamMember["TeamMember"]
User["User"]
UserConversation["UserConversation"]
Vendor["Vendor"]
VendorBenefitPlan["VendorBenefitPlan"]
VendorRate["VendorRate"]
Workforce["Workforce"]
Application --> Shift
Application --> ShiftRole
Application --> Staff
Assignment --> ShiftRole
Assignment --> Workforce
BenefitsData --> Staff
BenefitsData --> VendorBenefitPlan
Certificate --> Staff
ClientFeedback --> Business
ClientFeedback --> Vendor
Course --> Category
Invoice --> Business
Invoice --> Order
Invoice --> Vendor
InvoiceTemplate --> Business
InvoiceTemplate --> Order
InvoiceTemplate --> Vendor
MemberTask --> Task
MemberTask --> TeamMember
Message --> User
Order --> Business
Order --> TeamHub
Order --> Vendor
RecentPayment --> Application
RecentPayment --> Invoice
Shift --> Order
ShiftRole --> Role
ShiftRole --> Shift
StaffAvailability --> Staff
StaffAvailabilityStats --> Staff
StaffDocument --> Document
StaffRole --> Role
StaffRole --> Staff
TaskComment --> TeamMember
TeamHub --> Team
TeamHudDepartment --> TeamHub
TeamMember --> Team
TeamMember --> TeamHub
TeamMember --> User
UserConversation --> Conversation
UserConversation --> User
VendorBenefitPlan --> Vendor
VendorRate --> Vendor
Workforce --> Staff
Workforce --> Vendor
```

View File

@@ -0,0 +1,264 @@
# M4 Target Schema Models, Keys, and Relationships
Status: Draft for architecture workshop
Date: 2026-02-25
Owner: Technical Lead
## 1) Purpose
This document is the model-level view for slide creation:
1. Key fields per model (`PK`, `FK`, unique keys).
2. How models relate to each other.
3. A first-principles structure for scale across tenant, business, vendor, and workforce flows.
## 2) Identity and access models
### 2.1 Model keys
| Model | Primary key | Foreign keys | Important unique keys |
|---|---|---|---|
| `users` | `id` | - | `email` (optional unique) |
| `tenants` | `id` | - | `slug` |
| `tenant_memberships` | `id` | `tenant_id -> tenants.id`, `user_id -> users.id` | `(tenant_id, user_id)` |
| `business_memberships` | `id` | `tenant_id -> tenants.id`, `business_id -> businesses.id`, `user_id -> users.id` | `(business_id, user_id)` |
| `vendor_memberships` | `id` | `tenant_id -> tenants.id`, `vendor_id -> vendors.id`, `user_id -> users.id` | `(vendor_id, user_id)` |
| `roles` | `id` | `tenant_id -> tenants.id` | `(tenant_id, name)` |
| `permissions` | `id` | - | `code` |
| `role_bindings` | `id` | `tenant_id -> tenants.id`, `role_id -> roles.id`, `user_id -> users.id` | `(tenant_id, role_id, user_id, scope_type, scope_id)` |
### 2.2 Diagram
```mermaid
erDiagram
USERS ||--o{ TENANT_MEMBERSHIPS : has
TENANTS ||--o{ TENANT_MEMBERSHIPS : has
USERS ||--o{ BUSINESS_MEMBERSHIPS : has
BUSINESSES ||--o{ BUSINESS_MEMBERSHIPS : has
TENANTS ||--o{ BUSINESS_MEMBERSHIPS : scopes
USERS ||--o{ VENDOR_MEMBERSHIPS : has
VENDORS ||--o{ VENDOR_MEMBERSHIPS : has
TENANTS ||--o{ VENDOR_MEMBERSHIPS : scopes
TENANTS ||--o{ ROLES : defines
ROLES ||--o{ ROLE_BINDINGS : used_by
USERS ||--o{ ROLE_BINDINGS : receives
TENANTS ||--o{ ROLE_BINDINGS : scopes
```
```
## 3) Organization and stakeholder models
### 3.1 Model keys
| Model | Primary key | Foreign keys | Important unique keys |
|---|---|---|---|
| `businesses` | `id` | `tenant_id -> tenants.id` | `(tenant_id, business_name)` |
| `vendors` | `id` | `tenant_id -> tenants.id` | `(tenant_id, company_name)` |
| `teams` | `id` | `tenant_id -> tenants.id` | `(tenant_id, team_name)` |
| `team_hubs` | `id` | `team_id -> teams.id` | `(team_id, hub_name)` |
| `team_hud_departments` | `id` | `team_hub_id -> team_hubs.id` | `(team_hub_id, name)` |
| `stakeholder_types` | `id` | - | `code` |
| `stakeholder_profiles` | `id` | `tenant_id -> tenants.id`, `stakeholder_type_id -> stakeholder_types.id` | `(tenant_id, stakeholder_type_id, name)` |
| `stakeholder_links` | `id` | `tenant_id -> tenants.id`, `from_profile_id -> stakeholder_profiles.id`, `to_profile_id -> stakeholder_profiles.id` | `(tenant_id, from_profile_id, to_profile_id, relation_type)` |
### 3.2 Diagram
```mermaid
erDiagram
TENANTS ||--o{ BUSINESSES : owns
TENANTS ||--o{ VENDORS : owns
TENANTS ||--o{ TEAMS : owns
TEAMS ||--o{ TEAM_HUBS : has
TEAM_HUBS ||--o{ TEAM_HUD_DEPARTMENTS : has
STAKEHOLDER_TYPES ||--o{ STAKEHOLDER_PROFILES : classifies
TENANTS ||--o{ STAKEHOLDER_PROFILES : owns
STAKEHOLDER_PROFILES ||--o{ STAKEHOLDER_LINKS : from
STAKEHOLDER_PROFILES ||--o{ STAKEHOLDER_LINKS : to
TENANTS ||--o{ STAKEHOLDER_LINKS : scopes
```
```
## 4) Workforce, orders, and assignments
### 4.1 Model keys
| Model | Primary key | Foreign keys | Important unique keys |
|---|---|---|---|
| `staffs` | `id` | `tenant_id -> tenants.id`, `user_id -> users.id` | `(tenant_id, user_id)` (nullable until activation if needed) |
| `staff_roles` | `id` | `staff_id -> staffs.id`, `role_id -> roles.id` | `(staff_id, role_id)` |
| `workforce` | `id` | `tenant_id -> tenants.id`, `vendor_id -> vendors.id`, `staff_id -> staffs.id` | active `(vendor_id, staff_id)` |
| `orders` | `id` | `tenant_id -> tenants.id`, `business_id -> businesses.id`, `vendor_id -> vendors.id` | `(tenant_id, external_ref)` optional |
| `order_schedule_rules` | `id` | `order_id -> orders.id` | `(order_id, rule_type, effective_from)` |
| `shifts` | `id` | `tenant_id -> tenants.id`, `order_id -> orders.id` | `(order_id, start_time, end_time)` |
| `shift_roles` | `id` | `shift_id -> shifts.id`, `role_id -> roles.id` | `(shift_id, role_id)` |
| `shift_managers` | `id` | `shift_id -> shifts.id`, `team_member_id -> team_members.id` | `(shift_id, team_member_id)` |
| `applications` | `id` | `tenant_id -> tenants.id`, `shift_id -> shifts.id`, `role_id -> roles.id`, `staff_id -> staffs.id` | `(shift_id, role_id, staff_id)` |
| `assignments` | `id` | `tenant_id -> tenants.id`, `shift_role_id -> shift_roles.id`, `workforce_id -> workforce.id` | `(shift_role_id, workforce_id)` active |
### 4.2 Diagram
```mermaid
erDiagram
TENANTS ||--o{ STAFFS : owns
USERS ||--o| STAFFS : identity
STAFFS ||--o{ STAFF_ROLES : has
ROLES ||--o{ STAFF_ROLES : classifies
TENANTS ||--o{ ORDERS : scopes
BUSINESSES ||--o{ ORDERS : requests
VENDORS ||--o{ ORDERS : fulfills
ORDERS ||--o{ ORDER_SCHEDULE_RULES : configures
ORDERS ||--o{ SHIFTS : expands_to
SHIFTS ||--o{ SHIFT_ROLES : requires
ROLES ||--o{ SHIFT_ROLES : typed_as
SHIFTS ||--o{ SHIFT_MANAGERS : has
VENDORS ||--o{ WORKFORCE : contracts
STAFFS ||--o{ WORKFORCE : linked
SHIFT_ROLES ||--o{ APPLICATIONS : receives
STAFFS ||--o{ APPLICATIONS : applies
SHIFT_ROLES ||--o{ ASSIGNMENTS : allocates
WORKFORCE ||--o{ ASSIGNMENTS : executes
```
```
## 5) Attendance and offense governance
### 5.1 Model keys
| Model | Primary key | Foreign keys | Important unique keys |
|---|---|---|---|
| `attendance_events` | `id` | `tenant_id -> tenants.id`, `assignment_id -> assignments.id` | `(assignment_id, source_event_id)` |
| `attendance_sessions` | `id` | `tenant_id -> tenants.id`, `assignment_id -> assignments.id` | one open session per assignment |
| `timesheets` | `id` | `tenant_id -> tenants.id`, `assignment_id -> assignments.id`, `staff_id -> staffs.id` | `(assignment_id)` |
| `timesheet_adjustments` | `id` | `timesheet_id -> timesheets.id`, `actor_user_id -> users.id` | - |
| `offense_policies` | `id` | `tenant_id -> tenants.id`, `business_id -> businesses.id` nullable | `(tenant_id, name, business_id)` |
| `offense_rules` | `id` | `policy_id -> offense_policies.id` | `(policy_id, trigger_type, threshold_count)` |
| `offense_events` | `id` | `tenant_id -> tenants.id`, `staff_id -> staffs.id`, `assignment_id -> assignments.id` nullable, `rule_id -> offense_rules.id` nullable | `(staff_id, occurred_at, offense_type)` |
| `enforcement_actions` | `id` | `offense_event_id -> offense_events.id`, `actor_user_id -> users.id` | - |
### 5.2 Diagram
```mermaid
erDiagram
ASSIGNMENTS ||--o{ ATTENDANCE_EVENTS : emits
ASSIGNMENTS ||--o{ ATTENDANCE_SESSIONS : opens
ASSIGNMENTS ||--o{ TIMESHEETS : settles
TIMESHEETS ||--o{ TIMESHEET_ADJUSTMENTS : adjusts
USERS ||--o{ TIMESHEET_ADJUSTMENTS : made_by
TENANTS ||--o{ OFFENSE_POLICIES : defines
BUSINESSES ||--o{ OFFENSE_POLICIES : overrides
OFFENSE_POLICIES ||--o{ OFFENSE_RULES : contains
STAFFS ||--o{ OFFENSE_EVENTS : incurs
OFFENSE_RULES ||--o{ OFFENSE_EVENTS : triggered_by
OFFENSE_EVENTS ||--o{ ENFORCEMENT_ACTIONS : causes
USERS ||--o{ ENFORCEMENT_ACTIONS : applied_by
```
```
## 6) Compliance and verification
### 6.1 Model keys
| Model | Primary key | Foreign keys | Important unique keys |
|---|---|---|---|
| `documents` | `id` | `tenant_id -> tenants.id` | `(tenant_id, document_type, name)` |
| `staff_documents` | `id` | `staff_id -> staffs.id`, `document_id -> documents.id` | `(staff_id, document_id)` |
| `certificates` | `id` | `staff_id -> staffs.id` | `(staff_id, certificate_number)` optional |
| `compliance_requirements` | `id` | `tenant_id -> tenants.id`, `business_id -> businesses.id` nullable, `role_id -> roles.id` nullable | `(tenant_id, requirement_type, business_id, role_id)` |
| `verification_jobs` | `id` | `tenant_id -> tenants.id`, `staff_id -> staffs.id`, `document_id -> documents.id` nullable | `(tenant_id, idempotency_key)` |
| `verification_reviews` | `id` | `verification_job_id -> verification_jobs.id`, `reviewer_user_id -> users.id` | - |
| `verification_events` | `id` | `verification_job_id -> verification_jobs.id` | - |
### 6.2 Diagram
```mermaid
erDiagram
STAFFS ||--o{ STAFF_DOCUMENTS : uploads
DOCUMENTS ||--o{ STAFF_DOCUMENTS : references
STAFFS ||--o{ CERTIFICATES : has
TENANTS ||--o{ COMPLIANCE_REQUIREMENTS : defines
BUSINESSES ||--o{ COMPLIANCE_REQUIREMENTS : overrides
ROLES ||--o{ COMPLIANCE_REQUIREMENTS : role_scope
STAFFS ||--o{ VERIFICATION_JOBS : subject
VERIFICATION_JOBS ||--o{ VERIFICATION_REVIEWS : reviewed
VERIFICATION_JOBS ||--o{ VERIFICATION_EVENTS : logs
USERS ||--o{ VERIFICATION_REVIEWS : reviewer
```
```
## 7) Finance and settlement
### 7.1 Model keys
| Model | Primary key | Foreign keys | Important unique keys |
|---|---|---|---|
| `invoices` | `id` | `tenant_id -> tenants.id`, `order_id -> orders.id`, `business_id -> businesses.id`, `vendor_id -> vendors.id` | `(tenant_id, invoice_number)` |
| `invoice_line_items` | `id` | `invoice_id -> invoices.id`, `assignment_id -> assignments.id` nullable | `(invoice_id, line_number)` |
| `invoice_status_history` | `id` | `invoice_id -> invoices.id`, `actor_user_id -> users.id` | - |
| `payment_runs` | `id` | `tenant_id -> tenants.id` | `(tenant_id, run_reference)` |
| `payment_allocations` | `id` | `payment_run_id -> payment_runs.id`, `invoice_id -> invoices.id` | `(payment_run_id, invoice_id)` |
| `remittance_documents` | `id` | `payment_run_id -> payment_runs.id` | `(payment_run_id, document_url)` |
| `account_token_refs` | `id` | `tenant_id -> tenants.id`, `owner_business_id -> businesses.id` nullable, `owner_vendor_id -> vendors.id` nullable | one primary per owner |
### 7.2 Diagram
```mermaid
erDiagram
ORDERS ||--o{ INVOICES : billed
BUSINESSES ||--o{ INVOICES : billed_to
VENDORS ||--o{ INVOICES : billed_from
INVOICES ||--o{ INVOICE_LINE_ITEMS : contains
INVOICES ||--o{ INVOICE_STATUS_HISTORY : transitions
USERS ||--o{ INVOICE_STATUS_HISTORY : changed_by
PAYMENT_RUNS ||--o{ PAYMENT_ALLOCATIONS : allocates
INVOICES ||--o{ PAYMENT_ALLOCATIONS : receives
PAYMENT_RUNS ||--o{ REMITTANCE_DOCUMENTS : publishes
TENANTS ||--o{ ACCOUNT_TOKEN_REFS : scopes
BUSINESSES ||--o{ ACCOUNT_TOKEN_REFS : business_owner
VENDORS ||--o{ ACCOUNT_TOKEN_REFS : vendor_owner
```
```
## 8) Audit and reliability
### 8.1 Model keys
| Model | Primary key | Foreign keys | Important unique keys |
|---|---|---|---|
| `domain_events` | `id` | `tenant_id -> tenants.id`, `actor_user_id -> users.id` | `(tenant_id, aggregate_type, aggregate_id, sequence)` |
| `idempotency_keys` | `id` | `tenant_id -> tenants.id`, `actor_user_id -> users.id` | `(tenant_id, actor_user_id, route, key)` |
| `activity_logs` | `id` | `tenant_id -> tenants.id`, `user_id -> users.id` | `(tenant_id, created_at, id)` |
### 8.2 Diagram
```mermaid
erDiagram
TENANTS ||--o{ DOMAIN_EVENTS : scopes
USERS ||--o{ DOMAIN_EVENTS : actor
TENANTS ||--o{ IDEMPOTENCY_KEYS : scopes
USERS ||--o{ IDEMPOTENCY_KEYS : actor
TENANTS ||--o{ ACTIVITY_LOGS : scopes
USERS ||--o{ ACTIVITY_LOGS : actor
```
```
## 9) Practical note for current system
Current schema already has:
1. `businesses.userId` (single business owner user).
2. `vendors.userId` (single vendor owner user).
3. `team_members` (multi-user workaround).
Target schema improves this by adding explicit:
1. `business_memberships`
2. `vendor_memberships`
This is the key upgrade for clean client/vendor user partitioning.

View File

@@ -0,0 +1,233 @@
# M4 Verification Architecture Contract (Attire, Government ID, Certification)
Status: Partially implemented in dev (core endpoints + async in-memory processor)
Date: 2026-02-24
Owner: Technical Lead
## Implementation status today (dev)
1. Implemented routes:
- `POST /core/verifications`
- `GET /core/verifications/{verificationId}`
- `POST /core/verifications/{verificationId}/review`
- `POST /core/verifications/{verificationId}/retry`
2. Current processor is in-memory and non-persistent (for fast frontend integration in dev).
3. Next hardening step is persistent job storage and worker execution before staging.
4. Attire uses a live Vertex vision model path with `gemini-2.0-flash-lite-001` by default.
5. Government ID and certification use third-party adapter contracts (provider URL/token envs) and fall back to `NEEDS_REVIEW` when providers are not configured.
## 1) Goal
Define a single backend verification pipeline for:
1. `attire`
2. `government_id`
3. `certification`
This contract gives the team exact endpoint behavior, state flow, and ownership before coding.
## 2) Principles
1. Upload is evidence intake, not final verification.
2. Verification runs asynchronously in backend workers.
3. Model output is a signal, not legal truth.
4. High-risk identity decisions require stronger validation and human audit trail.
5. Every decision is traceable (`who`, `what`, `when`, `why`).
## 3) Verification types and policy
## 3.1 Attire
1. Primary check: vision model + rule checks.
2. Typical output: `AUTO_PASS`, `AUTO_FAIL`, or `NEEDS_REVIEW`.
3. Manual override is always allowed.
## 3.2 Government ID
1. Required path for mission-critical use: third-party identity verification provider.
2. Model/OCR can pre-parse fields but does not replace identity verification.
3. Final status should require either provider success or manual approval by authorized reviewer.
## 3.3 Certification
1. Preferred path: verify against issuer API/registry when available.
2. If no issuer API: OCR extraction + manual review.
3. Keep evidence of the source used for validation.
## 4) State model
1. `PENDING`
2. `PROCESSING`
3. `AUTO_PASS`
4. `AUTO_FAIL`
5. `NEEDS_REVIEW`
6. `APPROVED`
7. `REJECTED`
8. `ERROR`
Rules:
1. `AUTO_*` and `NEEDS_REVIEW` are machine outcomes.
2. `APPROVED` and `REJECTED` are human outcomes.
3. All transitions are append-only in audit events.
## 5) API contract
## 5.1 Create verification job
1. Route: `POST /core/verifications`
2. Auth: required
3. Purpose: enqueue verification job for previously uploaded file.
4. Request:
```json
{
"type": "attire",
"subjectType": "staff",
"subjectId": "staff_123",
"fileUri": "gs://krow-workforce-dev-private/uploads/<uid>/item.jpg",
"rules": {
"attireType": "shoes",
"expectedColor": "black"
},
"metadata": {
"shiftId": "shift_123"
}
}
```
5. Success `202`:
```json
{
"verificationId": "ver_123",
"status": "PENDING",
"type": "attire",
"requestId": "uuid"
}
```
## 5.2 Get verification status
1. Route: `GET /core/verifications/{verificationId}`
2. Auth: required
3. Purpose: polling from frontend.
4. Success `200`:
```json
{
"verificationId": "ver_123",
"type": "attire",
"status": "NEEDS_REVIEW",
"confidence": 0.62,
"reasons": ["Color uncertain"],
"extracted": {
"detectedType": "shoe",
"detectedColor": "dark"
},
"provider": {
"name": "vertex-attire",
"reference": "gemini-2.0-flash-lite-001"
},
"review": null,
"createdAt": "2026-02-24T15:00:00Z",
"updatedAt": "2026-02-24T15:00:04Z",
"requestId": "uuid"
}
```
## 5.3 Review override
1. Route: `POST /core/verifications/{verificationId}/review`
2. Auth: required (reviewer role later; auth-first now + explicit reviewer id logging)
3. Purpose: final human decision and audit reason.
4. Request:
```json
{
"decision": "APPROVED",
"note": "Document matches required certification",
"reasonCode": "MANUAL_REVIEW"
}
```
5. Success `200`:
```json
{
"verificationId": "ver_123",
"status": "APPROVED",
"review": {
"decision": "APPROVED",
"reviewedBy": "user_456",
"reviewedAt": "2026-02-24T15:02:00Z",
"note": "Document matches required certification",
"reasonCode": "MANUAL_REVIEW"
},
"requestId": "uuid"
}
```
## 5.4 Retry verification job
1. Route: `POST /core/verifications/{verificationId}/retry`
2. Auth: required
3. Purpose: rerun failed or updated checks.
4. Success `202`: status resets to `PENDING`.
## 6) Worker execution flow
1. API validates payload and ownership of `fileUri`.
2. API writes `verification_jobs` row with `PENDING`.
3. Worker consumes job, marks `PROCESSING`.
4. Worker selects processor by type:
- `attire` -> model + rule scorer
- `government_id` -> provider adapter (+ optional OCR pre-check)
- `certification` -> issuer API adapter or OCR adapter
5. Worker writes machine outcome (`AUTO_PASS`, `AUTO_FAIL`, `NEEDS_REVIEW`, or `ERROR`).
6. Frontend polls status route.
7. Reviewer finalizes with `APPROVED` or `REJECTED` where needed.
## 7) Data model (minimal)
## 7.1 Table: `verification_jobs`
1. `id` (pk)
2. `type` (`attire|government_id|certification`)
3. `subject_type`, `subject_id`
4. `owner_uid`
5. `file_uri`
6. `status`
7. `confidence` (nullable)
8. `reasons_json`
9. `extracted_json`
10. `provider_name`, `provider_ref`
11. `created_at`, `updated_at`
## 7.2 Table: `verification_reviews`
1. `id` (pk)
2. `verification_id` (fk)
3. `decision` (`APPROVED|REJECTED`)
4. `reviewed_by`
5. `note`
6. `reason_code`
7. `reviewed_at`
## 7.3 Table: `verification_events`
1. `id` (pk)
2. `verification_id` (fk)
3. `from_status`, `to_status`
4. `actor_type` (`system|reviewer`)
5. `actor_id`
6. `details_json`
7. `created_at`
## 8) Security and compliance notes
1. Restrict verification file paths to owner-owned upload prefixes.
2. Never expose raw private bucket URLs directly.
3. Keep third-party provider secrets in Secret Manager.
4. Log request and decision IDs for every transition.
5. For government ID, keep provider response reference and verification timestamp.
## 9) Provider configuration (environment variables)
1. Attire model:
- `VERIFICATION_ATTIRE_PROVIDER=vertex`
- `VERIFICATION_ATTIRE_MODEL=gemini-2.0-flash-lite-001`
2. Government ID provider:
- `VERIFICATION_GOV_ID_PROVIDER_URL`
- `VERIFICATION_GOV_ID_PROVIDER_TOKEN` (Secret Manager recommended)
3. Certification provider:
- `VERIFICATION_CERT_PROVIDER_URL`
- `VERIFICATION_CERT_PROVIDER_TOKEN` (Secret Manager recommended)
4. Provider timeout:
- `VERIFICATION_PROVIDER_TIMEOUT_MS` (default `8000`)
## 10) Frontend integration pattern
1. Upload file via existing `POST /core/upload-file`.
2. Create verification job with returned `fileUri`.
3. Poll `GET /core/verifications/{id}` until terminal state.
4. Show machine status and confidence.
5. For `NEEDS_REVIEW`, show pending-review UI state.
## 11) Delivery split (recommended)
1. Wave A (fast): attire verification pipeline end-to-end.
2. Wave B: certification verification with issuer adapter + review.
3. Wave C: government ID provider integration + reviewer flow hardening.

192
makefiles/backend.mk Normal file
View File

@@ -0,0 +1,192 @@
# --- Backend Foundation (Cloud Run + Workers) ---
BACKEND_REGION ?= us-central1
BACKEND_ARTIFACT_REPO ?= krow-backend
BACKEND_CORE_SERVICE_NAME ?= krow-core-api
BACKEND_COMMAND_SERVICE_NAME ?= krow-command-api
BACKEND_RUNTIME_SA_NAME ?= krow-backend-runtime
BACKEND_RUNTIME_SA_EMAIL := $(BACKEND_RUNTIME_SA_NAME)@$(GCP_PROJECT_ID).iam.gserviceaccount.com
BACKEND_CORE_DIR ?= backend/core-api
BACKEND_COMMAND_DIR ?= backend/command-api
BACKEND_WORKERS_DIR ?= backend/cloud-functions
BACKEND_DEV_PUBLIC_BUCKET ?= krow-workforce-dev-public
BACKEND_DEV_PRIVATE_BUCKET ?= krow-workforce-dev-private
BACKEND_STAGING_PUBLIC_BUCKET ?= krow-workforce-staging-public
BACKEND_STAGING_PRIVATE_BUCKET ?= krow-workforce-staging-private
ifeq ($(ENV),staging)
BACKEND_PUBLIC_BUCKET := $(BACKEND_STAGING_PUBLIC_BUCKET)
BACKEND_PRIVATE_BUCKET := $(BACKEND_STAGING_PRIVATE_BUCKET)
BACKEND_RUN_AUTH_FLAG := --no-allow-unauthenticated
else
BACKEND_PUBLIC_BUCKET := $(BACKEND_DEV_PUBLIC_BUCKET)
BACKEND_PRIVATE_BUCKET := $(BACKEND_DEV_PRIVATE_BUCKET)
BACKEND_RUN_AUTH_FLAG := --allow-unauthenticated
endif
BACKEND_CORE_IMAGE ?= $(BACKEND_REGION)-docker.pkg.dev/$(GCP_PROJECT_ID)/$(BACKEND_ARTIFACT_REPO)/core-api:latest
BACKEND_COMMAND_IMAGE ?= $(BACKEND_REGION)-docker.pkg.dev/$(GCP_PROJECT_ID)/$(BACKEND_ARTIFACT_REPO)/command-api:latest
BACKEND_LOG_LIMIT ?= 100
BACKEND_LLM_MODEL ?= gemini-2.0-flash-001
BACKEND_VERIFICATION_ATTIRE_MODEL ?= gemini-2.0-flash-lite-001
BACKEND_VERIFICATION_PROVIDER_TIMEOUT_MS ?= 8000
BACKEND_MAX_SIGNED_URL_SECONDS ?= 900
BACKEND_LLM_RATE_LIMIT_PER_MINUTE ?= 20
.PHONY: backend-help backend-enable-apis backend-bootstrap-dev backend-migrate-idempotency backend-deploy-core backend-deploy-commands backend-deploy-workers backend-smoke-core backend-smoke-commands backend-logs-core
backend-help:
@echo "--> Backend Foundation Commands"
@echo " make backend-enable-apis [ENV=dev] Enable Cloud Run/Functions/Build/Secret APIs"
@echo " make backend-bootstrap-dev Bootstrap artifact repo, runtime SA, and buckets"
@echo " make backend-migrate-idempotency Create/upgrade idempotency table in Cloud SQL"
@echo " make backend-deploy-core [ENV=dev] Build + deploy core API service"
@echo " make backend-deploy-commands [ENV=dev] Build + deploy command API service"
@echo " make backend-deploy-workers [ENV=dev] Deploy worker scaffold"
@echo " make backend-smoke-core [ENV=dev] Smoke test core /health"
@echo " make backend-smoke-commands [ENV=dev] Smoke test commands /health"
@echo " make backend-logs-core [ENV=dev] Read core service logs"
backend-enable-apis:
@echo "--> Enabling backend APIs on project [$(GCP_PROJECT_ID)]..."
@for api in \
run.googleapis.com \
cloudbuild.googleapis.com \
artifactregistry.googleapis.com \
secretmanager.googleapis.com \
cloudfunctions.googleapis.com \
eventarc.googleapis.com \
aiplatform.googleapis.com \
storage.googleapis.com \
iam.googleapis.com \
iamcredentials.googleapis.com \
serviceusage.googleapis.com \
firebase.googleapis.com; do \
echo " - $$api"; \
gcloud services enable $$api --project=$(GCP_PROJECT_ID); \
done
@echo "✅ Backend APIs enabled."
backend-bootstrap-dev: backend-enable-apis
@echo "--> Bootstrapping backend foundation for [$(ENV)] on project [$(GCP_PROJECT_ID)]..."
@echo "--> Ensuring Artifact Registry repo [$(BACKEND_ARTIFACT_REPO)] exists..."
@if ! gcloud artifacts repositories describe $(BACKEND_ARTIFACT_REPO) --location=$(BACKEND_REGION) --project=$(GCP_PROJECT_ID) >/dev/null 2>&1; then \
gcloud artifacts repositories create $(BACKEND_ARTIFACT_REPO) \
--repository-format=docker \
--location=$(BACKEND_REGION) \
--description="KROW backend services" \
--project=$(GCP_PROJECT_ID); \
else \
echo " - Artifact Registry repo already exists."; \
fi
@echo "--> Ensuring runtime service account [$(BACKEND_RUNTIME_SA_NAME)] exists..."
@if ! gcloud iam service-accounts describe $(BACKEND_RUNTIME_SA_EMAIL) --project=$(GCP_PROJECT_ID) >/dev/null 2>&1; then \
gcloud iam service-accounts create $(BACKEND_RUNTIME_SA_NAME) \
--display-name="KROW Backend Runtime" \
--project=$(GCP_PROJECT_ID); \
else \
echo " - Runtime service account already exists."; \
fi
@echo "--> Ensuring runtime service account IAM roles..."
@gcloud projects add-iam-policy-binding $(GCP_PROJECT_ID) \
--member="serviceAccount:$(BACKEND_RUNTIME_SA_EMAIL)" \
--role="roles/storage.objectAdmin" \
--quiet >/dev/null
@gcloud projects add-iam-policy-binding $(GCP_PROJECT_ID) \
--member="serviceAccount:$(BACKEND_RUNTIME_SA_EMAIL)" \
--role="roles/aiplatform.user" \
--quiet >/dev/null
@gcloud iam service-accounts add-iam-policy-binding $(BACKEND_RUNTIME_SA_EMAIL) \
--member="serviceAccount:$(BACKEND_RUNTIME_SA_EMAIL)" \
--role="roles/iam.serviceAccountTokenCreator" \
--project=$(GCP_PROJECT_ID) \
--quiet >/dev/null
@echo "--> Ensuring storage buckets exist..."
@if ! gcloud storage buckets describe gs://$(BACKEND_PUBLIC_BUCKET) --project=$(GCP_PROJECT_ID) >/dev/null 2>&1; then \
gcloud storage buckets create gs://$(BACKEND_PUBLIC_BUCKET) --location=$(BACKEND_REGION) --project=$(GCP_PROJECT_ID); \
else \
echo " - Public bucket already exists: $(BACKEND_PUBLIC_BUCKET)"; \
fi
@if ! gcloud storage buckets describe gs://$(BACKEND_PRIVATE_BUCKET) --project=$(GCP_PROJECT_ID) >/dev/null 2>&1; then \
gcloud storage buckets create gs://$(BACKEND_PRIVATE_BUCKET) --location=$(BACKEND_REGION) --project=$(GCP_PROJECT_ID); \
else \
echo " - Private bucket already exists: $(BACKEND_PRIVATE_BUCKET)"; \
fi
@echo "✅ Backend foundation bootstrap complete for [$(ENV)]."
backend-migrate-idempotency:
@echo "--> Applying idempotency table migration..."
@test -n "$(IDEMPOTENCY_DATABASE_URL)" || (echo "❌ IDEMPOTENCY_DATABASE_URL is required" && exit 1)
@cd $(BACKEND_COMMAND_DIR) && IDEMPOTENCY_DATABASE_URL="$(IDEMPOTENCY_DATABASE_URL)" npm run migrate:idempotency
@echo "✅ Idempotency migration applied."
backend-deploy-core:
@echo "--> Deploying core backend service [$(BACKEND_CORE_SERVICE_NAME)] to [$(ENV)]..."
@test -d $(BACKEND_CORE_DIR) || (echo "❌ Missing directory: $(BACKEND_CORE_DIR)" && exit 1)
@test -f $(BACKEND_CORE_DIR)/Dockerfile || (echo "❌ Missing Dockerfile: $(BACKEND_CORE_DIR)/Dockerfile" && exit 1)
@gcloud builds submit $(BACKEND_CORE_DIR) --tag $(BACKEND_CORE_IMAGE) --project=$(GCP_PROJECT_ID)
@gcloud run deploy $(BACKEND_CORE_SERVICE_NAME) \
--image=$(BACKEND_CORE_IMAGE) \
--region=$(BACKEND_REGION) \
--project=$(GCP_PROJECT_ID) \
--service-account=$(BACKEND_RUNTIME_SA_EMAIL) \
--set-env-vars=APP_ENV=$(ENV),GCP_PROJECT_ID=$(GCP_PROJECT_ID),PUBLIC_BUCKET=$(BACKEND_PUBLIC_BUCKET),PRIVATE_BUCKET=$(BACKEND_PRIVATE_BUCKET),UPLOAD_MOCK=false,SIGNED_URL_MOCK=false,LLM_MOCK=false,LLM_LOCATION=$(BACKEND_REGION),LLM_MODEL=$(BACKEND_LLM_MODEL),LLM_TIMEOUT_MS=20000,MAX_SIGNED_URL_SECONDS=$(BACKEND_MAX_SIGNED_URL_SECONDS),LLM_RATE_LIMIT_PER_MINUTE=$(BACKEND_LLM_RATE_LIMIT_PER_MINUTE),VERIFICATION_ACCESS_MODE=authenticated,VERIFICATION_REQUIRE_FILE_EXISTS=true,VERIFICATION_ATTIRE_PROVIDER=vertex,VERIFICATION_ATTIRE_MODEL=$(BACKEND_VERIFICATION_ATTIRE_MODEL),VERIFICATION_PROVIDER_TIMEOUT_MS=$(BACKEND_VERIFICATION_PROVIDER_TIMEOUT_MS) \
$(BACKEND_RUN_AUTH_FLAG)
@echo "✅ Core backend service deployed."
backend-deploy-commands:
@echo "--> Deploying command backend service [$(BACKEND_COMMAND_SERVICE_NAME)] to [$(ENV)]..."
@test -d $(BACKEND_COMMAND_DIR) || (echo "❌ Missing directory: $(BACKEND_COMMAND_DIR)" && exit 1)
@test -f $(BACKEND_COMMAND_DIR)/Dockerfile || (echo "❌ Missing Dockerfile: $(BACKEND_COMMAND_DIR)/Dockerfile" && exit 1)
@gcloud builds submit $(BACKEND_COMMAND_DIR) --tag $(BACKEND_COMMAND_IMAGE) --project=$(GCP_PROJECT_ID)
@gcloud run deploy $(BACKEND_COMMAND_SERVICE_NAME) \
--image=$(BACKEND_COMMAND_IMAGE) \
--region=$(BACKEND_REGION) \
--project=$(GCP_PROJECT_ID) \
--service-account=$(BACKEND_RUNTIME_SA_EMAIL) \
--set-env-vars=APP_ENV=$(ENV),GCP_PROJECT_ID=$(GCP_PROJECT_ID),PUBLIC_BUCKET=$(BACKEND_PUBLIC_BUCKET),PRIVATE_BUCKET=$(BACKEND_PRIVATE_BUCKET) \
$(BACKEND_RUN_AUTH_FLAG)
@echo "✅ Command backend service deployed."
backend-deploy-workers:
@echo "--> Deploying worker scaffold for [$(ENV)]..."
@if [ ! -d "$(BACKEND_WORKERS_DIR)" ]; then \
echo "❌ Missing directory: $(BACKEND_WORKERS_DIR)"; \
exit 1; \
fi
@if [ -z "$$(find $(BACKEND_WORKERS_DIR) -mindepth 1 ! -name '.keep' -print -quit)" ]; then \
echo "⚠️ No worker code found in $(BACKEND_WORKERS_DIR). Skipping deployment."; \
exit 0; \
fi
@echo "⚠️ Worker deployment is scaffold-only for now."
@echo " Add concrete worker deployment commands once worker code is introduced."
backend-smoke-core:
@echo "--> Running core smoke check..."
@URL=$$(gcloud run services describe $(BACKEND_CORE_SERVICE_NAME) --region=$(BACKEND_REGION) --project=$(GCP_PROJECT_ID) --format='value(status.url)'); \
if [ -z "$$URL" ]; then \
echo "❌ Could not resolve URL for service $(BACKEND_CORE_SERVICE_NAME)"; \
exit 1; \
fi; \
TOKEN=$$(gcloud auth print-identity-token); \
curl -fsS -H "Authorization: Bearer $$TOKEN" "$$URL/health" >/dev/null && echo "✅ Core smoke check passed: $$URL/health"
backend-smoke-commands:
@echo "--> Running commands smoke check..."
@URL=$$(gcloud run services describe $(BACKEND_COMMAND_SERVICE_NAME) --region=$(BACKEND_REGION) --project=$(GCP_PROJECT_ID) --format='value(status.url)'); \
if [ -z "$$URL" ]; then \
echo "❌ Could not resolve URL for service $(BACKEND_COMMAND_SERVICE_NAME)"; \
exit 1; \
fi; \
TOKEN=$$(gcloud auth print-identity-token); \
curl -fsS -H "Authorization: Bearer $$TOKEN" "$$URL/health" >/dev/null && echo "✅ Commands smoke check passed: $$URL/health"
backend-logs-core:
@echo "--> Reading logs for core backend service [$(BACKEND_CORE_SERVICE_NAME)]..."
@gcloud run services logs read $(BACKEND_CORE_SERVICE_NAME) \
--region=$(BACKEND_REGION) \
--project=$(GCP_PROJECT_ID) \
--limit=$(BACKEND_LOG_LIMIT)