Export a database's rows, so a tenant can move between deployments
Some checks failed
CI / test (push) Failing after 4m37s
CI / fixture (push) Failing after 9s

Local work has been stranded on one machine: `make seed` replays
seed/fixtures/seed.json, which is generated from krow-demo and has never
reflected what is in a database. Agents published through importagents,
knowledge ingested, applications screened — none of it had a path to
another deployment.

`make export-data` reads the database itself and writes replayable SQL.

Three things it does deliberately:

Tables are ordered topologically from pg_constraint rather than left in
pg_dump's own order, which sorts by name and so fails on foreign keys in a
way that depends on what the tables are called. Parents always precede
children, and Kahn's algorithm breaks ties alphabetically so two runs
against one schema produce a byte-identical file.

Every statement is INSERT ... ON CONFLICT DO NOTHING. The export can
create rows on a target and cannot modify or delete one. That is a
property of the generated file, not a rule someone has to remember when
they apply it.

The file refuses to apply to a schema older than the one it came from. A
restore into a half-migrated database half-succeeds, and a partial import
is harder to unpick than a failed one.

pg_dump 18 wraps its output in the psql meta-commands \restrict and
\unrestrict. Dumping per table left the closing one without its opener,
which fails with "not currently in restricted mode" and, under
--single-transaction, rolls back having inserted nothing — silently, if
the caller reads psql's output through a pipe instead of its exit code.
Each table's slice is therefore cut at the last line ending in a
semicolon, which no line of pg_dump's epilogue does and every generated
statement does.

sessions and schema_migrations are excluded: sessions are bound to cookies
one deployment issued, and a stale schema_migrations row would make the
target lie about its own version.

Verified against a scratch database migrated to 15: all 25 tables match
the source row for row, a second run is a no-op, and the guard refuses a
version-10 target.

The output is real tenant data — password hashes, personal details — so
seed/exports/ is gitignored. It moves over scp, not through git.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-22 11:27:42 +05:30
parent 3455ad022f
commit 8e36faff89
3 changed files with 249 additions and 0 deletions

View File

@@ -163,6 +163,19 @@ seed-fixture: ## Regenerate seed.json from the frontend seed module
seed-fixture-check: ## Fail if seed.json no longer matches the frontend seed module
cd "$(CURDIR)/../krow-demo" && npm run seed:check
# Moving a tenant between databases. `seed` replays the frontend's demo
# fixture; this replays what is actually IN a database, so work done through
# the API comes with it. The output is real tenant data and is gitignored —
# carry it to the target over scp, never through git.
.PHONY: export-data
export-data: ## Export this database's rows as replayable SQL: make export-data [OUT=path]
python3 scripts/export-local-data.py
.PHONY: import-data
import-data: ## Apply an export to a target: make import-data TARGET_URL=postgres://… [IN=path]
@test -n "$(TARGET_URL)" || { echo "import-data: TARGET_URL is required"; exit 1; }
psql "$(TARGET_URL)" --single-transaction -v ON_ERROR_STOP=1 -f "$(or $(IN),seed/exports/local-data.sql)"
.PHONY: gen-resources
gen-resources: ## Regenerate domain descriptors from the live schema
python3 scripts/gen_resources.py > go-api/internal/domain/resources_gen.go