Export a database's rows, so a tenant can move between deployments
Local work has been stranded on one machine: `make seed` replays seed/fixtures/seed.json, which is generated from krow-demo and has never reflected what is in a database. Agents published through importagents, knowledge ingested, applications screened — none of it had a path to another deployment. `make export-data` reads the database itself and writes replayable SQL. Three things it does deliberately: Tables are ordered topologically from pg_constraint rather than left in pg_dump's own order, which sorts by name and so fails on foreign keys in a way that depends on what the tables are called. Parents always precede children, and Kahn's algorithm breaks ties alphabetically so two runs against one schema produce a byte-identical file. Every statement is INSERT ... ON CONFLICT DO NOTHING. The export can create rows on a target and cannot modify or delete one. That is a property of the generated file, not a rule someone has to remember when they apply it. The file refuses to apply to a schema older than the one it came from. A restore into a half-migrated database half-succeeds, and a partial import is harder to unpick than a failed one. pg_dump 18 wraps its output in the psql meta-commands \restrict and \unrestrict. Dumping per table left the closing one without its opener, which fails with "not currently in restricted mode" and, under --single-transaction, rolls back having inserted nothing — silently, if the caller reads psql's output through a pipe instead of its exit code. Each table's slice is therefore cut at the last line ending in a semicolon, which no line of pg_dump's epilogue does and every generated statement does. sessions and schema_migrations are excluded: sessions are bound to cookies one deployment issued, and a stale schema_migrations row would make the target lie about its own version. Verified against a scratch database migrated to 15: all 25 tables match the source row for row, a second run is a no-op, and the guard refuses a version-10 target. The output is real tenant data — password hashes, personal details — so seed/exports/ is gitignored. It moves over scp, not through git. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
13
Makefile
13
Makefile
@@ -163,6 +163,19 @@ seed-fixture: ## Regenerate seed.json from the frontend seed module
|
||||
seed-fixture-check: ## Fail if seed.json no longer matches the frontend seed module
|
||||
cd "$(CURDIR)/../krow-demo" && npm run seed:check
|
||||
|
||||
# Moving a tenant between databases. `seed` replays the frontend's demo
|
||||
# fixture; this replays what is actually IN a database, so work done through
|
||||
# the API comes with it. The output is real tenant data and is gitignored —
|
||||
# carry it to the target over scp, never through git.
|
||||
.PHONY: export-data
|
||||
export-data: ## Export this database's rows as replayable SQL: make export-data [OUT=path]
|
||||
python3 scripts/export-local-data.py
|
||||
|
||||
.PHONY: import-data
|
||||
import-data: ## Apply an export to a target: make import-data TARGET_URL=postgres://… [IN=path]
|
||||
@test -n "$(TARGET_URL)" || { echo "import-data: TARGET_URL is required"; exit 1; }
|
||||
psql "$(TARGET_URL)" --single-transaction -v ON_ERROR_STOP=1 -f "$(or $(IN),seed/exports/local-data.sql)"
|
||||
|
||||
.PHONY: gen-resources
|
||||
gen-resources: ## Regenerate domain descriptors from the live schema
|
||||
python3 scripts/gen_resources.py > go-api/internal/domain/resources_gen.go
|
||||
|
||||
Reference in New Issue
Block a user