Rebase seeded time-series data to now, so the demo stops going quiet
Some checks failed
CI / test (push) Has been cancelled
CI / fixture (push) Has been cancelled

ShiftRecord is generated against now (shifts.go); everything else stayed on the
fixed calendar in seed.js while the calendar moved on. Twenty-three days after
that file was written the activity agent truthfully reported zero events in the
last seven days, and applications were sixteen days stale. Nothing was broken —
the data had simply aged out of every window the product reports over, and it
gets worse every day nobody reseeds.

RebaseToNow moves a set of records so the newest sits at now, keeping every gap
exactly as authored. The SHAPE is what every reader of this data is looking at:
three hires on one day, a screening the day after, a quiet fortnight before it.
Shifting the whole set by one delta preserves all of it. Scaling into a window
or scattering events across recent days would invent a rhythm nobody wrote.

Applied per entity — UserActivity, JobApplication, AIInterview — because each
anchors on its own newest record. One shared anchor would drag the quieter
entities by another entity's delta and invent relationships between them.
Reference data is untouched: a course's date is a fact about the course, not a
position in a window.

Rebasing rather than generating, so seed.json stays the single authored source,
still deterministic and still comparable byte-for-byte by the drift check. The
alternative — excluding these from the fixture the way ShiftRecord is — means a
second generator to keep in step with the frontend's copy.

The existing TestSeedPreservesSourceValues caught a real bug in the first
attempt, and its comment is why: "Applications carry updated_date in the
source, and the gap from created_date is what buildHires reads as
time-to-hire." I had shifted created_date alone, which turned five-day hires
into three-week ones. EVERY timestamp on a record now moves by the same delta,
and there is a test on that specifically.

That test now asserts the GAP rather than the absolute dates, because for a
rebased entity the dates are deliberately different — which is the one reason
it is supposed to allow. Its real subject was always the interval.

Verified against the database: before, activity was 23 days old with 0 events
in the last 7; after, all three entities are current, with 24 applications
spread across the last 30 days and 15 activity events inside 30.

Not fixed here: the Hiring activity chart on Control Center still renders
empty, and it was equally empty before this change. Its bucketing is correct —
replaying it in the browser against live data matched all 24 applications into
the right days — so the fault is further down in that component, not in the
data. Naming it rather than leaving it implied by a chart that still looks
wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PJvibeSc1JYXjatankqM1g
This commit is contained in:
2026-08-29 15:13:33 +05:30
parent 48ab9d1dad
commit c378bc00ed
4 changed files with 348 additions and 4 deletions

View File

@@ -0,0 +1,122 @@
package seeder
import (
"strings"
"time"
)
// RebaseToNow moves a fixture's records so the most recent one sits at `now`,
// keeping every gap between them exactly as authored.
//
// The problem this solves is that the demo goes quiet. ShiftRecord is generated
// against now (see shifts.go); everything else stayed on the fixed calendar in
// seed.js while the calendar moved on. Twenty-three days after that file was
// written the activity agent truthfully reported zero events in the last seven
// days, and the hiring chart on Control Center drew one bar in a thirty-day
// window. Nothing was broken; the data had simply aged out of every window the
// product asks about.
//
// Applied per entity rather than globally, because each is anchored on its own
// newest record. Rebasing them together against one shared anchor would drag
// the quieter entities forward or back by another entity's delta and invent
// relationships between them that nobody authored.
//
// Rebasing rather than generating, deliberately. ShiftRecord is excluded from
// seed.json because it is built fresh each run; doing the same here would mean
// a second generator to keep in step with the frontend's copy, and a fixture
// that no longer describes what a seeded workspace contains. Rebasing keeps
// seed.json the single authored source — still deterministic, still comparable
// byte-for-byte by the drift check — and moves the window instead of the data.
//
// The SHAPE is what matters to every reader of this data: three hires on one
// day, a screening the day after, a quiet fortnight before it. Shifting the
// whole set by one delta preserves all of that. Scaling it into a window, or
// scattering events across recent days, would invent a rhythm nobody authored.
//
// EVERY timestamp on a record moves by the same delta, not just the anchor.
// The gaps WITHIN a record are as load-bearing as the gaps between them: an
// application's created_date and updated_date are what buildHires subtracts to
// get time-to-hire, so shifting one and not the other invents a hire that took
// three weeks or minus one. The seeder's own TestSeedPreservesSourceValues
// caught exactly that, which is why it says what it says about the gap.
//
// Idempotent: the delta is recomputed from the fixture every run, so seeding
// twice rebases the same authored dates twice rather than compounding.
// Returns the records unchanged when there are none, or when none of them
// carry a usable anchor.
func RebaseToNow(records []map[string]any, field string, now time.Time) []map[string]any {
if len(records) == 0 {
return records
}
var newest time.Time
for _, rec := range records {
if at, ok := recordTime(rec, field); ok && at.After(newest) {
newest = at
}
}
if newest.IsZero() {
// Nothing parseable to anchor on. Leaving the fixture alone is the
// honest outcome: a wrong guess about these dates is worse than dates
// that are visibly old.
return records
}
delta := now.Sub(newest)
out := make([]map[string]any, 0, len(records))
for _, rec := range records {
if _, ok := recordTime(rec, field); !ok {
out = append(out, rec)
continue
}
// Copied, not mutated: the fixture is read once and seeded into
// possibly several organizations, and rewriting it in place would make
// the second one depend on the first.
shifted := make(map[string]any, len(rec))
for k, v := range rec {
shifted[k] = v
// Every timestamp moves together. A field that is not a timestamp,
// or a null one, is copied across untouched.
if at, isTime := recordTime(rec, k); isTime {
shifted[k] = at.Add(delta).UTC().Format(dateLayoutOf(rec[k]))
}
}
out = append(out, shifted)
}
return out
}
// recordTime reads one timestamp field, whatever shape the fixture used.
func recordTime(rec map[string]any, field string) (time.Time, bool) {
raw, ok := rec[field]
if !ok {
return time.Time{}, false
}
switch v := raw.(type) {
case time.Time:
return v, true
case string:
for _, layout := range []string{time.RFC3339Nano, time.RFC3339, "2006-01-02"} {
if at, err := time.Parse(layout, v); err == nil {
return at, true
}
}
}
return time.Time{}, false
}
// dateLayoutOf keeps a rewritten timestamp in the shape the fixture used, so a
// date-only field stays a date and a full timestamp keeps its precision.
// Anything else the seeder reads back would differ from the source for reasons
// that have nothing to do with the shift.
func dateLayoutOf(raw any) string {
if s, ok := raw.(string); ok {
if len(s) == len("2006-01-02") {
return "2006-01-02"
}
if strings.Contains(s, ".") {
return "2006-01-02T15:04:05.000Z"
}
}
return time.RFC3339
}

View File

@@ -0,0 +1,173 @@
package seeder_test
import (
"testing"
"time"
"github.com/krow/krow-backend/go-api/internal/seeder"
)
func activityFixture() []map[string]any {
// The authored shape: a burst, then a gap, then a later cluster.
return []map[string]any{
{"id": "act_1", "event_type": "create_position", "created_date": "2026-07-18T09:00:00.000Z"},
{"id": "act_2", "event_type": "apply_job", "created_date": "2026-07-24T09:00:00.000Z"},
{"id": "act_3", "event_type": "hire_candidate", "created_date": "2026-07-25T09:00:00.000Z"},
{"id": "act_4", "event_type": "login", "created_date": "2026-08-06T09:00:00.000Z"},
}
}
func mustTime(t *testing.T, rec map[string]any) time.Time {
t.Helper()
at, err := time.Parse(time.RFC3339, rec["created_date"].(string))
if err != nil {
t.Fatalf("created_date %v does not parse: %v", rec["created_date"], err)
}
return at
}
// The newest event lands on now, so the demo always has something recent.
func TestRebaseActivityAnchorsTheNewestEventToNow(t *testing.T) {
now := time.Date(2027, 3, 14, 12, 0, 0, 0, time.UTC)
out := seeder.RebaseToNow(activityFixture(), "created_date", now)
if len(out) != 4 {
t.Fatalf("got %d records, want 4", len(out))
}
newest := mustTime(t, out[3])
if !newest.Equal(now) {
t.Errorf("newest event at %s, want %s", newest, now)
}
for _, rec := range out {
if at := mustTime(t, rec); at.After(now) {
t.Errorf("%v is in the future at %s", rec["id"], at)
}
}
}
// Every gap is preserved. The shape of this data — three hires on one day, a
// quiet fortnight before it — is what every reader of it is looking at.
func TestRebaseActivityPreservesTheGaps(t *testing.T) {
in := activityFixture()
now := time.Date(2027, 3, 14, 12, 0, 0, 0, time.UTC)
out := seeder.RebaseToNow(in, "created_date", now)
for i := 1; i < len(in); i++ {
before := mustTime(t, in[i]).Sub(mustTime(t, in[i-1]))
after := mustTime(t, out[i]).Sub(mustTime(t, out[i-1]))
if before != after {
t.Errorf("gap %d changed from %s to %s", i, before, after)
}
}
}
// The whole point: events land inside the windows the activity tools ask about.
func TestRebaseActivityLandsInsideTheReportingWindows(t *testing.T) {
now := time.Date(2027, 3, 14, 12, 0, 0, 0, time.UTC)
out := seeder.RebaseToNow(activityFixture(), "created_date", now)
var last7, last30 int
for _, rec := range out {
age := now.Sub(mustTime(t, rec))
if age < 7*24*time.Hour {
last7++
}
if age < 30*24*time.Hour {
last30++
}
}
if last7 == 0 {
t.Error("no events in the last 7 days — the demo still looks dead, " +
"which is the condition this function exists to prevent")
}
if last30 != 4 {
t.Errorf("%d of 4 events in the last 30 days, want all of them", last30)
}
}
// Seeding twice must not compound the shift.
func TestRebaseActivityIsIdempotent(t *testing.T) {
now := time.Date(2027, 3, 14, 12, 0, 0, 0, time.UTC)
first := seeder.RebaseToNow(activityFixture(), "created_date", now)
second := seeder.RebaseToNow(activityFixture(), "created_date", now)
for i := range first {
if first[i]["created_date"] != second[i]["created_date"] {
t.Errorf("record %d differs between runs: %v vs %v",
i, first[i]["created_date"], second[i]["created_date"])
}
}
}
// The fixture is read once and may be seeded into several organizations, so
// rebasing must not rewrite it in place.
func TestRebaseActivityDoesNotMutateTheFixture(t *testing.T) {
in := activityFixture()
original := in[0]["created_date"]
seeder.RebaseToNow(in, "created_date", time.Date(2027, 3, 14, 12, 0, 0, 0, time.UTC))
if in[0]["created_date"] != original {
t.Errorf("the fixture was rewritten in place: %v became %v",
original, in[0]["created_date"])
}
}
func TestRebaseActivityHandlesNothingToDo(t *testing.T) {
now := time.Now()
if got := seeder.RebaseToNow(nil, "created_date", now); got != nil {
t.Errorf("nil in, %v out", got)
}
if got := seeder.RebaseToNow([]map[string]any{}, "created_date", now); len(got) != 0 {
t.Errorf("empty in, %d out", len(got))
}
// Unparseable dates are left alone rather than guessed at.
junk := []map[string]any{{"id": "x", "created_date": "not a date"}}
out := seeder.RebaseToNow(junk, "created_date", now)
if out[0]["created_date"] != "not a date" {
t.Errorf("an unreadable date was rewritten to %v", out[0]["created_date"])
}
}
// The gap WITHIN a record matters as much as the gaps between records.
// buildHires subtracts an application's created_date from its updated_date to
// get time-to-hire, so shifting one and not the other invents a hire that took
// three weeks or minus one. The seeder's TestSeedPreservesSourceValues caught
// this the first time; this pins it where the shifting happens.
func TestRebaseToNowShiftsEveryTimestampTogether(t *testing.T) {
in := []map[string]any{
{
"id": "app_1",
"created_date": "2026-07-20T09:00:00.000Z",
"updated_date": "2026-07-25T09:00:00.000Z", // five days later
"status": "hired",
"ai_score": 91,
},
{
"id": "app_2",
"created_date": "2026-08-06T09:00:00.000Z",
"updated_date": "2026-08-06T09:00:00.000Z",
"status": "applied",
},
}
now := time.Date(2027, 3, 14, 12, 0, 0, 0, time.UTC)
out := seeder.RebaseToNow(in, "created_date", now)
created := mustTime(t, out[0])
updated, err := time.Parse(time.RFC3339, out[0]["updated_date"].(string))
if err != nil {
t.Fatalf("updated_date did not survive as a timestamp: %v", out[0]["updated_date"])
}
if gap := updated.Sub(created); gap != 5*24*time.Hour {
t.Errorf("time-to-hire became %s, want 120h — the two timestamps moved "+
"by different amounts", gap)
}
// Non-date fields are untouched.
if out[0]["status"] != "hired" || out[0]["ai_score"] != 91 {
t.Errorf("a non-date field was rewritten: %+v", out[0])
}
// And a record whose two dates were equal still has them equal.
if out[1]["created_date"] != out[1]["updated_date"] {
t.Errorf("equal timestamps diverged: %v vs %v",
out[1]["created_date"], out[1]["updated_date"])
}
}

View File

@@ -127,6 +127,19 @@ func Load(path string) (*Fixture, error) {
return &f, nil
}
// rebasedEntities names the entities whose dates are moved to sit against now,
// and the field to move them by.
//
// ShiftRecord is absent because it is GENERATED against now rather than
// rebased (see shifts.go). Everything not listed here is reference data —
// courses, role categories, badges — where a date is a fact about the record
// rather than a position in a window, and moving it would be a lie.
var rebasedEntities = map[string]string{
"UserActivity": "created_date",
"JobApplication": "created_date",
"AIInterview": "created_date",
}
// New builds a seeder. `now` anchors the generated shift records.
func New(pool *pgxpool.Pool, fixture *Fixture, now time.Time) *Seeder {
return &Seeder{pool: pool, fixture: fixture, now: now}
@@ -158,6 +171,15 @@ func (s *Seeder) Run(ctx context.Context) (*Result, error) {
if entity == "ShiftRecord" {
records = BuildShifts(s.now)
}
// Time-series entities are authored on a fixed calendar and would
// otherwise age out of every window the product asks about — the
// activity tools, and the hiring chart on Control Center. See
// RebaseToNow. Each is anchored on its OWN newest record, so the gaps
// within an entity are preserved without inventing a relationship
// between entities.
if field, ok := rebasedEntities[entity]; ok {
records = RebaseToNow(records, field, s.now)
}
count, err := s.upsertEntity(ctx, tx, orgID, entity, records)
if err != nil {
return nil, fmt.Errorf("seed %s: %w", entity, err)

View File

@@ -3,6 +3,7 @@ package seeder_test
import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
@@ -153,11 +154,23 @@ func TestSeedPreservesSourceValues(t *testing.T) {
if status != want["status"] {
t.Errorf("%s status = %q, want %q", legacy, status, want["status"])
}
if created != want["created_date"] {
t.Errorf("%s created_date = %q, want %q", legacy, created, want["created_date"])
// Applications are REBASED (see RebaseToNow), so their absolute dates
// deliberately differ from the fixture — otherwise the demo ages out of
// every window the product reports over. What must survive is the GAP,
// because buildHires subtracts these two to get time-to-hire. Asserting
// the gap keeps this test's real subject and stops it failing for the
// one reason it is supposed to allow.
wantGap, err := fixtureGap(want["created_date"], want["updated_date"])
if err != nil {
t.Fatalf("%s: fixture dates: %v", legacy, err)
}
if updated != want["updated_date"] {
t.Errorf("%s updated_date = %q, want %q", legacy, updated, want["updated_date"])
gotGap, err := fixtureGap(created, updated)
if err != nil {
t.Fatalf("%s: seeded dates: %v", legacy, err)
}
if gotGap != wantGap {
t.Errorf("%s time-to-hire = %s, want %s (created %s, updated %s)",
legacy, gotGap, wantGap, created, updated)
}
if v, ok := want["ai_score"].(float64); ok && score != int(v) {
t.Errorf("%s ai_score = %d, want %d", legacy, score, int(v))
@@ -414,3 +427,17 @@ func TestFixtureIsGeneratedNotHandWritten(t *testing.T) {
t.Errorf("`_generated` does not name its source: %q", head.Generated)
}
}
// fixtureGap is the interval between two fixture timestamps.
func fixtureGap(from, to any) (time.Duration, error) {
const layout = "2006-01-02T15:04:05.000Z"
a, err := time.Parse(layout, fmt.Sprint(from))
if err != nil {
return 0, fmt.Errorf("parse %v: %w", from, err)
}
b, err := time.Parse(layout, fmt.Sprint(to))
if err != nil {
return 0, fmt.Errorf("parse %v: %w", to, err)
}
return b.Sub(a), nil
}