Add long-term memory: org-scoped, attributed to a subject, and expiring

The first store whose contents are deliberately fed back into a prompt, which
makes it a different kind of table from everything around it. Org-scoped by
product decision: a memory written while one recruiter worked is available to
the next, because a workspace's view of its own hiring should not reset per
seat.

It remembers both kinds asked for — operational facts and observations about
named people — and the second is why most of this code is provenance rather
than payload. "This applicant seemed unreliable", stored automatically and
read into a later hiring answer, is profiling under GDPR and is the artefact an
employment claim is built on. The only thing that makes holding it defensible
is that it can be listed, shown and erased, so:

  - subject_type and subject_id are mandatory for anything personal, refused at
    the door rather than defaulted, because a memory about somebody that names
    nobody cannot be shown to them or deleted for them;
  - Held() answers a subject access request and Forget() answers an erasure,
    each in one statement, and Forget is a soft delete so the erasure itself is
    recorded;
  - every memory carries its author and the run that wrote it, so "why did it
    say that" survives memory entering the picture, and an inference is never
    read back as if a person had written it;
  - everything expires. Ninety days by default: a hiring workspace changes
    shape over a quarter, and a stale fact read as a current one is worse than
    no memory at all.

The block the model sees is fenced and labelled on the same terms as retrieved
documents, for a stronger reason — a memory is text this system wrote about its
own users, so a model that treated it as an instruction would let one run steer
every run after it. It states the origin of each line and says plainly that a
memory is never a reason on its own to accept or reject anybody. That sentence
is pinned by a test.

Recall is semantic where an embedder exists and newest-first where it does not,
and says which happened rather than quietly returning recency. Five memories by
default: this competes for the same prompt as the tool catalogue and the
retrieved block, against a ceiling of 8,000 tokens a minute.

Migration 000017 is WRITTEN AND NOT APPLIED. Nothing is wired into the runtime
yet — this is the store and its rules, reviewable on its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-10-07 19:55:08 +05:30
parent 54309635a3
commit 8c51c22c86
5 changed files with 699 additions and 0 deletions

View File

@@ -0,0 +1,6 @@
SET search_path = public;
DROP INDEX IF EXISTS agent_memories_expiry_idx;
DROP INDEX IF EXISTS agent_memories_subject_idx;
DROP INDEX IF EXISTS agent_memories_org_live_idx;
DROP TABLE IF EXISTS agent_memories;

View File

@@ -0,0 +1,106 @@
-- ============================================================================
-- Long-term memory: what an agent may carry from one run into the next.
--
-- A run is one turn and agent_runs is an audit record that is never replayed.
-- This is the first store whose CONTENTS are deliberately fed back into a
-- prompt, which makes it a different kind of table from everything around it
-- and is why so much of it is provenance rather than payload.
--
-- ORG-SCOPED, per the product decision of 2026-10-07: a memory written while
-- one recruiter worked is available to the next, because a workspace's view of
-- its own hiring should not reset per seat. I5 still applies — org_id is NOT
-- NULL and every read carries the predicate.
--
-- WHY `subject_type` AND `subject_id` ARE NOT OPTIONAL.
-- Memories are of two kinds and the second one is regulated. A workspace fact
-- ("this venue staffs on Thursdays") is operational. An observation about a
-- named candidate is personal data that will influence a later hiring answer,
-- which under GDPR is profiling and under employment law is an artefact a
-- claim can be built on. The distinction has to be queryable, or "show me
-- everything held about this person" and "erase it" are not answerable:
--
-- SELECT … WHERE subject_type = 'candidate' AND subject_id = $1
-- DELETE … WHERE subject_type = 'candidate' AND subject_id = $1
--
-- so a subject access request and an erasure are each one statement.
--
-- WHY `source_run_id` IS NOT OPTIONAL EITHER. A memory that influenced an
-- answer must be traceable to the run that wrote it, or "why did it say that"
-- stops being answerable the moment memory is involved. ON DELETE SET NULL so
-- pruning runs does not destroy the memory, but the column exists so the chain
-- is there while the run is.
--
-- WHAT THIS TABLE DOES NOT DO. It does not decide. A memory enters a prompt as
-- context on the same terms as a retrieved document — fenced, labelled as data
-- — and every write still passes the confirmation gate. Nothing here can
-- reject a candidate; it can only be read alongside the records.
-- ============================================================================
SET search_path = public;
CREATE TABLE agent_memories (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
-- I5. The predicate goes in every read; a memory cannot cross a tenant.
org_id uuid NOT NULL REFERENCES organizations (id) ON DELETE CASCADE,
-- Who the memory is ABOUT, which is not who wrote it.
-- workspace — an operational fact with no personal subject
-- candidate — a job_applications or worker_profiles subject
-- user — a preference stated by a person about their own working
subject_type text NOT NULL,
subject_id uuid,
-- The memory itself, in the words it will be read back in.
text text NOT NULL,
-- Provenance. `author` distinguishes a memory a person wrote from one a
-- model inferred, because the second needs review and the first does not.
author text NOT NULL DEFAULT 'model',
source_run_id text REFERENCES agent_runs (run_id) ON DELETE SET NULL,
written_by uuid REFERENCES users (id) ON DELETE SET NULL,
-- Retrieval, on the same terms as knowledge_chunks so one implementation
-- serves both. Vectors from two models are not comparable, hence the model.
embedding real[],
embedding_model text NOT NULL DEFAULT '',
-- Memory decays. A fact with no expiry accumulates forever and is read back
-- long after it stopped being true, which is worse than not remembering.
created_date timestamptz NOT NULL DEFAULT now(),
expires_at timestamptz,
-- Soft delete, so an erasure is recorded as having happened rather than
-- leaving no trace that anything was there.
redacted_at timestamptz,
CONSTRAINT agent_memories_subject_check CHECK (
subject_type IN ('workspace', 'candidate', 'user')
),
-- A personal memory without a subject cannot be shown to the person it is
-- about, which makes it undeletable in practice. Refused at write time.
CONSTRAINT agent_memories_subject_id_required CHECK (
subject_type = 'workspace' OR subject_id IS NOT NULL
),
CONSTRAINT agent_memories_author_check CHECK (author IN ('model', 'person')),
CONSTRAINT agent_memories_text_not_blank CHECK (length(btrim(text)) > 0)
);
-- The read path: this tenant's live memories, newest first.
CREATE INDEX agent_memories_org_live_idx
ON agent_memories (org_id, created_date DESC)
WHERE redacted_at IS NULL;
-- Subject access and erasure, both of which are by subject.
CREATE INDEX agent_memories_subject_idx
ON agent_memories (org_id, subject_type, subject_id)
WHERE redacted_at IS NULL;
-- The sweep that enforces decay.
CREATE INDEX agent_memories_expiry_idx
ON agent_memories (expires_at)
WHERE expires_at IS NOT NULL AND redacted_at IS NULL;
COMMENT ON TABLE agent_memories IS
'What an agent may carry between runs. Org-scoped, attributed to a subject so it can be shown and erased, '
'and traceable to the run that wrote it. Read into prompts as context, never as a decision.';