Knowledge layer

Learns across your customers. Never leaks them.

Scoped memory answers “what happened with customer A.” The knowledge layer answers “what have we learned across customers” — objection patterns, segment behaviors, playbooks that work, failure modes to avoid. The learning is valuable precisely because it crosses scopes, and dangerous for exactly the same reason. Verity's answer: generalization is a privilege earned through provable de-identification, never a default behavior of recall. It is arguably Verity's most differentiated capability — no OSS memory system (Mem0, Zep, Letta, Cognee) does cross-customer learning at all.

in one line

The knowledge layer publishes de-identified generalizations (“healthcare deals need DPA redlines”) that any scoped agent can read — and it's safe because each one is positively verified entity-free: past a deterministic de-identification gate, supported by ≥3 distinct customers, and human-approved (off by default). It is the one thing that safely crosses scopes.

Everything below is the depth behind that sentence — who generalizes, the gates, the merge cascade, and worked lifecycles.
the one-line thesis Learning crosses streams; specifics never do. A knowledge item is the one thing that safely crosses scopes — and it is safe for a precise, checkable reason: it is de-identified and human-gated. This is the permissions thesis (“scoped in the index, not the prompt”) applied to what an org learns.

The idea: an entity-free knowledge item

A knowledge item is a semantic memory whose subject is a category, never an entity. It carries no customer names, no quoted spans, no identifying amounts — only the generalization and the categories it applies to:

text
knowledge_item {
  statement:    "Healthcare-segment customers consistently require DPA
                 redlines before security review; budget ~2 extra weeks."
  categories:   ["industry:healthcare", "objection:dpa", "stage:security_review"]
  support:      { distinct_entities: 3, writers: 2, first_seen, last_reinforced }
  status:       candidate | quarantined | eligible | published | invalidated | rejected
  support_tier: emerging | established | extensive     // bucketed; what agents see
  visibility:   broad (org principal) once published
  lineage:      → supporting L2 facts → L0 episodes   // NEVER in the recall payload
  valid_from/to: bi-temporal like everything else — knowledge gets superseded
}

Knowledge is bi-temporal like every other memory in Verity: a generalization can be superseded (“post-repricing, this objection stopped appearing”). It ranks on similarity × confidence with a slow decay driven by last_reinforced — knowledge that stops being reinforced ages out of the top-k long before it is invalidated.

Episodic → semantic consolidation

The knowledge layer is a consolidation pipeline: it promotes scoped, episodic memory (what one agent saw with one customer) into org-level, semantic memory (a category-level pattern). It is async and sleep-time — it never runs on any hot path, never on a read.

Episodic-to-semantic consolidation Scoped episodic memory from many distinct customers feeds hypotheses; the consolidation worker clusters them across entities and accrues support; a deterministic de-identification gate, a k-distinct-entity floor, a category-size floor, and corroboration guard the candidate; a human review queue publishes an entity-free knowledge item, retrievable by any scoped agent via the section 7g carve-out. scoped episodic memory customer A · customer B · customer C (each agent sees only its own) consolidation worker cluster across DISTINCT entities · accrue support the gates (deterministic) de-id · k ≥ 3 · category floor · corroboration eligible → human review → publish nothing publishes automatically (auto-publish off by default) entity-free knowledge item (kind: knowledge) retrievable by any scoped agent — the §7g carve-out any gate failure → quarantined (auditable), never published
The promotion is async and sleep-time — never on a hot path. Lineage links every knowledge item back through its supporting L2 facts to L0 episodes, which is what makes the retraction cascade a lineage walk rather than a search.

Who generalizes: three roles, one honest asymmetry

An agent inside a scoped session sees one customer's interactions. It can notice something, but it structurally cannot know it is a pattern — it has no view across customers. So the roles split, and the split is the load-bearing safety property:

Agents — hypothesis generators & reinforcement voters not generalizers
A scoped agent calls memory_propose_learning. From one session this typically carries n=1 evidence — an expected-weak signal, unpublishable alone by construction (k-support). When many agents each propose something similar from their own single-customer view, the worker merges the similar proposals, adding each new entity and writer to the existing candidate's evidence. Many agents each seeing one interaction collectively assemble k-support — without any of them ever seeing across scopes.
The consolidation worker — the actual generalizer trusted server plane
The worker runs in the trusted server plane, like a connector. It is not an agent: it has no conversational output channel and never talks to a customer. It has legitimate cross-scope read access, and that is safe precisely because its only outflow is a candidate that must survive the de-identification gate. Contextual integrity is about where information flows; this component's only flow is gated. It clusters similar hypotheses across entities, drafts category-level candidates, and accrues support.
Humans (or configured policy) — publish the final gate
The review queue is the final gate. A candidate that crosses k-support becomes eligible, not published; a reviewer approves, edits, or rejects. Auto-publish is a per-tenant opt-in, off by default.
the asymmetry, stated plainly An agent that could read across scopes would be an information-flow leak. The worker can read across scopes and is not a leak, for one reason only: it cannot talk to anyone, and the one thing it can emit is forced through a de-identification gate and a human. Cross-scope reading is safe when the only outflow is gated.

The publish gates

Between a proposal and a published, org-wide knowledge item stand four deterministic gates. They are enforced in code and covered by tests — a false generalization is structurally harder to publish than a real one.

1 · De-identification gate deterministic, not vibes
The candidate statement is screened against the L1-derived lexicon of entity names, aliases, and domains; quoted-span detection against the source episodes; and identifying-value checks (amounts and dates that match restricted-class facts). Any hit is rejected back to scoped memory. An LLM may have written the candidate; a deterministic gate decides whether it can leave its scope. GET /v1/admin/knowledge/{id} exposes the gate result (passed + reason) on the review surface.
2 · k-distinct-entity support default k = 3
Published only when supported by evidence from ≥ k distinct entities (default K_SUPPORT_MIN = 3, per-tenant configurable). k=2 is explicitly refused as a default: with two supporting customers, either one can subtract their own interaction and infer the other's (membership inference). Support must also span ≥ 2 distinct writers or include Tier-1 evidence — one agent repeating itself across k entities must not self-promote (the poisoning path).
3 · Category-size floor deterministic
Every category a statement references must contain enough entities in L1 that the statement cannot deanonymize a single customer. “Our aerospace customers negotiate hard” deanonymizes perfectly when there is one aerospace customer, regardless of k.
4 · Human review the final gate
Gate-passing candidates land in the review queue as eligible. A human (or an explicitly configured policy) calls the publish endpoint. Auto-publish thresholds are configurable but OFF by default; publishing is what grants broad visibility and mints the retrievable carve-out chunk.
the provenance firewall Lineage from a knowledge item back to its supporting episodes exists — it powers invalidation, poisoning rollback, and audit — but it is never included in a recall or brief payload, and is readable only under an audit-class scope. Support counts exposed to agents are bucketed (a tier, never an exact number), to blunt membership inference.

The §7g retrieval carve-out — and why it's safe

Verity's zero-tag rule normally excludes untagged content from an entity-bound scope, because a missing tag might mean unclassified sensitive content (“untagged therefore almost nowhere”). A published knowledge item is the one principled exception:

the one exception A published knowledge item is not un-tagged — it is positively verified entity-free. It passed the de-identification gate, carries k-distinct-entity support, and holds status: published. So an agent in a session scoped to customer A retrieves exactly two things: (a) content tagged within its entity scope, and (b) published knowledge items matching the query — and nothing else.

The enforcement detail is what makes this trustworthy: the carve-out keys on the item's verified status, stamped into the index payload as kind: knowledge at publish time — never on the absence of tags. Candidates, eligible, and quarantined items stay invisible outside audit scopes. The scope-soundness fuzzer has dedicated cases: a quarantined item surfacing in any non-audit scope, or any tagged chunk sneaking through the carve-out, fails the build. The full treatment is on the permissions page.

The merge cascade

Merging is the only way k-support is ever reached: two agents on two different customers propose “the same” generalization in different words, and the worker must recognize them as the same and combine their evidence. This is where precision is won or lost.

the governing asymmetry A false merge is far worse than a missed merge. A missed merge means a real generalization fails to publish — a capability gap. A false merge fuses two distinct patterns into one item and fabricates support for it — and that item, once it crosses k=3, becomes broadly visible to every customer's agents via the carve-out. Precision dominates recall. Every tuning decision is a precision problem first.

So the merge is a three-stage cascade. Each stage can only reduce what merges; none can force a merge past the human gate:

The three-stage merge cascade A proposed candidate is canonicalized; an exact canonical match merges with no model call. Otherwise a cheap bi-encoder blocker with a low cosine threshold and category overlap shrinks the candidate set; an LLM judge in the worker makes the sameness decision, failing closed on uncertainty and recording a reason; a merge only accrues candidate support, and the human gate before publish is unchanged. canonicalize exact match → merge 1 · blocker low-τ cosine + category 2 · LLM judge same? fail-closed, reason 3 · human gate before PUBLISH merge only accrues candidate support blocker is recall-only (high recall / low precision) — it only bounds how many the judge sees uncertain / error / LLM unavailable → NO merge (the acceptable failure)
The blocker (a cheap bi-encoder, tuned low) exists only to shrink the search space; the real decision is the judge, an LLM already resident in the worker that returns a recorded, auditable reason. Fail-closed everywhere: any uncertainty yields the acceptable failure, a missed merge.

The old bare-cosine auto-merge (a single 0.85 threshold on the write path) has been removed: the server no longer decides a semantic merge on cosine alone, because a false merge fabricates cross-customer support. The stored statement embedding now feeds the blocker's candidate-set query, not an auto-merge.

Measured precision/recall — the honest operating point

You cannot tune what you cannot measure, and “trust us, we lowered a threshold” fails a security review. The merge decision is a benchmark metric (Scoped Recall Benchmark metric #6) over a labeled eval set of statement pairs — 206 pairs: 94 positives, 90 hard negatives, 22 easy negatives — where the hard negatives are statements that are topically close but genuinely distinct (“requires a DPA before security review” vs “requires a SOC 2 report before security review”). Encoder: all-MiniLM-L6-v2 (384-d).

operating pointprecisionrecallfalse-merge rate
shipped threshold 0.85 (cosine-only baseline)1.00000.00000.0000
lowest threshold holding precision ≥ 0.99 (0.73)1.00000.10640.0000
threshold 0.50 (for reference — below the contract)0.80520.65960.1339
the number we publish At the ≥99% precision the trust contract requires (false-merge rate ≤ 1%), the deterministic cosine-only judge reaches ~30% of true paraphrase merges — precisely, recall 0.1064 at threshold 0.73, with precision 1.0000 and a measured false-merge rate of 0.0000. Recall is the capability disclosure; precision is the guarantee. The LLM judge in the cascade above is designed to lift recall further — but that recall is not yet measured, so we don't claim a number for it. A CI regression gate fails the build if the false-merge rate on the eval set ever rises above target.

Acceptability controls

Auto-derived, cross-customer knowledge is the most trust-sensitive thing Verity does. These controls are the operator- and buyer-facing statement of what is never automatic. Everything here is enforced in code and covered by tests.

Publishing is never automatic on the read path
Merging only accrues candidate support. Crossing k-support (≥ 3 distinct entities, default) makes an item eligible — a status between candidate and published — not published. Nothing an agent reads is ever the trigger for a publish. POST /v1/knowledge/{id}/publish (admin bearer) stays the human gate; it re-enforces k-support + corroboration + de-id before it mints the carve-out chunk.
Auto-publish is opt-in, default OFF
A per-tenant setting (knowledge_auto_publish, absent = OFF). With the default posture, an eligible item waits for a human or an explicitly configured policy. Even when ON, promotion runs through the same publish gate on the async/admin path — still never on the read path — and if no default audience is configured the item is held eligible rather than published to an unknown audience (fail-safe). The OSS build ships OFF.
Kill switch — VERITY_KNOWLEDGE_AUTO_MERGE
Default ON. Set to 0 to disable the worker-judged merge leg entirely: the server then ignores worker-supplied merge_into, and only the deterministic canonical-exact fast path can merge. Consolidation degrades to assisted / human-clustered — candidates queue for human review — never a silent judged merge. The canonical-exact merge and every gate keep working under the kill switch.
Support tiers — buckets, never exact counts
Published and eligible items carry a bucketed support_tier derived from the distinct-entity count. A consuming agent sees only the tier on a kind=knowledge recall hit, so it can weight the knowledge without a false precision. Exact distinct_entities counts are admin-only.
tierdistinct entities
emerging3–4
established5–9
extensive10+
Reject-with-memory
POST /v1/admin/knowledge/{id}/reject lets a reviewer refuse a candidate or eligible item. Rejection is remembered: status becomes rejected and the same canonical_statement will not resurface as a fresh candidate — the propose path returns the remembered rejected row unchanged. Rejecting a published item is refused: retraction is memory.forget's job, which runs the k-support recount and auto-invalidation cascade.
the one-paragraph stance for a security review Verity learns patterns across your customers but never lets one customer's specifics reach another. Generalizations are de-identified deterministically, must be independently supported by ≥ 3 distinct customers, are judged for sameness by a model whose reasoning is recorded and auditable, and are never published without human approval — which is off by default. A wrong generalization is structurally harder to publish than a real one is, and both are fully reversible.

A lifecycle, end to end

The whole loop, from a scoped agent's hypothesis to what a different, later agent retrieves. Every command matches a real route or tool. Assume a running server, $VERITY, $TENANT, and $VERITY_ADMIN_TOKEN.

The propose call

A scoped agent proposes what it noticed. Over MCP this is the memory_propose_learning tool; over REST it is POST /v1/knowledge. The statement is about a category, never a customer — a statement carrying a known entity identifier is quarantined, not published.

MCP tool call — memory_propose_learning

json
{
  "tool": "memory_propose_learning",
  "arguments": {
    "scope_handle": "<acme-scoped handle from memory_open_scope>",
    "statement": "Healthcare-segment customers require DPA redlines before security review; budget ~2 extra weeks.",
    "categories": ["industry:healthcare", "objection:dpa"],
    "evidence": ["ep-8f1c…"]        // supporting episode ids; attribution read server-side
  }
}
// → { "status": "candidate", "knowledge_id": "k-3f1…" }  — visible only in audit

From one scoped session this is n=1: a candidate, not a publish, and unpublishable alone. A similar proposal from a different customer's agent is what accrues support — the worker's cascade decides whether the two are the same generalization and, if so, merges their evidence.

propose → eligible → review → publish

Once support crosses k across distinct entities and writers, the item becomes eligible and appears in the admin review queue. A human inspects the evidence and publishes.

bash
# the review queue (admin) — status, admin-exact distinct_entities, bucketed tier, merge reason
curl -s "$VERITY/v1/knowledge?tenant_id=$TENANT&status=eligible" \
  -H "authorization: Bearer $VERITY_ADMIN_TOKEN" \
  | jq '.items[] | {statement, status, support_tier, distinct_entities, merge_reason}'

# full detail for one item — adds the de-identification gate result (passed + reason)
curl -s "$VERITY/v1/admin/knowledge/$ID?tenant_id=$TENANT" \
  -H "authorization: Bearer $VERITY_ADMIN_TOKEN"

# publish — the human gate; k_min is clamped to ≥3 server-side, de-id re-checked
curl -s -X POST "$VERITY/v1/knowledge/$ID/publish" \
  -H "authorization: Bearer $VERITY_ADMIN_TOKEN" \
  -d '{"tenant_id":"'"$TENANT"'","visibility":[1],"k_min":3}'

# or refuse it — rejection is REMEMBERED; the same canonical statement won't resurface
curl -s -X POST "$VERITY/v1/admin/knowledge/$ID/reject" \
  -H "authorization: Bearer $VERITY_ADMIN_TOKEN" \
  -d '{"tenant_id":"'"$TENANT"'","reason":"too broad; not actually DPA-specific"}'

The read-only /ui “Knowledge review” panel lists candidates, eligible, and published items with a status badge, support tier, merge reason, and evidence count — a pure inspector; approve and reject are documented API actions, never buttons on the page. The consolidation worker itself drives /v1/admin/consolidation/lease → /merge-candidates (the blocker) → /complete (with the judge's {merge_into, judge_reason}).

What a consuming agent sees

A completely different agent — scoped to a new healthcare customer, months later — recalls the published lesson via the carve-out, alongside its own account's tagged content, and nothing else. It sees the bucketed support tier, never an exact customer count.

bash
curl -s -X POST "$VERITY/v1/recall" \
  -d "{\"scope_handle\":\"$NEWCO_SCOPE\",\"text\":\"what slows down healthcare security review?\"}" \
  | jq '.[] | {kind, content, support_tier, acl_provenance}'
// → {
//     "kind": "knowledge",
//     "content": "Healthcare-segment customers consistently require DPA redlines before security review…",
//     "support_tier": "emerging",        // 3–4 distinct entities — a bucket, never an exact count
//     "acl_provenance": "admin-assigned"
//   }
safe by construction The item is retrievable in this entity-bound scope only because it is positively verified entity-free, keyed on status: published and kind: knowledge. The lineage back to the three original customers exists for audit and retraction — but it is never in this payload, and the count the agent sees is a bucket. What those three customers' interactions were never crossed streams; what was learned across them is available here.

Honest limits

The site's credibility is a selling point, so the limits are stated as plainly as the capabilities: