2026-07-10 · entity-bound BM25 breach fixed: 542.7ms → 12.6ms p50 (43x)

Fix (migration 0008 + adapter): entity_tags and kind join the bm25 index with keyword tokenizers (the default tokenizer splits account:0, so term_set could never match raw values — first attempt returned 0 hits), and entity-bound sparse queries become two stages: a Tantivy boolean pre-filter bounding the candidate set to the entity's own chunks, then the exact <@ subset residual over the materialized candidates. Filter-then-rank, never truncate-then-authorize.

• All retrieval paths back inside the <50ms p95 envelope at 1M.

2026-07-09 (later) · QPS under load, and the entity-bound BM25 breach

Setup: 1,000,000 chunks, same machine/profile. All numbers are in-process adapter calls — no HTTP hop, no query encoder (dense/hybrid end-to-end adds ~12ms p50 of client CPU). One shared adapter, 16-connection pool. New latency case: broad principal token AND entity_scope=["account:0"] — broad visibility maximizes the pushed-down candidate set the heap-side entity_tags <@ filter must chew through.

Load (load --sweep --duration-secs 20): closed loop, zero think time; N tasks looping a mixed workload — 70% hybrid recall (1%-selectivity token), 20% current_fact point reads, 10% activity-timeline reads. Latencies include waiting for one of the pool's 16 connections.

• This box saturates at ~165–170 QPS for this mix, by concurrency 4. 16x-ing offered concurrency buys zero throughput, only queue depth. The honest operating point is N=4: hybrid 48.9ms p95 excluding the encoder — with ~12ms encode added, hybrid under load is outside the 50ms envelope even at low concurrency.
• SPEC §4d targets are not met on this setup — target ≥300 QPS hybrid at p95 <50ms per 8-vCPU node; measured ~117 hybrid ops/s inside a 166-QPS mix, on a laptop running Postgres under a Docker VM. The ~2.5x gap is real; measuring on the reference shape is the required next pass.
• Point reads queue behind recall under load: current_fact p95 is 5.4ms at N=4 (vs 0.43ms idle). The get target (≥5k QPS at p95 <5ms) cannot be met from an undifferentiated pool; the planned L3 in-memory projection is the designed fix.
• Entity-bound + broad-visibility BM25 is the worst number ever recorded here: 724ms p95, ~14x the envelope. Dense recall is immune (the router sends tiny entity-bound subsets to exact scan).
• Honesty notes: the bench tenant's activity timeline holds ~2 rows — path coverage, not a scale claim. No HTTP hop, no rate limiting, no encoder anywhere in the load loop; a served deployment adds all three.

2026-07-09 (later) · BM25 pushdown: all paths inside the envelope

Root cause (pg_search 0.24.1): the array-overlap operator && is not pushable into Tantivy, and valid_to wasn't indexed — BM25 queries heap-fetched the entire text match set (~540k rows for a 2-term OR query) to evaluate scope filters before top-k. Fix (migration 0004 + adapter): visibility expressed as a Tantivy term_set — exact overlap semantics inside the boolean, matches nothing on an empty principal set (fail-closed preserved) — and valid_to added to the bm25 index. Zero heap-filter nodes remain (~1.9k buffers/query vs ~480k).

• Scoreboard at 1M, worst case per path (p95): get 0.5ms · dense 25.4ms · BM25 17.8ms · hybrid 17.9ms · +12ms encode ⇒ every retrieval path inside the <50ms p95 envelope, encoder included.
• Caveats recorded: term_set shifts raw scores by a constant (+1.0) — ranking identical, RRF unaffected; entity_tags <@ subset filtering remains a heap filter — an entity-bound + broad-visibility case must be benched before claiming that combination; the migration briefly drops search availability during rebuild (~1.6s at 1M).

2026-07-09 (later) · 1M chunks: the valley collapses, then the selectivity router kills it

Setup: 1,000,000 chunks, same machine/profile. Bulk-loaded with secondary indexes dropped (292s; the incremental path was ~12x slower). Along the way: Docker's 64MB default shm silently fails parallel HNSW builds (compose now sets shm_size: 1gb), and an accidental run against the index-less table showed brute-force at 11ms p50 beating HNSW's 72.6ms on the same 1%-filtered query — the planner picks graph traversal exactly where it loses. Fix: an adapter-side selectivity router — a sub-millisecond EXPLAIN row estimate routes ≤20k estimated matches to exact top-k (perfect recall, no graph), the rest to HNSW iterative scan.

• The <50ms p95 claim holds at 1M for scoped dense recall, encoder included: worst dense case 25.4ms p95 + ~12ms encode ≈ ~37ms p95 end-to-end. The valley is eliminated by routing, not tuning.
• The get path is flat with corpus size: 0.29ms p50 at 1M vs 0.33ms at 100k.
• BM25 at 1M is the new breach: ~280ms p50. The fast-field fix that held at 100k does not carry — pg_search scores the full match set before the filters bite; hybrid inherits it. Until fixed, the honest hybrid number at 1M is ~330ms p95.
• Planner-estimate routing depends on healthy pg_stats — ANALYZE after bulk loads is load-bearing.

2026-07-09 (later) · local query encoder measured

Setup: all-MiniLM-L6-v2 ONNX (384-d, matching the chunk schema), CPU-only via ONNX Runtime, single thread, 100 short queries, Apple M3 Pro. Semantic ordering verified by test (related queries must beat unrelated by >0.2 cosine).

• Inside the 5–15ms local-encoder budget. Warm-cache encoder load ~190ms; first-run model download ~4.6s.
• End-to-end dense recall including encoding at 100k / 1% selectivity: ~37ms p50 / ~48ms p95 (additive worst case) — the number the <50ms p95 claim must hold against as corpus size grows.

2026-07-09 (later) · BM25 fast-field fix

Change: scope-filter columns (tenant_id, visibility, confidentiality) added to the bm25 index as fast fields (migration 0003), letting pg_search filter inside Tantivy instead of on the heap. Same corpus/machine as above.

• 2.7x p99 improvement — BM25 back inside the 50ms envelope at p95/p99. Hybrid tail now dominated by the dense side's 1%-selectivity valley.

2026-07-09 · first curve (Milestone A, week 1)

Setup: 100,000 chunks (384-d vectors, HNSW m=16 / ef_construction=64), Postgres profile (ParadeDB pg17: pgvector iterative scans + pg_search BM25) in Docker on Apple M3 Pro / 36 GB. k=10, 200 queries per case, in-process client — no network hop, no query embedding (local ONNX encoder not yet built).

• The get path already beats its claim — sub-millisecond L1 point reads straight from Postgres, wide margin under the ~2–5ms budget.
• The filtered-ANN valley is real and now located. Latency is non-monotonic in selectivity: 0.1% is fastest (exact scan over a tiny posting set), 50% is near-unfiltered, and ~1% is the worst case — too selective for cheap graph traversal, too large for exact scan.
• All dense paths meet <50ms p95 at this corpus size — but 100k is 1–2 orders of magnitude below the honest ceiling. The next entry must be 1M+ before any latency claim leaves the repo.
• BM25's 87ms p99 is the first SLO breach — pg_search filtering on the heap after the Tantivy match.
• Missing from these numbers: query embedding, network hop, concurrency. Single-query latency only.