Skip to main content

​
Searching Everything at Once

What this solves: Strategic knowledge lives in four different places. Searching them one at a time means you miss things, forget to check a source, or spend ten minutes on retrieval before the real work starts.

​
The Problem

After six months of building a knowledge system, I had a documentation vault with 2,600+ markdown files, a live memory database, a session history store, and the files themselves as a keyword-searchable corpus. Useful information could be in any of them, and often the most valuable insight appeared in multiple sources at once. Searching meant running four separate queries, mentally deduplicating, and figuring out which results mattered. I’d forget to check session history, or search the vault but miss a memory observation that contradicted what the vault article said. The retrieval step was eating 5-10 minutes before real work could start.

​
How It Works

The recall skill fires all four search layers at once and uses the LLM as a natural-language reranker to synthesize results. A full federated search runs in 3-5 seconds. Layer 1 — Vault Search uses a hybrid retrieval engine combining BM25 keyword matching, vector embeddings, and LLM reranking across 2,600+ documents. The hybrid approach matters because marketing language is inconsistent — the same concept gets called different things in different contexts. Layer 2 — Live Memory queries a persistent observation database filtered to decisions and session summaries. The full database has 13,000+ entries, but most are tool-level minutiae. Filtering by type is what makes this layer useful rather than noisy. Layer 3 — Session History searches structured records of past work sessions with type-scoping. Session summaries with their request/learned/completed fields are far more useful than raw observation dumps. Layer 4 — Keyword Index runs fast exact-match grep across key directories. This catches things semantic layers miss — specific names, acronyms, or identifiers that don’t embed well. The synthesis layer does two things individual searches can’t: cross-source detection (flagging when the same concept appears in multiple layers) and gap reporting (noting when a topic has zero hits across all sources — itself useful information).

​
In Practice

The most common trigger is “what do we know about X?” before starting new work. Before this existed, I’d search the vault, maybe remember to check memory, forget session history, and start with incomplete context. A recent example: before drafting a competitive positioning brief, I ran a recall query on a competitor. The vault returned two analysis documents. Live memory surfaced a decision from a stakeholder meeting about how to frame the comparison. Session history showed we’d discussed this competitor in three prior sessions, with the most recent noting our initial approach had been corrected by leadership. The keyword index caught a mention in meeting notes the semantic search had missed. The whole retrieval took about 4 seconds. Assembling that manually would have taken 8-10 minutes, and I almost certainly would have missed the session history correction, which turned out to be the most important piece.

​
What I Learned Building This

The design principle that made this work was “always search all four layers.” Early versions tried to be smart about which layers to query based on the question type. That optimization was wrong. The whole point of federation is catching things you didn’t expect to find in a particular source. The decision stored in session history that contradicts the vault article is exactly the result you’d miss with clever routing. Cross-source hits turned out to be the most valuable signal in the system. When the same concept appears in a vault document and a live memory observation and a session record, that’s not redundancy — it’s confirmation. I started flagging these explicitly in results, and they consistently point to the highest-confidence information. The live memory layer was the trickiest. Querying 13,000+ observations unfiltered drowns results in noise. Filtering by type before searching fixed it — decisions and session summaries update strategic understanding. The thousands of discovery-type observations are useful for the system’s internal operation but useless for strategic recall. That single filter took the layer from noisy to essential.