GraphRAG vs hybrid RAG: testing Microsoft Research's claim on 13,000 private notes

The claim being tested

Microsoft Research's 2024 GraphRAG paper (Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization) argues that when a language model first turns a corpus into a graph of entities, relationships and claims, the answers to questions about the whole corpus become more comprehensive than with conventional vector retrieval.

This case study tests that claim on a real, private corpus: an Obsidian vault of 13,284 Markdown notes (14 million words) that I use as a personal and business knowledge base. It is not a client deployment. The test started with one folder, the 623 notes behind Mindreader, the sales-psychology software company where I am CTO, because that folder holds five years of decisions, pivots, metrics and contradictions.

It follows an earlier piece, a private search engine over the same vault, which built the hybrid search (keyword plus semantic, fused by rank) that GraphRAG had to beat.

That hybrid baseline is already a step beyond what most AI chat tools do. When ChatGPT, Claude or a similar assistant searches company documents through a connector, it mostly relies on sparse (keyword) retrieval, because the connected systems hold no embeddings of the documents. Sparse retrieval finds the words in a question, not its meaning. Dense retrieval, which matches on embeddings, is the missing piece that improves search quality most, and it no longer needs a hosted service: the embedding model used here, bge-m3, is a 1.2 GB local model that runs on a laptop.

The constraints

Four constraints shaped every decision.

1. Private by default. The notes include customer calls, finances and journals. The graph had to live on a laptop (Neo4j Desktop, local), embeddings had to be computed locally, and the only text leaving the machine was the one-time extraction call to the Anthropic API, folder by folder, with personal folders opt-in.
2. Notes contradict each other over time. Paying-customer counts, positioning and pricing appear in dozens of notes from different years, and the numbers differ. A useful system must say "was X in 2025, later Y", not average them.
3. Plans are not facts. Many notes are strategy drafts or saved ChatGPT conversations. "We will close 10 enterprise clients" is a plan, and "ChatGPT suggested this headline" is not a decision. Vector search cannot tell those apart from a record of what happened.
4. Most files were online-only. 82% of the vault's Markdown files were Google Drive placeholders. Reading them meant fetching text on demand (99 MB of Markdown out of an 11.4 GB vault) without pulling down attachments.

The model: what goes into the graph and why

Three layers, one rule about evidence

The graph has three layers, and every fact in the top layer points back to a passage in the middle one.

Layer	Nodes	Built by	Cost
Structure	Note, Tag, Folder, with LINKS_TO, TAGGED, IN_FOLDER	Parsing the Markdown (wikilinks, front matter, folders)	Free, ~2 minutes for 13,284 notes
Evidence	Chunk, with a 1,024-dimension embedding and NEXT_CHUNK	Local chunker + local embedding model	Free, ~36 minutes for 4,779 chunks
Knowledge	Entity, Claim, and 29 typed relationships	Claude Haiku 5.5, once per chunk, cached	~$3.20–3.60 per full pass of the folder

The structure layer alone already holds 45,000 note-to-note links, and it showed 1,571 links pointing at notes that do not exist. The vault's own wiki notes (wiki/entities, wiki/concepts) became canonical records that extracted entities link to with DESCRIBED_BY.

Chunking: one-time, local, and sized for the extractor

The first attempt used the embeddings from Obsidian's Smart Connections plugin. Both it and Neo4j run locally, but Smart Connections keeps its vectors in its own index, apart from the links, dates and keyword search, which made them hard to traverse. Neo4j stores the vectors next to the notes, links and claims, so retrieval and traversal happen in one query. The plugin's blocks also had a median of 49 characters, too small to carry a decision and its reason. The chunker was rebuilt with one rule: a note of up to 1,500 tokens stays whole; a longer note is split at its headings into sections of at most 1,000 tokens (minimum 100), with 200 characters of overlap, and every chunk carries a breadcrumb (Note title > Heading > Subheading) and the note's date. The Mindreader folder produced 4,779 chunks with a median of 837 tokens.

The embedding model changed too, from Smart Connections' 384-dimension bge-micro-v2 (66 MB, English only) to bge-m3 through Ollama: 1,024 dimensions, an 8,000-token window, and multilingual, which matters because some notes are in Chinese. Each chunk stores a hash of its breadcrumb and text, so a rebuild re-embeds only what changed.

Competency questions decided the schema

The schema was not designed from a generic ontology. It started from the questions the graph should answer: receipts and proof, timelines, people and introductions, decisions and their reasons, lessons, clients and outcomes, changes and conflicts, and connections across projects. Each entity type and relationship had to serve at least one of them.

• Ten entity types: Person, Organization, Product, Project, Concept, Event, Location, Document, Decision and Lesson. Decision and Lesson are first-class nodes, so "why did we decide X and what did it lead to" is a path (DECIDED, LED_TO, LEARNED_FROM), not a text search.
• 29 relationship types, from CLIENT_OF, INVESTED_IN and INTRODUCED to EVIDENCED_BY and POSITIONED_AGAINST, plus a vague RELATED_TO fallback.
• Every link stores its evidence: how many chunks observed it, up to five evidence phrases, the source chunk ids, first and last seen dates, and the modalities it was stated in.

Claims are nodes, with two clocks and a modality

The most important modelling decision was to make claims nodes rather than properties. A claim such as "paying customers: 100+" is a Claim node with the attribute, the value, the exact quote, and three time-related fields:

(:Chunk)-[:SUPPORTS]->(:Claim {attribute, value, quote, modality,
                               valid_from, valid_to, note_date, is_current})-[:ABOUT]->(:Entity)
(:Claim)-[:SUPERSEDED_BY]->(:Claim)      a later note gives a different value
(:Claim)-[:CONFLICTS_WITH]-(:Claim)      two notes of the same date disagree

This is a bitemporal model: valid_from is when the fact became true in the world, note_date is when it was written down. Outdated facts are never deleted, only marked superseded, so "what did the notes say in February?" still works.

Each claim also carries a modality: fact, reported, plan, opinion, hypothetical or ai_suggestion. That one field is what lets an answer say "this was planned" or "ChatGPT suggested this" instead of reporting it as done. The extraction prompt also forbids conversation mechanics ("Tony asked…", "ChatGPT generated…") as relationships, which cut that noise by 84% between the first and second prompt versions.

Hubs are labelled so traversal can avoid them

A few entities are mentioned everywhere: the company itself (2,866 chunks) and the note author. Any traversal that passes through them reaches half the graph. Entities mentioned in more than 300 chunks get a :HubEntity label, and the search never passes through one. A hub can be where a search starts, never the middle of a path.

Hover or tap a node to see what it links to. Drag nodes to rearrange.

• Hub (300+ mentions)
• Company, product or technology
• Method or framework
• Client, backer or press

Extraction once, inference locally

The split between paid and local work was deliberate.

Step	Where it runs	How often	Why there
Entity, relationship and claim extraction	Claude Haiku 5.5 (API)	Once per chunk; cached by a hash of model, prompt version and text	Needs real language understanding; at $0.10 / $0.50 per million tokens a full folder costs a few dollars
Chunk and question embeddings	bge-m3 in Ollama, on the laptop	Chunks once; one question embedding per query	Private, free, fast enough
Graph traversal, keyword search, ranking	Neo4j, on the laptop	Every query	Deterministic, free, inspectable
Relevance trimming of graph context	bge-m3, on the laptop	Every query (~5 s for ~100 lines)	Free, no data leaves the machine

Extraction used structured outputs (a JSON schema enforced by the API) with low effort, and up to 64 parallel requests, well under the key's 10-million-input-tokens-per-minute limit. Prompt caching was evaluated and switched off: the system prompt was below the minimum cacheable size and every passage was unique, so caching added write cost without reads. A query costs nothing; the graph's running cost is the one-time extraction.

Schema evolution without re-extracting

The first full extraction filed 35% of links under RELATED_TO. Grouping those links by their evidence phrases showed recurring real relationships: positioned against, example of, party to an agreement, covers, affects, promotes. Re-extracting every chunk with six new types would have cost about $3.60. Instead, Haiku re-typed only the vague links in batches of 40, reading just the two endpoints and the evidence phrase. That cost $0.11 and brought RELATED_TO down to 16%.

Entity resolution: rules first, then two LLM passes

The extractor produced 15,345 entity mentions. Rule-based resolution (case, punctuation, .ai, company suffixes, one shared namespace for organisations, products, projects and concepts) brought that to 5,882. Two failure modes appeared on the way:

• Alias chaining. One chunk calling A an alias of B, and another calling B an alias of C, merged unrelated entities: the company node briefly absorbed 466 aliases. The fix was that an alias only merges when nobody uses it as an entity name in its own right, and only when it points to a single target.
• Near-duplicates. "MindReader AI" and "Mindreader" survive rules. Local bge-m3 similarity plus name containment proposed 1,161 candidate pairs; Haiku accepted 175. A second, stricter pass with explicit rejection rules confirmed 158, because the first pass alone had merged a company with its product and even two different tech CEOs. Five more were vetoed by hand in a tracked overrides file.

A separate audit checked every link seen only once against its own evidence phrase and dropped 613 of 5,394 that the phrase did not support ("discussed alongside" is not "affects"). The loaded graph: 5,673 entities, 5,304 typed relationships and 8,005 claims, with 112 supersessions and 27 conflicts.

The structure: how the system runs

Three skills and a read-only MCP server

The pipeline is packaged as three agent skills, so Claude Code and Codex follow the same rules:

• /neo4j-obsidian-kb-admin refreshes the knowledge base: it checks what changed in the vault, re-chunks only edited notes, re-extracts only new chunks after a cost check (anything over $0.50 needs approval), proposes new relationship types from the vague links, and rebuilds the entity layer in about ten seconds.
• /obsidian-hybrid-retrieval handles everyday lookups: keyword (sparse) and embedding (dense) search fused by rank, then a reranker, then a check against the source notes.
• /obsidian-graphrag-neo4j-search handles questions where relationships and multi-step hops matter (who introduced whom, what a decision led to, how a figure changed over time), with dated citations to note paths.

A read-only Neo4j MCP server lets the agent run ad-hoc Cypher for counts and filters, capped at 8,000 tokens and 30 seconds per response so a careless query cannot flood the conversation.

The search rules live in code, not in the prompt

Every limit is enforced by the search script, so an agent cannot talk itself past it:

question ─► BM25 + bge-m3 over chunks, fused by rank (k = 60) ─► top 20 notes
          ─► entry entities: named in the question + 8 seeds from those notes
          ─► 1 hop (2 at most), never through a hub, 25 links per named entity
          ─► dated claims matching the question, with their superseded neighbours
          ─► lines scored against the question with bge-m3; anything 0.15 below the best is trimmed
          ─► claim and link lines (40% of the budget) + ranked chunks (60%)

Questions that name a relationship ("who introduced…", "which clients…") are mapped to the relationship types they imply, and those links are fetched first and never trimmed. Older claims are returned and marked superseded, never hidden.

The evidence: the eval and its results

The eval set

The retrieval eval has 47 questions in nine categories, each written to stress a different property: single facts (8, the control group), multi-hop (8), entity roll-ups (7), temporal (6), modality traps (5), broad themes (5), paraphrase and alias (4), disambiguation (1) and unanswerable (3). They were drafted from the notes, not from the graph (45 of 47), then reviewed and approved, and split into a dev set (17) and a test set (30).

Each question has gold notes graded 2 (essential) or 1 (supporting, or a deliberate distractor such as an outdated figure), and key facts with an exact quote from a gold note. Generation quality is deliberately out of scope: this set measures retrieval only, and answer quality belongs in a separate eval.

The scoring

Metric	What it answers
recall@10, recall@20, all_gold@20, MRR, nDCG@10	Did the right notes rank, and how early?
fact_recall@8k	Did each key fact's exact quote reach the first 8,000 tokens the answerer would read?
note_recall@8k, precision@8k	How many gold notes made the budget, and how much of the budget went on them?
current_first	On temporal and modality questions, did the current note rank above the outdated distractor?
Sign test	Over per-question wins and losses, is the gap more than chance?

Three fairness fixes were needed before the numbers meant anything. The hybrid search printed only a 420-character excerpt per note, which would have penalised it on text-based metrics, so the eval looks up the full chunk it chose. The graph returns chunks rather than notes, so it was asked for 60 chunks to compete at the same 20-note depth. And the 8,000-token budget uses the same 40/60 split between graph lines and chunks that the search script uses.

Round 1: graph search on its own

Question type	n	Facts reaching the answerer (8k), hybrid / graph	Gold notes in top 10, hybrid / graph
Single fact	8	0.88 / 0.88	1.00 / 0.88
Multi-hop	8	0.26 / 0.22	0.39 / 0.30
Entity roll-up	7	0.55 / 0.35	0.29 / 0.24
Temporal	6	0.28 / 0.14	0.40 / 0.19
Modality trap	5	0.40 / 0.40	0.60 / 0.50
Broad theme	5	0.27 / 0.23	0.43 / 0.22
All 44 scored	44	0.47 / 0.37	0.53 / 0.39

Hybrid search won on every type, including the types where the graph was expected to win. On gold notes in the top 10 it won 15 questions, lost 5 and tied 24 (sign test p = 0.04).

The failure analysis showed three causes, none of them the graph's data. The graph's chunk ranking had no keyword search, so exact-term matches were lost. Chunks reached through claims and links were fused with the same weight as direct matches, pushing gold notes from ranks 1–3 down to 12–17. And on temporal questions, deliberately fetching each claim's earlier versions promoted exactly the outdated notes the gold labels marked as distractors (current note first: 0 of 6, against 0.40 for hybrid).

Round 2: the graph layered on hybrid search

The second design keeps hybrid search's ranked chunks untouched and adds the graph's claim and link lines inside the same 8,000-token budget, so the question becomes: does the graph add anything on top?

Question type	Facts reaching the answerer (8k), hybrid / hybrid + graph	Gold notes within budget, hybrid / hybrid + graph
Temporal	0.28 / 0.36	0.40 / 0.53
Modality trap	0.40 / 0.40	0.60 / 0.47
Broad theme	0.27 / 0.30	0.38 / 0.43
All 44 scored	0.47 / 0.48	0.52 / 0.52

Overall it tied: 3 questions better, 3 worse, 38 the same. The dated claim lines brought current notes into the budget on temporal questions. On modality traps, the graph lines displaced chunks that held the gold note. Everywhere else they repeated what the chunks already said. With five to eight questions per type, no single category gap is significant; the temporal gain is a lead to test, not a proven result.

Round 3: the same questions on a graph seven times larger

After twelve more folders went in (31,047 chunks instead of 4,779), the 47 Mindreader questions were run again, this time with latency recorded per question and a third variant that switches off the relevance trim. All three share the same hybrid base, so the comparison stays like for like.

Over the 44 scored questions	Hybrid	Hybrid + graph	Hybrid + graph, no trim	Hybrid + graph, after the lookup fix
Key facts within 8,000 tokens	0.51	0.50	0.40	0.49
Gold notes within 8,000 tokens	0.54	0.56	0.50	0.56
Gold notes in the top 10	0.55	0.55	0.55	0.55
Questions better / worse than hybrid on key facts	–	1 / 5 (p = 0.22)	1 / 12 (p = 0.003)	1 / 6 (p = 0.13)
Median latency	0.8 s	9.2 s	0.5 s	4.2 s
95th-percentile latency	0.9 s	13.2 s	0.6 s	6.0 s

Three results held from Round 2, and one is new:

• Time-based questions still gain. Gold notes within budget rose from 0.44 to 0.53 on temporal questions, and multi-hop questions now gain too (0.39 to 0.47, key facts 0.34 to 0.39).
• Plans-that-never-happened questions still lose (0.70 to 0.63), and so do entity roll-ups and broad themes on key facts (0.55 to 0.48 and 0.30 to 0.23): the graph lines take budget from chunks that already held the answer.
• Overall it is still a tie.
• The trim is what keeps it a tie, and it is most of the cost. The graph lookup itself takes about half a second. Embedding every candidate claim and link line with the local model to cut the irrelevant ones takes the other eight, and the larger graph produces more candidate lines (median latency rose from 7.3 s to 9.2 s). Without the trim, the graph's lines flood the budget and key facts fall from 0.51 to 0.40, significantly worse than hybrid alone.

A routine refresh of the knowledge base afterwards found why the trim had so much to do. The entity lookup matched question words against every alias the extractor had recorded, and some of those aliases were ordinary words: "his" was an alias on a dozen people, "framework" and "map" on unrelated entities. A question about "his role" started its traversal from people it never mentioned. Ignoring pronouns and requiring one-word aliases to match with their capitals ("HIS" the framework, not "his") cut the median latency from 9.2 to 4.2 seconds and the 95th percentile from 13.2 to 6.0, with retrieval unchanged (key facts 0.49, gold notes 0.56).

The baseline moved between rounds as well (key facts 0.47 to 0.51 for hybrid alone) without any Mindreader note changing. The cause is an edge case worth knowing, described below.

What the evals did not measure

The Microsoft Research result is about global questions, summaries over a whole corpus. Most of these 47 questions are local: a fact, a timeline, a set of related notes. Only the five broad-theme questions resemble the paper's setting, and there the layered graph was slightly ahead (0.30 vs 0.27 of key facts), too few questions to call. So this eval does not contradict the paper; it shows the claim does not carry over to everyday lookups on its own.

The key-fact metric checks for an exact quote, and the graph's claim lines carry the extractor's quotes, which are sometimes shorter than the gold quote, a bias against the graph. More importantly, the graph's main extras are labels a retrieval metric cannot see: "superseded on 2026-02-01 by…", "this was a plan", "ChatGPT suggested this". Whether those stop a model from reporting a stale or planned-only figure as current is a generation question, and it is where the next eval goes.

Decisions and trade-offs

Decision	Chosen	Rejected	Why
Where the graph lives	Neo4j Desktop, local (1 GB heap)	Neo4j Aura (cloud)	Private notes stay on the machine; the graph fits comfortably in a 1 GB heap
Embeddings	bge-m3 locally (1,024 dims, multilingual)	Hosted embedding API; keeping 384-dim bge-micro	Privacy, zero per-query cost, Chinese notes
Extractor	Claude Haiku 5.5, structured outputs	Larger models	A few dollars per folder; schema-checked JSON; quality checked on samples first
Facts	Claims as nodes with modality and two dates	Properties on entities	Contradictions and plans become queryable instead of overwritten
New relationship types	Re-type vague links ($0.11)	Re-extract the folder (~$3.60)	Same result for 3% of the cost
Merging entities	Rules, then two LLM passes, then hand overrides	One LLM pass	One pass merged unrelated entities; precision matters more than recall for identity
Search limits	Enforced in the search script	Written as prompt instructions	An agent can ignore a prompt; it cannot ignore code
Graph as a product	An add-on layer on hybrid search	A replacement for it	Round 1: alone it lost; Round 2: on top it tied and helped time-based questions
Relevance trim	Local bge-m3 scoring of every graph line (~4 s per query at 13 folders)	No trim (~0.5 s)	Without it, key facts fell from 0.51 to 0.40 (p = 0.003); the trim is what makes the graph safe to add, and it is acceptable for research questions, not for autocomplete
Speed budget	Graph layer only for questions that need it	Graph on every query	Hybrid answers in 0.8 s and ties overall; the graph's ~4 s pays off only on time-based and multi-hop questions

Some problems only surfaced because the results were checked by hand: timelines first sorted by valid_from instead of the note date, which made a 2025 note saying "launched in 2022" look older than a 2023 note; search results changed between runs because Python's randomised set ordering broke ties between seed entities; and the first eval run would have understated hybrid search because of its excerpt length.

When the graph is worth the wait

The eval turns into a routing rule. Hybrid search is the default: it answers in under a second and ties the graph overall. The graph layer earns its extra three to five seconds only when the answer has to be assembled across dates or entities, or when the answer depends on whether a statement is still true or was ever more than a plan.

Question shape	Example	Use	Evidence from the eval
A single fact	"What did the pilot cost?"	Hybrid	Same score (0.88 of key facts), about 5× faster
Something planned that may not have happened	"Did the dating MVP launch?"	Hybrid	Gold notes 0.70 vs 0.63: graph lines displaced the chunk with the answer
A roll-up or broad theme	"Everything about one client", "why the pivot"	Hybrid	Key facts 0.55 vs 0.48 and 0.30 vs 0.23
A paraphrase or alias	"Who handled the money side?"	Hybrid	Tie (0.50 each)
How something changed over time	"How did paying-customer numbers change?"	Graph	Gold notes 0.44 → 0.53; outdated values marked superseded, same-date disagreements flagged
A multi-step link	"Who introduced X, and what came of it?"	Graph	Gold notes 0.39 → 0.47, key facts 0.34 → 0.39
Why a decision was made and what it led to	"Why move away from SaaS, and what followed?"	Graph	Follows DECIDED, LED_TO and LEARNED_FROM paths that text search has to guess
Whether something was a fact, a plan or an AI suggestion	"Was 10 enterprise clients a result or a target?"	Graph	Every claim carries a modality; a retrieval score cannot see it, the answer needs it
Counts and filters	"How many client relationships started in 2026?"	Graph (Cypher)	A query over typed links, not a search
How two things connect	"How is the framework linked to the AIA rollout?"	Graph (path)	Shortest path of up to four hops, never through a hub

When in doubt, start with hybrid search, and switch to the graph when it returns passages that disagree or might be out of date.

Edge cases that only appeared at scale

Scaling from one folder to thirteen surfaced a second set of problems, none of which the one-folder build had hit.

• Filtered vector search changes when the index grows. The vector and keyword indexes rank all 31,047 chunks, and the folder filter applies afterwards to an eight-fold over-fetch. Adding twelve folders changed every one of the 47 hybrid rankings for Mindreader questions, although no Mindreader note changed, because both the candidate pool and the keyword weights (which depend on how rare a word is across the whole index) shifted. A production system that serves one team's documents should filter inside the index, or keep an index per permission scope.
• Short aliases collide. The extractor recorded "HIS" (the name of a sales framework) as an alias of a dozen entities, because it is also an English pronoun. Matching by alias pulled in 12 unrelated people, both in the public graph export above and in live search, until lookups ignored pronouns and required one-word aliases to match with their capitals. In search, that fix alone halved latency.
• Merge quality drops with breadth. Across the larger graph, the two LLM passes confirmed 1,145 of 10,900 candidate merges. A hand check of 30 confirmed merges found about four wrong, all the same pattern: a generic name absorbed into a specific one ("Distribution" into "Distribution Docs README"). The one-folder graph had far fewer generic names to confuse.
• Long runs need to resume. The extraction hit the API account's monthly spend limit halfway through a folder, and the local embedding server dropped connections under sustained load. Because every extraction is cached per chunk and every embedding batch is saved before the next, both resumed where they stopped, with nothing paid twice.
• Not every saved conversation belongs in a graph. Of 1,251 saved ChatGPT conversations, a $0.11 Haiku triage kept 153 with business decisions. Long chats were re-read at four points, because a chat's title only reflects its first message; that recovered 23 the opening alone had missed.
• Namesakes merge into the author. Any person the notes call just "Tony" was resolved to the vault's author, which held for the one company folder. Once customer reviews from lead research went in, a renovation firm's branch manager and senior boss, both called Tony, became the author, along with their job titles. Mentions of a bare "Tony" now stay separate when the extractor describes someone else's role and gives another full name, and the relationships and claims from that passage move with them.

Entity resolution is maintenance, not a one-off

Every one of these problems came from adding nodes, not from changing code. Rules that were correct for 5,673 entities from one company's notes failed at 40,903 entities from thirteen folders, because new folders bring new namesakes, new generic words and new alias collisions. So a large addition is followed by the same refresh every time, as a step in the knowledge-base admin skill:

1. Re-run the clean-up on the new candidates

   Near-duplicate detection, the two LLM merge passes and the audit of single-observation links run again. Decisions are cached, so only new candidate pairs are judged (about $0.86 for the twelve-folder addition).

2. Rebuild the entity layer

   Resolution, merges and claim timelines are rebuilt from scratch in about 20 seconds, locally and for free, so a rule fix applies to every folder at once.

3. Spot-check what the new folders changed

   Read the largest entities' aliases for names that belong to someone else, sample 30 confirmed merges, and run a handful of questions to see which entities they start from. Fixes go into code or a tracked overrides file, never into the database by hand.

4. Re-run the eval

   The retrieval eval runs again for free. If the scores or latency moved, the change is explained before the graph is used. The first time this routine ran, its spot-check found the pronoun aliases, and the fix halved graph query time (median 9.2 to 4.2 seconds, 95th percentile 13.2 to 6.0) with retrieval unchanged.

From one vault to a company

This vault has one user and no access rules. In client deployments, the most common challenges are rarely retrieval quality alone. Three come up again and again:

1. Permissions. Not every employee may see every document, and access changes after indexing: someone moves team, a folder is locked down. Access lists copied at index time go stale, so right before the answer is generated and returned, the system calls the source's permissions API to confirm the user is still cleared for each document in the context, and drops any they are not. Confidential information never reaches the answer for a user who cannot open the original.
2. Updated, conflicting and new documents. A revised price list or procedure must replace the old one in answers, not sit beside it. The claim model above was built for this: a later value supersedes the earlier one, same-date disagreements are flagged as conflicts, and incremental refreshes bring new documents in without a full rebuild.
3. The interface. Where employees reach the chat decides whether they use it, and it carries their identity through to the permissions check.

On the model side, the competency-question approach carries over: the graph's relationship types should match what users actually ask. If staff at a repair company keep asking which tools suit which appliance brands, the schema should enforce a direct relationship between tool and brand entities, rather than leaving the extractor to file it under a vague RELATED_TO.

What happens next

The graph now covers 13 folders: 31,047 chunks, 40,903 entities, 34,459 typed relationships and 52,866 claims, with 944 supersessions and 204 conflicts. Extending it from one folder cost about $21 of one-time extraction and clean-up, against an estimate of $20, for about $28 in total. The search now runs across every folder by default.

Three things come next. The eval set needs questions about the other projects, so the comparison is no longer one folder's. The merge confirmation needs a stricter rule for one-word generic names. And a separate generation eval needs to answer the open question this one could not: whether dated, modality-tagged claims make the final answer more truthful, even when they do not make retrieval better.