How it works

OpenContext works in five steps: a document, link, or synced item comes in; a local privacy preflight can hold it before model processing; model calls distill it into entities that a deterministic writer renders as markdown files; the vault records that write as one git commit; and search, chat, and insights read the result. The pipeline runs on your machine; enabled AI steps send content to your configured model provider. Personal-content classification can also use a model before distillation. (Acronyms on this page: MCP is the Model Context Protocol, the standard chat clients use to call tools; an API key is the credential that identifies your account with a model provider; an OAuth grant is the sign-in grant a service such as Google issues when you connect it.)

1. Intake

Drop files onto the Intake page: PDF, Word, Excel, PowerPoint, OpenDocument, EPUB e-books, rich text (.rtf), web pages saved as HTML, email (.eml), markdown, plain text, comma-separated tables (.csv), and source code, up to 10 files at a time and 25 MB each. The type is sniffed from the bytes, so a renamed file still routes correctly. The original is stored under _sources/ and deduplicated by its content hash: an identical re-upload is skipped, and the same name with new content is treated as a new version and merged, not duplicated.

Links and pasted text feed the vault the same way. Paste a link and the server fetches the page once, from your machine, through a guard that refuses private and internal addresses: readable article text is extracted, PDFs are read directly, and x.com post links are read through X's public endpoints, no account needed. A site that answers with a bot check or a paywall is reported honestly, with the fallback: copy the text from your own browser and paste it in.

Connected sources queue items through the same path, each with your own read-only credentials and only when you press sync: Google Drive folders (Docs export as markdown, Sheets as xlsx, Slides as pptx), Gmail threads after a metadata-only scan and an interview that becomes persistent include and exclude rules, GitHub and GitLab repositories through a personal access token, new X bookmarks through your own X developer app, and ChatGPT, Claude, and Gemini exports split per conversation locally with the raw export discarded in the same request. Before a large Drive or Gmail batch runs you see how many documents it would queue, and a batch of more than 100 needs explicit confirmation, because every document is a model run.

2. Privacy preflight

Document preflight checks the prepared text locally. With the default rules, detected high-severity identifiers such as a Social Security number, bank account number, or secret key hold the document before a model reads it. In Privacy, you can review it, sanitize a copy, release it as-is, or discard it. Sanitization can itself use AI to identify personal passages, so it is not always a wholly local operation. PDFs are extracted locally; image-only PDFs require local OCR first, and extraction warnings identify missing text coverage.

Detection is checksum-backed where it can be (the Luhn check for card numbers, the mod-97 check for international bank account numbers, the routing-number checksum for United States bank routing numbers, the Social Security Administration's structure rules), and findings never carry the raw value, only a mask. Names, email addresses, phone numbers, and company financials are deliberately outside the deterministic detector's scope, because a knowledge graph is made of those. Recognized street addresses are flagged by default, not automatically removed. A second check catches personal content in prose, such as a medication, a diagnosis, or a friend's secret: a free vocabulary scan runs on everything and a cheap model reads only what it flags. Files over a size threshold are held with a cost estimate before anything is spent. The privacy page covers the whole layer.

3. Distillation

Distillation uses structured model calls over the prepared document text. Larger documents are split into bounded sections. The text and an existing-entity catalog go to the model; a deterministic writer turns the structured output into entity files and maintains the README indexes. Unusable output stops the structured path rather than silently buying an agent fallback. Image-only PDFs need local OCR before distillation. An explicitly selected agent mode uses file tools scoped to the vault (read, list, search, write, edit, no shell, path-guarded) under the same vault specification.

Every write on either path passes the same three checks: a path sandbox that keeps writes inside the vault, a filename convention (one kind of thing, one shape of name: atlas-briefing.md for a project, 2026-05-04-q3-kickoff.md for a meeting), and a privacy screen that removes detected values configured for redaction before they land in an entity file. This write-time check cannot undo data already sent to a provider. The original in _sources/ is never altered.

Every step streams to the interface as it happens and is persisted to _logs/<jobId>.jsonl. Every agent run can be stopped from where you started it, or from the agent dock that lists everything the model is doing at once. A queued job dies before a token is spent; a running one has its agent aborted, with whatever it already wrote kept, labelled (stopped) in history, and undoable from there.

4. The vault is a git repository

The vault is a normal folder of markdown, and it is its own git repository. Every ingest, sync, edit, dedupe, and index repair lands as one commit, the History pane shows that log, and any single operation can be rolled back. Inside it:

An entity file has frontmatter (type, name, aliases, sources), facts, and relationships written as relative markdown links to other entity files, which is what makes the folder a graph. A source note per document records provenance. The root README.md is audited mechanically, not trusted: an index audit checks that every populated domain is linked from it and every entity file is indexed, and a one-click repair lands as one revertible commit with no model involved.

Because it is git, the vault travels. Push it to a GitHub or GitLab remote of your own from Settings, and the push carries the entities, every index, and a vault-root AGENTS.md primer that tells any agent how to read it; _state/, _logs/, _sources/, _extracted/, and _repos/ are gitignored and never leave. Or download the whole thing as a zip: every knowledge file, the _sources/ originals, and the full .git history, with _state/, _logs/, _extracted/, and _repos/ excluded. Restoring is a clone plus a pointer.

5. Search, chat, insights

Search is deterministic and free: focused with Cmd/Ctrl+K in the app, it ranks name and alias matches above file-name and body terms and answers with a list of entity files. The command-line tool and the Model Context Protocol (MCP) server hand the same ranking engine to external agents.

Chat runs as persisted sessions. An agent answers from the vault with citations as links, may edit the vault when you clearly ask it to (every write goes through the same guards as ingestion and lands as a chat-edit commit), and every answer is audited: its load-bearing claims are checked mechanically against the files it cites, with a per-answer receipt of tokens and estimated cost. A Checkpoint distills what a conversation settled into a conversations/ note. Conversations can be anchored to one entity, a whole collection, or a repository's project graph.

Insights drops a fresh-context agent into the vault on demand, over three lenses (Connections, Ideas, Path) and deterministic graph signals it must verify against real files. Results are staged, not knowledge: a card enters the graph only when you add it. Agents and Skills does the same for recurring work shapes and suggests reusable agents, each of which can be saved to the vault, downloaded as a Claude Code subagent file or a SKILL.md, run in chat, or invoked by an external agent through the MCP server. Duplicates groups files that look like the same thing and asks you to sign off on a source of truth, stamped into the file itself. Evidence is one chronological ledger of measured answers, builds, and dedupe skips, every figure measured or marked as an estimate.

Repositories are projects

Point a local checkout or a git URL at OpenContext and a multi-pass agent distills what the repository is, its architecture, tech stack, recorded decisions, and contributors into a self-contained graph at _projects/<slug>/, kept apart from your personal graph. The personal graph's repositories/<slug>.md note carries the pointers (local_path, remote_url, graph:) so an agent walking your context can find the repository's graph and walk into it, and an agent working in the project can reach back. Re-syncs are incremental: only commits since the last ingested one are distilled, and an identical HEAD is skipped with no model run. The for-agents page shows how an agent reads all of this.