Skip to content
Alexandre Mahe
All posts

An agent's context: client knowledge, memory and permissions

The agent explores the code on its own. Refining its context means bringing it what it can't find: knowledge of the client, a memory of its sessions and permissions around them, calibrated to confidentiality and compliance.

Category
Note
Published
Reading time
4 min read
Tags
#agents#context-engineering#mcp#rag#memory

TranslationTranslated by AI from the original in French.Read the original

The agent explores the codebase on its own: it picks the commands, knows where to look, reads the git history, runs the tests and, integrated into the editor, uses the same tools as the developer, up to driving a browser or taking screenshots with Playwright MCP. That part of the context is the job of the agent and the harness. What's left to refine is what it won't find in any file.

Prototyping a page, writing interface copy, generating a document in the company's colours: each of these tasks requires knowing its brand guidelines. Without them, the agent produces something correct, but anonymous. Knowing the client's interests, positioning and priorities also yields better-scoped tasks, with milestones that make sense to the client, not just to the code.

Training an open model on the company's data could give better results, but it's a project in its own right, rarely within a freelancer's reach. A freelancer stands out elsewhere: anyone can have a well-tooled agent; deep knowledge of the trade and the client, far fewer can.

This knowledge makes the difference, but what you give the agent, and how it accesses it, depend on confidentiality, compliance and permissions.

The ontology, behind authorised accessLink to this section

This body of knowledge (the client's business, brand, structure, governance, interests, positioning, vision) has a name: the ontology. It's often the most useful context you can give an agent, and the most sensitive.

Client's company Freelancer the chosen context authorised MCP traced RAG search and introspection Agent and subagents LLM provider Ontology Large corpus Code, tickets,session archive Skills and preferences

I keep the ontology separate, often in a private repository, and the agent accesses it through an MCP server, protected by the protocol's authorisation when it's remote, and by system permissions when it's local. The agent gets only the part of the ontology its task justifies, instead of the whole corpus pasted into its prompt, and the client's knowledge stays under the client's control.

RAG becomes useful in two cases: when the corpus is too large to browse, like years of documentation or meeting notes, and when every access an agent makes to a piece of data must be authorised and traced. Retrieval then goes through a service that checks permissions before answering.

Calibrate to sensitivityLink to this section

The brand's colours, typography and tone are often public: a short file in the repository is enough, and the agent reads it on every task. The client's interests, priorities and roadmap are confidential: they go through the authorised MCP, and only for the tasks that need them. Its users' personal data has no place in the ontology: if a task needs it, it's anonymised before it leaves the machine, or processed by a local model, as in the post on verification.

So the same knowledge takes three different paths. The choice doesn't depend on what would be most convenient for the agent, but on what the client has authorised.

What enters the context can also carry instructions. A document from the corpus, a response from an MCP server or an old session can contain a prompt injection; a source only enters the context if it's trusted, or after filtering by the proxy.

A memory for introspectionLink to this section

Going back over a previous session helps the agent recover a forgotten task or a security warning that was never followed up. That memory has to exist first, and not everything should be lumped together.

The session archive. Every session leaves a trace: the prompt, the response, the tokens consumed, the timestamp. I store it with the client's project, in git or in a git-ignored folder when it must not leave the machine: it contains the client's data, and leaves with the rest at the end of the engagement. When these files pile up, a PostgreSQL database takes over: the same content, but queryable, and protected by the database's access rights.

Context shared between agents. An agent and its subagents need a common base, which can live in that same database. Placed at the start of requests, it benefits from prompt caching, whose lifetime is measured in minutes.

Task tracking. Checkboxes in tickets and statuses that move: the agent knows where it stands, so does the developer, and the next session picks up without re-explaining.

The ontology, for its part, stays separate, with its restricted access: it's not working memory, it's reference knowledge.

An ontology in three filesLink to this section

Without an MCP server or RAG, a few short files are enough to start. The public folder lives in the repository; the confidential one, elsewhere, behind authorised access:

client/
  brand.md        colours, typography, tone
  business.md     what it does, what it stands for, what it refuses
  priorities.md   confidential: roadmap, interests, to serve over MCP

The confidential file goes through the reference MCP server, restricted to its folder, with its write tools denied in the agent's permissions:

claude mcp add ontology -- npx -y @modelcontextprotocol/server-filesystem ~/clients/acme/ontology
"deny": ["mcp__ontology__write_file", "mcp__ontology__edit_file", ...]

And a ticket the agent keeps up to date itself, so the next session picks up without re-explaining:

- [x] Parse the CSV export
- [ ] Add tests for invalid dates
- [ ] Validate the format with the client

These files stay short: the ETH Zurich study cited in the post on the harness showed that automatically generated context files slightly degrade results while increasing cost. The rest gets added when the project calls for it, starting with what the client has authorised.