Your Agent Forgets Everything: File, Vector, and MCP Memory Compared

Last month I watched Claude Code rediscover, for the fourth time, that my project uses pnpm and not npm. It ran npm test, watched it fail, apologized, and tried again with pnpm. I had explained this in three previous sessions. None of them survived. The agent doesn't carry anything between runs unless I make it.

That's the whole problem. A coding agent is stateless between runs. Every session starts with an empty context window. Whatever it figured out yesterday — your build commands, why one test is flaky, the fact that the staging database resets at midnight — is gone. You re-explain, it re-derives, you both waste tokens.

So you give it memory. And the moment you go looking, you find three different things all called "memory," and most of the writing about them is either a vendor pitch or a tutorial that won't tell you when the approach falls apart.

The short version: for a solo builder, flat files are the default and you should exhaust them before adding anything. Vector stores earn their keep when you're searching a large corpus you can't fit in context. MCP memory servers are the most flexible and the easiest to quietly poison. Most people reach for the fancy one first. That's backwards.

What "memory" actually means here

The three options aren't competing implementations of the same thing. They solve different problems and fail differently.

Flat files are markdown or text the agent reads at startup — CLAUDE.md, AGENTS.md, a scratch NOTES.md. You or the agent write them, and they load into context every session. No search, no embeddings — just "read this file first."

Vector stores turn text into embeddings and let the agent retrieve by meaning instead of by filename. You write something once, the store finds the relevant chunk later even if the words don't match.

MCP memory servers are separate processes the agent talks to over the Model Context Protocol, which exposes tools, resources, and prompts to any compatible host. A memory server adds tools like "store this fact" and "search my memory" the agent calls during a task.

I've shipped with all three. Here's the honest read on each.

Files: boring, cheap, and the right default

File memory is the one you already have. Claude Code reads CLAUDE.md at the start of every session, loaded in full as a user message after the system prompt — not as enforced configuration, which matters, because the agent can still ignore it. Files load from a hierarchy: a managed org-wide file, your personal ~/.claude/CLAUDE.md, the project ./CLAUDE.md, and a gitignored CLAUDE.local.md, ordered broadest to most specific so the closest instruction is read last.

There's also auto memory now: Claude writes its own notes to ~/.claude/projects/<project>/memory/MEMORY.md, and the first 200 lines (or 25KB) of that file load every session. It saves build commands and debugging insights it discovers, without you typing them. I leave it on; it's caught a few "oh right, the API tests need a local Redis" facts I'd have re-explained otherwise.

I won't rehash what to put in a CLAUDE.md — I wrote a whole post on that. The point here is architectural: files are good at exactly one thing, and it's the thing most agents need — a small, curated set of facts that should be true in every session.

What files are good at:

• Zero infrastructure. It's a text file in your repo. Version-controlled, diffable, reviewable in a PR.
• You can read your own memory. When the agent does something dumb, you open the file and see exactly what it was told. Try doing that with an embedding.
• It survives. Project-root CLAUDE.md even gets re-injected after a /compact, so a long session doesn't lose it.

What files are bad at:

• They cost tokens every single request. Everything in CLAUDE.md is in your context window for the whole session. The docs say target under 200 lines, and they're right — past that, adherence drops and you're paying for instructions the agent skims.
• No retrieval. You can't keep a thousand facts in a file the agent reads every time. It's a working set, not a database.
• Staleness is on you. If your build command changes and you forget to update the file, the agent confidently runs the old one. The file doesn't know it's wrong.

For most solo projects, files are enough. I've shipped products where CLAUDE.md plus auto memory was the entire memory system and I never wanted more.

Vector stores: good recall, no judgment

When the thing you want to remember is too big to fit in a file the agent reads every time — a year of architecture decisions, a large internal wiki, every past bug and its fix — you move to retrieval. A vector database stores high-dimensional embeddings of your text and finds the nearest matches to a query by meaning, typically in tens of milliseconds.

You don't need a dedicated vector DB to start. If you already run Postgres, pgvector adds vector storage and search to the database you already have, which is usually the right first move for a solo builder. The 2026 default is hybrid search — keyword plus vector — because pure semantic search misses exact-match cases like a specific function name or error code.

What vector stores are good at:

• Recall at scale. This is the only one of the three that handles "I have ten thousand notes, find the relevant three." Files can't and MCP graph servers struggle.
• Meaning over wording. You wrote "the checkout flow double-charges on retry," you later ask "why does payment fire twice," and it finds the note. Keyword search wouldn't.

What vector stores are bad at:

• Retrieval quality is a pipeline, not a setting. Embedding model, chunking strategy, index type, and update cadence all interact. Get chunking wrong and you retrieve half a thought. This is real work, and it's work most solo builders underestimate.
• No notion of true. A vector store returns what's similar, not what's correct or current. If you stored a fact in March and the truth changed in May, both versions sit in the index and the March one can win on similarity.
• It's a poisoning surface. Anything that feeds the store — a web page the agent indexed, a doc someone dropped in a shared folder — can plant text that gets retrieved later as authoritative context. More on that below.

I reach for a vector store when the corpus is genuinely too large for context and I need recall. For "remember my five project conventions," it's wildly over-engineered.

MCP memory servers: flexible, structured, and a bigger attack surface

The third option is a memory server you connect over MCP. The reference one is Anthropic's knowledge-graph memory server. It models memory as a graph: entities (named nodes with a type), relations (directed, active-voice connections between them), and observations (atomic facts attached to an entity). It persists to a local memory.jsonl file and exposes tools like create_entities, add_observations, search_nodes, and read_graph.

The pitch is that structure beats a flat blob. "User prefers pnpm" becomes an observation on a User entity, related to a Project entity, and the agent traverses that instead of grepping prose. In practice it's between files and vectors: more queryable than a text file, more interpretable than an embedding.

Worth knowing: Anthropic's own memory tool is a different mechanism — a client-side tool where the model issues view, create, str_replace, delete commands against a /memories directory and you implement where the bytes live. Its system prompt tells the model "ALWAYS VIEW YOUR MEMORY DIRECTORY BEFORE DOING ANYTHING ELSE... your context window might be reset at any moment." That tells you something: even Anthropic's "memory tool" is files underneath.

What MCP memory servers are good at:

• Structured recall the agent can reason over. Graph traversal answers "what do I know about this project" more precisely than a similarity search.
• Portability. MCP is a standard, so the same server works across Claude Code, Cursor, and any compatible host. One memory, many tools.
• Explicit writes. The agent decides what's worth storing as a discrete observation, which keeps memory cleaner than dumping whole transcripts into a vector index.

What MCP memory servers are bad at:

• Setup and a running process. It's another moving part to install, configure, and keep alive. For one project, that's a lot of ceremony.
• The graph drifts. Without discipline, you get duplicate entities ("Project", "the project", "my app") and contradictory observations. The structure that was supposed to help now needs gardening.
• It's the easiest to poison. Persistent, writable, retrieved-as-authoritative — that's the exact profile attackers target.

The two failure modes nobody puts on the landing page

Every "give your agent memory" post sells the upside. Here are the two that bit me.

Memory poisoning. This is the one to take seriously. Unlike a prompt injection, which dies when the session closes, memory poisoning is a persistence attack — a malicious instruction planted in your store today executes weeks later, triggered by an unrelated task. The attack and its effect are temporally decoupled. Any pipeline that writes to memory is an entry point: a web page the agent indexed, a dependency's README it summarized, a shared doc. Vector stores and MCP servers are both exposed because both ingest content automatically and serve it back as trusted context. Files are the safe one, precisely because a human writes them and can read them in a diff.

Stale facts overriding reality. Less dramatic, more common. Your memory says the pricing tier is $20. It went to $25 last Tuesday. The agent pulls the cached fact with no time-to-live check and serves the old number as authoritative. The agent has no way to know the page changed an hour ago. Every memory system has this problem, and the more "automatic" the system, the worse it is, because nobody's watching what got written. I've shipped a wrong config value this exact way. The memory was confident. The memory was wrong.

The uncomfortable truth: the more you automate memory, the less you notice when it's lying to you. Files are annoying to maintain by hand, and that annoyance is also the audit.

What I'd actually run as a solo builder

Start with files. CLAUDE.md for the facts you keep re-explaining, auto memory on so the agent captures the rest. This covers most of what a solo project needs, costs nothing to stand up, and you can read every byte of it. Don't add a second system until files visibly fail you — and "fail" means a specific symptom: the agent can't find a decision you know you wrote down, or your CLAUDE.md has blown past 200 lines of stuff that only matters sometimes.

When that happens, ask one question: is the problem recall or structure? If you have a large corpus and the agent can't find the right piece, that's recall — add a vector store, start with pgvector if you're already on Postgres, and use hybrid search. If you want the agent to reason over relationships across sessions and you're working across multiple tools, that's structure — an MCP memory server earns its setup cost.

Whatever you add, treat memory as a thing that can be wrong. Put a date on facts that expire. Periodically read what got written — the Claude Code /memory command shows you exactly that for files, and it's plain markdown you can edit or delete. Never let an automated pipeline write to memory from a source you wouldn't paste into your terminal yourself.

The agent forgetting everything is annoying. The agent confidently remembering something false is worse. Pick the simplest memory that solves your actual problem, and keep it small enough that you can still tell when it's wrong.