Why Markdown Files Aren't Enough
for AI Agent Memory

The most common advice for giving AI agents memory is: "just use a markdown file." Create a LEARNING.md or MEMORY.md in your project root, let the agent append to it, and load it into context on startup.

This works. If you have 20 learnings on a single project, a markdown file is genuinely the right tool. It's simple, auditable, version-controlled, and any LLM can read it. I used this approach myself for months before building OMEGA.

But somewhere between 50 and 200 entries, things start breaking in ways that are hard to notice and harder to fix. This post describes the five failure modes I encountered, what a production memory pipeline actually looks like, and why the benchmark data backs this up.

A markdown file has exactly one retrieval mechanism: load the entire file into the context window. Every query gets every memory, every time.

At 20 entries this is fine. At 200, you're burning thousands of tokens on irrelevant context. At 500, you hit context limits. And the real problem isn't token cost: it's attention dilution. When an LLM sees 500 entries and needs one, the odds of it focusing on the right entry drop significantly.

You could add grep. But grep only finds exact keyword matches. "We decided to use PostgreSQL" won't match a query about "database choice." The vocabulary mismatch problem is well-studied in information retrieval, and it's why search engines moved beyond keyword matching decades ago.

A real memory system runs queries through vector similarity (semantic meaning), BM25 (term frequency), and reciprocal rank fusion to merge both signals. OMEGA's query pipeline adds intent classification, contextual re-ranking based on the file you're currently editing, and multi-hop graph traversal to surface connected facts across sessions. The result: 10 relevant memories out of 500, delivered in under 50ms.

Ask an agent to remember a lesson, and it will. Ask it again in a different session with slightly different wording, and it will store a second copy. Then a third. Then a fourth.

I've seen markdown memory files with the same architectural decision recorded six times in six paraphrases. Each entry is slightly different, so naive string comparison doesn't catch it. The file grows, retrieval degrades, and the agent starts citing three versions of the same fact in its responses.

OMEGA runs every incoming memory through three layers of deduplication before it touches the database. First, a canonical hash catches reformatted duplicates (same content, different whitespace or markdown formatting). Second, an exact content hash catches verbatim copies. Third, embedding-based cosine similarity catches semantic duplicates: "use PostgreSQL for the database" and "we chose Postgres as our DB" land on the same memory node.

Above the storage layer, the bridge runs Jaccard similarity with per-type thresholds. Decisions dedup at 0.80, lessons at 0.85, error patterns at 0.70 (with path and ID normalization so the same stack trace from different files still deduplicates). If a near-duplicate is found, the existing memory's access count increments and no new node is created.

When a memory is similar but not identical, and it contains genuinely new information, the evolution system merges the new insight into the existing memory. Zettelkasten-style: the knowledge base refines rather than repeats.

Markdown files are append-only in practice. Entries accumulate. Old decisions stick around long after they've been reversed. A note about a bug that was fixed six months ago sits right next to a note about a bug you introduced yesterday, with nothing to distinguish their relevance.

Time-decay is not a feature you can bolt onto a flat file. It requires tracking when each memory was created, when it was last accessed, how many times it's been retrieved, and what type of memory it is (a user preference should decay slower than a debugging note).

OMEGA assigns TTL categories by event type: error patterns expire after 30 days, session summaries after 90, user preferences never. The query pipeline applies a continuous decay factor that weighs recent memories higher, with the decay curve varying by type. Lesson nodes with high access counts resist decay; stale nodes that have never been retrieved once are candidates for pruning on the next consolidation pass.

In January you write: "We use REST for all API endpoints." In February you write: "We migrated the real-time endpoints to WebSockets." Both are true at the time of writing. Neither is deleted. When the agent reads both, it has no way to know which one is current.

This is the most insidious failure mode because it produces confident, wrong answers. The agent doesn't say "I found conflicting information." It picks whichever entry it attends to first and presents it as fact.

OMEGA runs contradiction detection on every store operation. After a new memory is persisted, the system finds the most similar existing memories via embedding search and checks for temporal supersession: if the new memory has the same event type, high cosine similarity, and a later timestamp, the older memory is automatically marked superseded. Superseded memories are filtered out of all future query results.

For cases that are not simple supersession (genuinely conflicting facts from different domains), the system runs heuristic contradiction detection and annotates both memories with a contradiction edge. The query pipeline then surfaces the contradiction explicitly, letting the agent reason about it rather than silently choosing one version.

The moment you have two agents running concurrently, or one agent working across two projects, a markdown file becomes a coordination bottleneck.

File locks are the obvious problem: two processes appending to the same file can corrupt it. But the deeper issue is isolation. An agent working on your frontend project does not need to see memories from your backend project. An agent reviewing code does not need to see memories from an agent running tests. Without scoping, every agent sees every memory from every context, and the noise overwhelms the signal.

OMEGA stores memories in a single SQLite database with WAL mode for safe concurrent reads and writes. Each memory is scoped by project path, session ID, entity ID, and agent type. Queries automatically filter to the caller's scope. Cross-session handoff works through session summaries and checkpoints: when an agent session ends, it writes a summary; when the next session starts, it reads the summary and continues where the previous one left off. No file locks. No merge conflicts. No cross-talk.

The gap between "append a line to a file" and "store a memory" is larger than it appears. Here is what happens in OMEGA when an agent calls omega_store:

Retrieval is even more involved. When an agent calls omega_query, the pipeline runs 14 phases:

None of this is theoretical. Every phase maps to a function in sqlite_store.py, all running locally on SQLite and sqlite-vec. No external APIs, no cloud calls. Total retrieval latency: under 50ms for a database with 500+ memories.

LongMemEval (Wang et al., ICLR 2025) is a 500-question benchmark that tests whether a memory system can actually remember, update, and reason about information across conversations. It covers six capabilities: single-session recall, preference application, multi-session reasoning, knowledge updates, temporal reasoning, and within-session updates.

Without any memory system, an LLM scores about 49.6%. A markdown file loaded into context would do better on the single-session questions, but it would fail on knowledge updates (no way to mark old information as superseded), temporal reasoning (no timestamps on individual entries), and multi-session queries (no way to search across sessions without loading everything).

OMEGA scores 95.4%, placing #1 on the global leaderboard. The pipeline described above is what makes those numbers possible. Each of the five failure modes maps directly to a LongMemEval category:

A flat file can't pass this benchmark. Not because it's a bad tool, but because the benchmark specifically tests the capabilities that flat files lack: semantic retrieval, temporal reasoning, knowledge updating, and cross-session synthesis.

If you're using Claude Code on a single project with a handful of things to remember, a LEARNING.md file is the right tool. It's zero-dependency, transparent, and it works. Use it.

When you start noticing duplicates, when old entries mislead your agent, when you need memories to flow between sessions or across projects: that's when the 11-step store pipeline and the 14-phase query pipeline start earning their complexity. Every step exists because a simpler version failed at scale.

OMEGA is open source, local-first, and installs in two commands. The markdown file got you here. When you outgrow it, we'll be here.

- Jason Sosa, builder of OMEGA