Files and folders vs. agents
The infrastructure debate over filesystems and vector databases is the same argument as context design, one layer down the stack.
The claim
There's a debate running through AI engineering circles that looks, on the surface, like an infrastructure argument: files and folders, or agents and frameworks; a filesystem, or a vector database; plain markdown a person can open and read, or embeddings and similarity search that nobody reads at all. It's being fought as a tooling question. It isn't one. It's the same argument about context, just happening one layer down the stack, in the plumbing instead of the prompt.
The pitch for complexity
For a couple of years, the assumed shape of a serious AI system was: chop documents into chunks, embed them, drop them in a vector database, and let the model pull back whatever's semantically closest to the question. That's a reasonable answer to a specific problem, a static pile of documents and a model with a limited context window. Retrieval-augmented generation earned its place solving that. But an agent doing real work doesn't ask one question and leave. It reads a file, notices a reference, follows it, reads the next file, runs something, checks the result, and adjusts. That's closer to how a person navigates a project, and it turns out plain files and sensible folder names hold up better under that kind of exploring than a database of disconnected chunks does.
The folder is the agent
Anthropic's own engineering team has made the filesystem case directly: agents that work in a filesystem behave the way people do, using external structure like folders and file names instead of trying to hold everything in memory at once. A file called test_utils.py means something different sitting in a tests folder than it does sitting in src/core_logic. Nobody wrote that meaning down anywhere. The location is the information. The sharpest version of this argument goes further still, framing an agent as not really the model at all: it's a project folder, a file that tells the AI how to work, a handful of skill definitions, and months of accumulated context, with a generic model plugged in on top. Swap the folder, and the same model becomes a different specialist. The intelligence people are impressed by isn't coming from the model. It's coming from what's been curated into the folder around it.
Why this is the same argument
Which means the "files vs. agents" debate isn't really about files. It's about whether the work of curation happened before the model was asked to do anything, or whether everyone's hoping a cleverer retrieval mechanism can substitute for that work never having been done. It can't. A vector database full of unfiltered chunks is context debt with an API in front of it. It looks like infrastructure. It's actually an unmade decision about what matters, deferred to query time and hoping the maths sorts it out.
Files make you show your work
A folder of files forces a human to make explicit, ahead of time, exactly the decisions context design is about everywhere else: what belongs here, what doesn't, what needs to sit next to what, what the name of this thing should signal to whoever encounters it next. A messy folder is visibly a messy folder. A vector database hides the same laziness behind a similarity score. Nothing forces a decision about what's relevant before the fact; the embedding always returns something plausible-looking, and a plausible-looking wrong answer is harder to catch than an obviously empty folder. A model asked to work from an incomplete picture doesn't pause and ask a clarifying question the way a new hire would. It fills the gap with something confident-sounding and moves on. A retrieval layer that quietly returns the wrong chunk does exactly the same thing, one level removed from where anyone's looking.
The honest caveat
The "just use files, vector databases are dead" take going around is overstated. At real scale, with genuinely fuzzy queries and concurrent access, structured retrieval still earns its place, and most serious production systems are landing on some hybrid of the two, not a clean win for either side. That infrastructure question isn't mine to referee. What's mine is the pattern underneath it: however it's stored, files or vectors, the thing that determines whether an AI system is any good was decided before either of those choices, in whether someone did the work of deciding what mattered, structuring it clearly, and being honest about what to leave out. Files and folders are just the format that makes it hardest to fake that the work happened. That's why, right now, they're winning the argument.
Frontmatter: content/thinking/files-and-folders-vs-agents.md
title: Files and folders vs. agents slug: files-and-folders-vs-agents summary: The infrastructure debate over filesystems and vector databases is the same argument as context design, one layer down the stack. stat: 1 layer stat_label: how far down the stack the same argument about context repeats
If this way of thinking is relevant to a problem you're facing, I'd be glad to talk it through.
Start a conversation