Context Hierarchy as Cache Hierarchy
Context management in agentic systems is a caching problem. We already solved this once.
Agents · Distributed Systems · Context
The core insight
Context management in agentic systems is a caching problem. The primitives are identical — we're just rediscovering them with LLMs in the loop.
Phil Karlton's quote still applies: "There are only two hard problems in computer science: cache invalidation and naming things." Agentic systems are running into both.
The mapping
| Caching concept | Agent/harness equivalent |
|---|---|
| Cache hit | Answering from in-context recall |
| Cache miss | Triggering a tool call to retrieve fresh data |
| TTL (time-to-live) | How long a loaded doc can be trusted before re-fetching |
| Cache invalidation | Stale context — the hardest problem in both domains |
| Cache hierarchy (L1/L2/L3) | Memory → pre-loaded docs → session artifacts → live retrieval |
| Write-through cache | Harness hook that updates the index whenever a file is written |
| Cache-aside pattern | Model checks context first; on miss, fetches from source and updates |
| Authoritative source / origin | Filesystem / API — never stale by definition |
| Cache-Control headers | Retrieval trigger rules in CLAUDE.md or harness config |
The concrete example that surfaced this
Asked Claude "how many specs are in the folder?" It read from the README index already in context — which was stale. Two files existed on disk that hadn't been indexed. Classic cache hit on an invalidated L2 entry.
The fix isn't "always retrieve everything." It's the same as CDN cache policy: the origin declares the TTL, the cache enforces it.
In harness terms: existence queries ("what files", "how many", "do we have X") should always trigger a tool call. Reference data (vertical ontology, benchmarks) can be recalled within a session. Permissions and billing state should never be cached.
Cache policy as a business/product decision
Freshness requirements aren't a technical default — they're a product decision per data type.
| Data type | Policy | Rationale |
|---|---|---|
| File/folder inventory | Always retrieve | Writes can happen outside the session |
| Spec file contents | Session-scoped TTL | Read once per session, recall after |
| Benchmark data | Long TTL (monthly) | Slow-changing by design |
| Partner contact status | Retrieve before act | Before sending email, re-read tracker |
| Permissions / access control | No cache | Business and compliance risk |
| Collaborative state (multi-agent) | Always retrieve | No single node knows what others wrote |
When stale is acceptable
- Immutable or slow-changing data (vertical ontology, industry classifications)
- Single-session with no external writes — if nothing outside the session could have changed it
- Idempotent reasoning — if the answer is the same regardless of re-reading, retrieval cost isn't worth it
- Reference data with known update cadence — if TTL is understood and respected
The distributed systems connection
In multi-agent systems, the stale context problem gets worse. No single agent can know what other agents have written. This is the distributed cache invalidation problem — exactly what made building consistent distributed databases hard. Solutions from that world transfer:
- Write-through: harness hook updates index on every file write
- Read-through: model always retrieves on existence queries, populates context as side effect
- Eventual consistency: acceptable for reference data, dangerous for state/inventory
X post draft
Working with AI agents requires revisiting some old ideas from software engineering.
Phil Karlton said: "There are only two hard problems in computer science: cache invalidation and naming things."
Turns out, context management in agentic systems is just a caching problem in disguise.
The mapping is almost 1:1:
- Cache hit → model answers from in-context recall
- Cache miss → triggers a tool call to retrieve fresh data
- TTL → how long a loaded doc can be trusted before re-fetching
- Cache invalidation → stale context (still the hardest problem)
- L1/L2/L3 hierarchy → memory → pre-loaded docs → live retrieval
- Cache-Control headers → retrieval trigger rules in your harness config
I ran into this directly. Asked an agent "how many files are in this folder?" It answered from a README index already in context — which was stale. Two files existed on disk that hadn't been indexed. Classic cache hit on an invalidated L2 entry.
The fix isn't "always retrieve everything." Same lesson as CDNs: the origin declares the TTL, the cache enforces it. Existence queries always hit the filesystem. Reference data can be recalled. Permissions are never cached.
In multi-agent systems it gets harder — no single agent knows what others have written. That's the distributed cache invalidation problem. The solutions from distributed databases transfer almost directly.
20 years of distributed systems thinking doesn't go stale. It just finds new substrates.
Angles to develop
- X post: "Context management in AI agents is a caching problem" — mapping table as the body, Phil Karlton quote as the opener. Thread format.
- Substack: Deeper — connect to Primitive 3 (Context Management) from agent primitives framework. Retrieval/Ranking/Compaction = eviction and promotion policies.
- Forbes angle: "The hidden infrastructure problem in enterprise AI agents" — cache invalidation at scale, multi-agent consistency
- Academic angle: Formal model of context freshness decay + policy framework — possible CIKM industry track
Pillars
1 (AI agent architecture), 2 (Enterprise AI), 5 (Builder/practitioner)