A TTL Cache Needs To Know the Query

Plenty of teams cache query results with a TTL. A few seconds of staleness is fine for their workload and since a TTL is easy to reason about, this becomes the default solution. The trouble however, is the primitive underneath. Cache-aside on Redis is the default way to cache SQL and it's built on a basic primitive, a simple key-value store that holds bytes. We hold the view that a SQL query cache should not only hold the results, but also the query that generated them and preferably additional metadata. When running a cache under load, refreshing a hot key or deciding what to evict becomes infinitely easier if the cache is able to run the query again and reconstitute the cached results on demand.

Since a TTL cache built on Redis or Memcached doesn't know what a query is, it can't re-run one to refresh a hot key, collapse a burst of misses into a single call, or differentiate a 900 ms result from a 2 ms one. That work is left up to your application.

We've written before about why key-value stores struggle to cache relational data when freshness is the goal. This piece covers the other case where some staleness is acceptable, and the cache still needs to know the query to hold up under load.

A note on names. Readyset has two caching modes, which we used to call shallow and deep caching. We now call them pull-based and push-based caching, after how each one keeps results fresh. A pull-based cache pulls new results from the database on a schedule, while a push-based cache has every write pushed into the cached results as it happens. The products are the same, and earlier posts use the old names.

Adding hardware is a harder way out than it used to be. Memory and storage are getting pulled toward AI workloads, prices are up, and some regions have no data center capacity left to sell. Database bills are climbing faster than traffic, so getting more out of the database you already run is worth real engineering effort.

All of this matters more every month. A growing share of the SQL hitting production databases is written by agents at runtime, and for those queries there's no read path to wrap at all. We'll come back to that at the end.

What a key-value store knows about your data

Very little. Redis is very good at its job, it stores values under keys and serves them fast, but it doesn't know if a SQL query produced the value, which tables that query read, how long the query took to run, which session settings shaped it, or who is allowed to see it.

A cache-aside implementation fills the gap in application code. Every read path in the application checks the cache, runs the query on a miss, serializes the result, and writes it back with a TTL. Each of those steps is a decision your team makes by hand, query by query.

A query cache starts from the query. Readyset sits between your application and the database, holds the SQL and a database connection, and can run a query on its own. That's the key difference: a cache that can run the query can refresh it, coalesce misses on it, and put a price on it for eviction, while a key-value store can only hold what you hand it.

Outside databases, caching already works in this way. A reverse proxy like Varnish, or a CDN, keeps the URL, so it can refetch from the origin, serve a stale copy while it refreshes (HTTP's stale-while-revalidate), and collapse a burst of misses into one origin request. Readyset's pull-based cache does the same for SQL.

	Cache-aside on a key-value store	Readyset pull-based cache
What it stores	Bytes your app serialized, under a key your app chose	Query results, keyed by SQL, parameters, schema, and session settings
Read path	Cache-aside code in every read: check, query on miss, serialize, write back	Unchanged; the app sends SQL as before
On a miss	Your app runs the query and fills the cache	Readyset forwards the query to the database and fills the cache
Refresh	A job you build, since the store can't run a query	Readyset re-runs the query in the background
Stampedes	A lock you build	Concurrent misses collapse into one query
Eviction	LRU or LFU, access patterns only	Weighs size, query cost, and read frequency
Session and tenant safety	Only as good as your key design	Session settings and security context are part of the key

Where a TTL cache falls short

We ran into each of the problems below while building Readyset's pull-based cache, which we introduced earlier this year under the shallow cache name. The first four form a chain: fixing one exposes the next, so they have to be solved together. The last four show up in every TTL cache regardless. Every fix depends on something a key-value store doesn't have.

1. Hot keys expire under load

A fixed TTL forces a specific tradeoff. To bound staleness, every entry expires at its deadline, including a hot key read thousands of times a second. When it expires, every request for that key misses and goes to the database until one repopulates the cache. Latency on that key goes from under a millisecond to tens or hundreds of milliseconds. The p50 looks fine. Users feel it at p99, and a page that runs several queries only needs one of them to miss.

This is also one reason why hit rate is not always the right metric to watch. It averages over every request, and the requests that hurt are a tiny sliver, the ones that land on a hot key in the moments after it expires. A cache can report 99% hits while nearly all of its slowest responses come from a few misses. Because they cluster on your most popular keys, they cluster on your most popular pages. Watch p99 latency on cached queries instead.

What teams do today: accept the spikes, or lengthen the TTL and accept staler data.

Readyset: the TTL counts from the last read, not from when the entry was created. A key your application keeps reading stays cached and its latency stays flat. Keys that go quiet, age out and free memory. Uses: read tracking per cached result.

2. Hot keys that never expire go stale

Sliding TTL fixes the spike but it opens a new gap. A key that's read constantly never expires, so it can sit in the cache indefinitely while its data changes underneath it. A refresh also takes time. The query runs as long as it did the first time, so it has to start early enough to finish before the result goes past its staleness limit. For a 900 ms query, that lead time is real.

What teams do today: this problem only appears once expiry is sliding, which is why most TTL caches stay fixed and keep the spikes from problem 1.

Readyset: while a key is still being read, Readyset re-runs its query against the database and swaps in the new result. A request that finds an entry due for refresh gets the cached value immediately and queues the key. All of it runs on Readyset's own internal workers, off the request path.

3. Refresh multiplies database load

Once refresh exists, a hot key with thousands of readers can trigger thousands of refreshes.

What teams do today: nothing, because without refresh there's nothing to deduplicate.

Readyset: one refresh per key is in flight at a time. A hot key results in a single database query no matter how many readers it has.

4. One refresh schedule fits nobody

Keys change at different rates. Some results move (i.e. they are updated) every few seconds, while others sit unchanged for days. Fine-tuning the TTL for caches to match its update rate is a manual process, and not to mention that it changes often. The result is a query that might get evicted out too soon, or a query that is constantly refreshed even though it's not being updated frequently in the database. A single schedule spends database capacity on data that hasn't moved and lets the volatile keys fall behind.

What teams do today: pick one TTL for everything, usually tuned for the most volatile data, and pay for it in database load.

Readyset: each refreshed result is compared with the previous one. Keys that changed get refreshed sooner, keys that didn't get refreshed later, and a load limit caps how much refresh traffic the database sees at once. In our Adaptive Cache benchmarks, this cut p95 staleness by 97% compared with a straight TTL, for about 10% more refreshes. Uses: each key's history of changes.

5. Misses arrive all at once

Refresh covers keys already in the cache. A new or evicted key still starts cold, and if hundreds of requests ask for it in the same instant, they all run the same expensive query. That's a cache stampede.

What teams do today: absorb it with database headroom, which is capacity paid for and idle most of the time.

Readyset: concurrent misses on one key collapse into a single upstream query. The rest wait for that result.

6. Eviction drops the wrong results

LRU and LFU look only at access patterns. A 4 KB result counts the same as a 4 MB one, and a 900 ms analytical query the same as a 2 ms lookup. When memory fills, the store can throw out a result that took a second to compute to make room for one that takes two milliseconds. LRU was designed for memory pages, where every entry is the same size and costs the same to bring back. That assumption holds for a CPU or an operating system's page cache. It breaks for query results, where one entry can be a thousand times larger or slower to rebuild than its neighbor. Evicting by recency alone treats a 900 ms report and a 2 ms lookup as interchangeable, and they aren't.

What teams do today: add memory until eviction rarely happens.

Readyset: every entry records its size and how long its query took on the database. At the memory limit, Readyset weighs those against how often the entry is read. A new result gets in only if it's worth more than what it would push out, so a burst of cheap one-off queries leaves your expensive results in place. Uses: result size and query cost.

7. The same query returns different answers

The same SQL with the same parameters can return different results depending on the session. Character set and collation change how text compares and sorts. Timezone changes how timestamps come back. SQL mode decides what happens on division by zero or an invalid date. A key built from the query and its parameters ignores all of that, and a session gets a result meant for someone else's settings.

What teams do today: mostly nothing. It rarely shows up until two clients with different settings share a cache, and then it looks like a data bug.

Readyset: the key includes the statement, its parameters, the schema it resolves against, and the session settings that affect the result. A hit returns what the database would have sent to that session. Collation alone has plenty of edge cases; we wrote up how Readyset handles them. Uses: the session's settings, seen on the wire.

8. The cache doesn't know who's asking

A key-value store hands the value to anyone with the key, including a user whose access was revoked an hour ago. Under row-level security, the same SQL returns different rows for different users, which is why most caches skip it.

On a key-value store: permission checks in every read path and per-tenant key namespaces.

Readyset: it checks each user's permissions when the query is first created, keeps it up to date by querying the database every 5 minutes, stores a local copy, and consults it before returning data. When it isn't sure, it forwards the query to the database. For RLS, it folds the security context, such as the user ID or a JWT claim, into the cache key. If it hits a policy it can't interpret, it doesn't cache that query. Uses: the database user and security context behind each query.

What knowing the query makes possible

Every fix above comes from the same place. Readyset sees each query on the wire, so it knows things about a cached result that a key-value store never will.

The result is a cache that keeps p99 latency flat under load, while background workers keep results fresh at the right interval for each one. That's what a TTL cache was supposed to be. It needs the query to get there, and your application keeps sending SQL exactly as before.

What about Redis's own answers?

The strongest evidence for this argument comes from Redis itself. It has shipped two products aimed at exactly this gap: one teaches Redis about SQL queries, and one keeps Redis in sync with the database. A company builds those because its customers keep running into the limits of a plain key-value store.

Redis Smart Cache wraps your JDBC driver and caches SQL results in Redis without code changes. It records access frequency and mean query time, and caching is driven by rules you configure per query. Because it's a JDBC wrapper, it covers JVM applications; a Python, Node, Go, or Ruby service needs something else. Freshness is still rule-based TTL on top of Redis.

Redis Data Integration (RDI) uses change data capture to copy rows from your database into Redis hashes or JSON documents, typically within seconds. That keeps rows fresh, but it replicates tables, not query results. Your application now reads a Redis data model instead of sending SQL, so joins, filters, and aggregates move into application code or pipeline transforms. Per Redis Cloud's docs, it assumes the app can tolerate eventual consistency.

Readyset takes the third path: keep the SQL, keep your driver in any language, and put the query-aware work in a proxy between the application and the database.

When a few seconds of staleness is too much

Some workloads can't accept any staleness. For those, Readyset has a second mode: push-based caching. Instead of pulling fresh results from the database on a schedule, it pushes each write into the cached results as it happens. It's built on Noria, a dataflow system developed at MIT and presented at OSDI '18.

When you cache a query, Readyset turns it into a graph of SQL operators ending in a node that holds the result set. It follows the database's replication stream and turns each INSERT, UPDATE, and DELETE into a delta that flows through the graph. Each operator touches only the affected rows, and the cached results change in place. The graph holds only the keys your application has asked for.

There's no invalidation logic to write and no TTL to tune. Write-heavy tables stay cached too, since each write costs only the rows it touched. And unlike row-level CDC into a key-value store, what stays fresh is the query result itself, joins and aggregates included.

For more on this mode, see Don't use key-value stores to cache from relational databases and Cache invalidation for SQL queries.

Agents make the case stronger

Everything above assumes a person wrote the read path. That assumption is fading. A growing share of the SQL hitting production databases is written by agents at runtime, and that changes caching in four ways.

Cache-aside has nothing to wrap. Cache-aside works by putting cache code around a read path a developer wrote. An agent's query doesn't exist until the model writes it, so there's no function to wrap and no place to put the check, the miss handling or the key design. Only a cache that sees the SQL itself can cache an agent's reads at all. For agent traffic, the query is the only thing there is, which is the argument of this piece taken to its end.

Nobody can pick the hot queries in advance. In an app written by people, the set of read paths changes at release speed, and someone can keep the cache list roughly current. Agent queries change with every prompt. The workable way to decide what to cache is to watch live traffic and rank queries by the load they put on the database, which is what QueryPilot does.

Agent traffic has the shape that breaks naive caches. Agents fan out: parallel sub-agents and concurrent sessions often ask for the same data at the same moment, which is a stampede by another name. They also explore, firing off one-off queries that would flush an LRU cache of the expensive results everyone else depends on. Request coalescing and cost-aware eviction were built for exactly these patterns.

A wrong answer becomes a wrong action. A person looking at a slightly stale dashboard usually notices. An agent takes the stale number, acts on it, and hands it to the next step. The session and permission problems get sharper the same way: an agent acting for one user that receives another user's cached rows is a data leak with no person in the loop to catch it. Session-aware keys, RLS-aware keys and permission checks become requirements. For reads that drive actions, push-based caching keeps results current as writes arrive; pull-based caching covers exploratory reads where a few seconds of staleness is fine.

This is also where database load is heading. Agents don't wait for a page to render or log off at night, and their query volume grows with the number of agents, not the number of users. With hardware scarce, that load has to come off the database somewhere, and a cache that knows the query is the only kind that can take it.

We'll be honest about what we don't know yet, how cacheable agent SQL is in practice. Our internal tests indicate that it's very repetitive. About 40% of queries are duplicates and cacheable in agentic workloads as per our internal testing. Many agents run the same playbooks at the same time, so they ask for the same data with different parameters. Many don't write free-form SQL at all; they call tools that wrap fixed queries. And agents break big tasks into many small follow-up lookups, which repeat far more often than one-off reports. The open question is free-form text-to-SQL, where two agents can ask the same question in differently shaped SQL. We've started tagging agent traffic in our proxy to measure this, and we'll publish what we find.

One connection string

Both modes sit behind the MySQL and PostgreSQL wire protocols (here's how a wire-compatible caching proxy works). Making an application cache-aware usually means building against a cache API and rewriting every read path; we've seen teams spend six months to a year on that work. With Readyset, it's a connection string change, in any language.

Keep Redis for sessions, counters, and queues. For query results, use a cache that knows what a query is.

Try it at readyset.io.