Nobody wants a cache

Your laptop is full of caches you have never once thought about. There are several in the CPU, each one orders of magnitude faster than the last, one in the memory controller, one in the operating system's page cache, one on the disk itself, another in your DNS resolver. Application reads pass through most of them without knowing them. You have never invalidated any of them and you would likely struggle to name them all.

That invisibility is deliberate. The rule systems designers have held themselves to, in hardware and in software both, is quite strict: a system with caches has to produce exactly the same results as the same system with the caches taken out. A cache may change how long an answer takes, but never the answer itself. These caches also decide continuously, because a branch predictor has no configuration file and the page cache never asks which files will matter this week. They stay correct, and they stay out of your way, even as the workload moves. In other words, the cache is "functionally invisible to software".

Database caching is one of the few places left where we still do it by hand. The cache that matters most usually sits above the database and it is built by application developers. It is the layer closest to the problem but also the one with no view of when the data underneath changes. The reality is nobody actually wants a cache. What people want is for reads to be fast, and for nobody to have to own that.

So what would it take for a database cache to earn that same invisibility: to decide correctly, to keep deciding as the workload moves, and to stay predictable?

Today we're launching the Readyset Platform, along with several components that make this invisibility possible. The platform combines new building blocks with ones we've shipped before. Chief among the new pieces are QueryPilot, our proxy, and Guard, which handles safety. Together they give you query acceleration that tunes itself continuously and safely. The goal hasn't changed: point your application at us, and we take care of the rest.

Caching that runs itself

In most database caching, you're the one who decides what gets cached. You write the rule, pick the expiry, revisit both when traffic moves, and then own the bug if you get it wrong. Plenty of teams do exactly that and get real results from it. They pay for it in developer time, and they keep paying.

Deciding what to cache, without asking anyone, needs two kinds of knowledge. You have to know what is being asked, which means seeing the whole stream of queries as it really is rather than as the schema suggests it might be. And you have to know how the data underneath is moving, because that is what determines when an answer stops being true. Neither one on its own is enough. An application cache has only a partial view of the first and none of the second, which is exactly why the developer ends up hand-writing invalidation: they are guessing at a fact the database already knows. And because that cache has no global view, it can't tell a query that stopped mattering last quarter from one that still does, so it spends memory on both, which means either a bigger Redis bill or hot keys getting evicted to make room. A change-data pipeline has the second and not the first, so it can tell you what changed but not whether anyone cared.

Databases settled this argument internally a long time ago. Nobody picks their own access paths any more, because the query optimizer does it, and it can do it because it holds both halves: the parsed query and the statistics about the data. Rule-based optimizers, which only ever saw the query, lost to cost-based ones, which saw both. The compiler took this over too. Whether it collects garbage or checks ownership at build time, the machinery now works out when a thing is no longer needed, because the person writing the code often can't say at the moment of writing. Cache invalidation is manual memory management for the read path.

Take the diagram below. Readyset sits in front of your database, on the wire, where every query passes through and it can see how the data is changing. That's why QueryPilot can make the call for you: it lives there, not in your app. For each query it decides whether it's safe to cache, hot enough to matter, and worth the memory, then caches the ones that are. One decision, made in the one place that sees everything.

Safety is settled before performance. If the answer to the first question is no, it goes to your database, and a miss costs you nothing.

Figure 1. How QueryPilot decides. Safety is settled before performance, a no at any gate sends the query to your database, and observed usage feeds back so the decision changes as the workload does.

It also keeps watching, which is what matters over a year rather than a week. Traffic moves: a query nobody ran in March is on your critical path by June, and data that changed hourly starts changing every few seconds. So the Platform keeps re-deciding. It swaps out what's gone cold, admits what's turned hot, and sizes its own memory from observed behavior. The cache you turn on in March is a different cache by June, and nobody on your team touched it.

'Automatic' has been overpromised

Automatic is a word that's been oversold, and a system that tunes itself can tune itself into a hole. Anyone who has been burned by a self-tuning system has good reason to feel skeptical.

The reasons to be skeptical are concrete. A plain LRU cache gets flushed by one-off queries, and SQL workloads produce a lot of those. A cache that ignores query cost fills up with cheap queries and leaves your slow ones uncached, so the hit rate looks good and the p99 doesn't move. A single hand-set expiry is either too stale for fast-moving data or too conservative for slow-moving data, and can't be both. And a cache that can't distinguish stale from wrong will eventually serve wrong, the clearest case being two users entitled to different rows getting the same cached result, which is the first thing the safety gate checks for.

Each of those is a way for automatic to be worse than manual, and closing them is most of the heavy engineering work. There are two commitments that fall out of it. A stampede on an expired entry costs your database just a few queries instead of the whole storm. And when something goes wrong inside Readyset, the query falls through to your database rather than failing. We can be slow, but we can't be the reason you're down.

None of that work is visible from the outside, which is most of why it is harder than it looks. The failure modes and the mechanics deserve their own piece, and we'll write that up separately.

Freshness is a dial

Coherence is a property of a memory bus. A cache in front of a database sits on a network, behind a replication stream, and can't offer it. What we can offer instead is a bound. You set the ceiling, at ten seconds or one second or whatever the query can stand, and no result is ever served to you older than that. Inside the bound the cache only gets fresher.

Inside that bound, it watches how fast each query's data actually changes and tunes that query's refresh to match. A products table that changes twice a day and a cart table that changes every few seconds don't get the same treatment, because they shouldn't. Keeping that straight across hundreds of queries is what the Platform handles on its own. Some queries deserve incremental maintenance in real time and some are fine a few seconds behind, and QueryPilot decides which, per query.

Why a proxy, and why you should run one anyway

All of this happens in one place: in the path, in front of your database, while owning none of it. That place is a proxy, and a proxy is worth more than most teams give it credit for. It pools connections, it shows you what your traffic actually is, it gives you somewhere to exert control, and it gets more life out of the cluster you already have.

A year ago we launched QueryPilot as automatic caching that ran on top of ProxySQL. Today QueryPilot is the proxy itself: our own, written in Rust, with the caching built in. We wrote it because everything above needs a layer that understands a query rather than one that forwards it. Deciding whether a result can be served correctly, working out what a query costs, knowing that two statements that look different are the same query with different parameters, and later judging whether a query should be allowed at all, are all the same work on the same parsed statement. Split across a pooler and something behind it, that means parsing twice, disagreeing occasionally, and running two things. So QueryPilot is one process on the wire: pooling, routing, the caching decision, and the cache itself.

And if you're not ready for ours, run something. ProxySQL is excellent and has been for years. PgBouncer is the default for good reason. PgDog is doing interesting work on routing and sharding. Any of them is a large win for a small amount of infrastructure, and I'd rather you deployed one of those than nothing.

Readyset Platform

Figure 2. QueryPilot in the path. It routes, decides what is worth caching, and does the caching. The database is untouched. Guard is dashed because it has not shipped yet.

The Platform is one layer in front of Postgres and MySQL: one process, one thing to run, with caching, routing and pooling as modules on it rather than products beside it. What used to be several pieces you assembled is now one you deploy. Here is what's in the release.

Caching that continuously tunes. QueryPilot already found and cached your high-impact queries automatically, with no query list and no TTLs to manage. What's new is underneath and around it: it now runs on our own proxy rather than a third party's, it spans both low-overhead and high-performance caching, and the whole loop, selection, memory, refresh and eviction, runs continuously as one layer that keeps re-deciding while your traffic moves.

QueryPilot, our own proxy. It parses and understands each query rather than forwarding it, which is what lets the caching decision, the cost estimate and, later, policy all work from the same parsed statement in the same process.

Freshness as a dial, per query. You set the staleness ceiling. Inside it, each query's refresh rate is tuned to how fast that query's own data actually changes, without you touching it.

A substrate for what comes next. Once you have a layer that sees every query on its way to the database, and it's correct enough to trust, the useful work is what you put in that layer. Guard, which checks what a query is allowed to touch before it reaches the database, was previewed at Web Summit Rio and is being prepared for full release. It runs in the same process and needs nothing new from you. Guard is the first of several things that become possible once every query passes through one place that understands it.

Try it

Hussle turned this on across their read path. Cached reads come back in about 0.1ms, against 1.6ms to the database, and roughly 83% of their reads now never reach MySQL at all. That's about five reads served for every one that gets through to the primary.

Khanh Tran, VP of Engineering at Hussle shares: "Readyset has helped us reclaim capacity from our database infrastructure. Our optimization has moved from low config to no config. Readyset automatically identifies which queries are most costly, caches them, and tunes the freshness of each one. That ongoing optimization is now fully automated, and it has meaningfully slowed the growth of our database infrastructure."

rdst is our command line tool, and it's the simplest way to see the whole flow on your own machine, against your own database. Install it, point it at a database, and turn caching on and off against your own traffic. It runs locally, it's free, and it ships with an MCP server so your own tools can ask it the same questions. Give it a try.

Most people I know who run databases would sooner build their own tools than rent someone else's. You should be able to check all of this yourself.

Cheers,

Gautam