Shallow Caching

Shallow caching is a TTL-based query result caching mode in Readyset. When a query is registered as a shallow cache using CREATE SHALLOW CACHE, Readyset stores the result set in memory and serves it directly to clients until the TTL expires, without building or maintaining a materialized view in the dataflow graph.

Because shallow caching does not require a materialized view, it supports a broader range of queries than streaming dataflow-based caches, including queries with constructs that the dataflow engine does not currently handle.

How it works

Query parameterization

When Readyset registers a shallow cache, it automatically replaces literal values in the query with placeholders. For example:

SELECT * FROM products WHERE id = 42

is normalized to:

SELECT * FROM products WHERE id = $1

This means queries that differ only in their literal values (WHERE id = 1, WHERE id = 2, and so on) share a single cache entry template. Each unique combination of parameter values gets its own cached result, but the cache infrastructure is shared across all parameterizations of the same query shape.

Parameterization also applies to IN lists. The entire set is treated as a single parameter, so WHERE id IN (1, 2, 3), WHERE id IN (4, 5, 6), and WHERE id IN (7, 8) all share the same cache template regardless of how many elements the list contains.

TTL expiration

Each cache entry has a lifetime configured by the POLICY TTL clause (or the server default --default-ttl-ms if omitted). TTL is calculated based on last access time: an entry is considered stale when it has not been accessed within the TTL window. When a stale entry is accessed, Readyset fetches a fresh result from upstream and replaces the entry.

TTLs need not be manually configured unless desired. If omitted, Readyset falls back to server defaults. Refresh rates adjust automatically based on the TTL and system load. In the future, Readyset may optionally support dynamic TTL adjustment based on observed data change rates.

Refresh behavior

A refresh is a Readyset-generated query sent to your upstream database solely to update a stale shallow cache entry. Refreshes happen on a per-key basis: each unique set of parameter values is refreshed independently. The right refresh interval balances cache freshness against the query load each refresh places on your database.

Readyset supports two refresh strategies:

On-demand refresh (REFRESH <n> SECONDS): When a request hits a shallow cache and the cached value is at least n seconds old, Readyset preemptively issues a background refresh. The current request receives the existing cached value immediately; the refreshed result is available for subsequent requests. If REFRESH is omitted, Readyset applies an implicit default refresh interval equal to half the TTL.

Scheduled refresh (REFRESH EVERY <n> SECONDS): Readyset unconditionally re-executes the query against upstream on a fixed schedule, regardless of whether any client has requested it. Automatic refreshes continue as long as the cache entry has not been evicted due to TTL expiration. Note that if the query takes longer to execute than the refresh interval, multiple refreshes for the same key can be in-flight concurrently, producing consistently fresh results at the cost of higher database query volume.

In both cases, the refresh interval must be less than the TTL.

Database load considerations

A primary goal of shallow caching is to reduce database load. However, it is possible to configure shallow caches in a way that increases load. For example, setting an aggressive REFRESH EVERY interval on a high-cardinality cache with many distinct parameter values can generate more upstream queries than the cache saves.

To bound the total number of concurrent background refreshes across all shallow caches, use the --shallow-refresh-workers option (env: SHALLOW_REFRESH_WORKERS). This defaults to 100 concurrent refreshes. When choosing refresh intervals for individual caches, consider both the per-query cost and the aggregate load across all caches.

Coalescing

When multiple client requests for the same query (same parameter values) arrive simultaneously and all miss the cache, Readyset can coalesce them: only the first request goes to upstream, and the rest wait for that result to be populated before responding. This prevents a thundering herd from overwhelming upstream on a cache miss.

The coalescing window is configured with the optional COALESCE clause (or the server default --default-coalesce-ms if omitted). During this window, concurrent identical requests wait for the first to complete rather than each issuing an independent upstream query.