Shallow Caching
Shallow Caching
Shallow caching is a TTL-based query result caching mode in Readyset. When a query is registered as a shallow cache using CREATE SHALLOW CACHE, Readyset stores the result set in memory and serves it directly to clients until the TTL expires, without building or maintaining a materialized view in the dataflow graph.
Because shallow caching does not require a materialized view, it supports a broader range of queries than deep caching-based caches, including queries with constructs that the dataflow engine does not currently handle.
Cache entries are keyed on the query and its parameters, not on the connecting user, so any client that authenticates to Readyset can read any shallow cache regardless of its upstream database permissions. Cache only queries whose results are safe for every permitted user. Per-user access control is in active development. See the release notes for updates.
The exception is Postgres row-level security: a cache over an RLS-protected table is automatically keyed on the session state its policies read, so it cannot return rows across tenants.
How it works
Query parameterization
When Readyset registers a shallow cache, it automatically replaces literal values in the query with placeholders. For example:
SELECT * FROM products WHERE id = 42is normalized to:
SELECT * FROM products WHERE id = $1This means queries that differ only in their literal values (WHERE id = 1, WHERE id = 2, and so on) share a single cache entry template. Each unique combination of parameter values gets its own cached result, but the cache infrastructure is shared across all parameterizations of the same query shape.
Parameterization also applies to IN lists. The entire set is treated as a single parameter, so WHERE id IN (1, 2, 3), WHERE id IN (4, 5, 6), and WHERE id IN (7, 8) all share the same cache template regardless of how many elements the list contains.
TTL expiration
Each cache entry has a lifetime configured by the POLICY TTL clause (or the server default --default-ttl-ms if omitted). TTL is calculated based on last access time: an entry is considered stale when it has not been accessed within the TTL window. When a stale entry is accessed, Readyset fetches a fresh result from upstream and replaces the entry.
TTLs need not be manually configured unless desired. If omitted, Readyset falls back to server defaults. Caches created with the ADAPTIVE option also adjust each entry's refresh rate based on how often its data is observed to change; see Adaptive refresh.
Refresh behavior
A refresh is a Readyset-generated query sent to your upstream database solely to update a stale shallow cache entry. Refreshes happen on a per-key basis: each unique set of parameter values is refreshed independently. The right refresh interval balances cache freshness against the query load each refresh places on your database.
Readyset supports two refresh strategies:
On-access refresh (REFRESH <n> {SECONDS | MILLISECONDS | MS}): When a request hits a shallow cache and the cached value is at least n old (in the specified unit), Readyset preemptively issues a background refresh. The current request receives the existing cached value immediately; the refreshed result is available for subsequent requests. If REFRESH is omitted, Readyset applies an implicit default refresh interval equal to half the TTL.
Scheduled refresh (REFRESH EVERY <n> {SECONDS | MILLISECONDS | MS}): Readyset unconditionally re-executes the query against upstream on a fixed schedule, regardless of whether any client has requested it. Automatic refreshes continue as long as the cache entry has not been evicted due to TTL expiration. Note that if the query takes longer to execute than the refresh interval, multiple refreshes for the same key can be in-flight concurrently, producing consistently fresh results at the cost of higher database query volume.
In both cases, the refresh interval must be less than the TTL.
Adaptive refresh
A cache created with the ADAPTIVE option tunes each entry's refresh interval to the rate at which its data is observed to change. When a refresh returns changed data, the entry's interval shrinks so future changes are picked up sooner; when refreshes keep returning the same data, the interval grows back toward the configured interval, which acts as the ceiling.
Adaptive refresh works with both on-access and scheduled refresh. To keep shortened intervals from overloading the upstream database, the extra refresh load they generate is capped by the --shallow-adaptive-max-extra-load-percent server option. At the cap, intervals stop shrinking until aggregate load decreases.
Caches created by automatic shallow caching are always adaptive. The current per-entry refresh interval is visible in the refresh_period_ms column of SHOW SHALLOW CACHE ENTRIES.
Database load considerations
A primary goal of shallow caching is to reduce database load. However, it is possible to configure shallow caches in a way that increases load. For example, setting an aggressive REFRESH EVERY interval on a high-cardinality cache with many distinct parameter values can generate more upstream queries than the cache saves.
To bound the total number of concurrent background refreshes across all shallow caches, use the --shallow-refresh-workers option (env: SHALLOW_REFRESH_WORKERS). This defaults to 40 concurrent refreshes. For adaptive caches, the extra load from shortened refresh intervals is additionally capped by --shallow-adaptive-max-extra-load-percent. When choosing refresh intervals for individual caches, consider both the per-query cost and the aggregate load across all caches.
Memory and eviction
All shallow caches share a single in-memory store. Its capacity is a share of the server's --memory-limit, configured with --shallow-memory-percent.
When the store is full, admission and eviction are cost-aware. Each cached result records how long its query took to execute upstream, and Readyset weighs that against how often the result is requested when deciding what to keep. A result that is requested less often but is slow to recompute can be kept ahead of one that is requested more often but is cheap to recompute, so memory is spent where caching saves the most database time.
As a backstop against single entries crowding out the rest of the store, result sets larger than --shallow-max-entry-bytes are served from upstream without being cached.
Coalescing
When multiple client requests for the same query (same parameter values) arrive simultaneously and all miss the cache, Readyset can coalesce them: only the first request goes to upstream, and the rest wait for that result to be populated before responding. This prevents a thundering herd from overwhelming upstream on a cache miss.
The coalescing window is configured with the optional COALESCE clause (or the server default --default-coalesce-ms if omitted). During this window, concurrent identical requests wait for the first to complete rather than each issuing an independent upstream query.