Readyset Docs

Benchmarking with cache-compare

Run the same query against the upstream database and against Readyset, side-by-side, to measure the impact of a cache.

rdst query cache-compare runs the same query against your upstream database and against a deployed Readyset cache concurrently, then reports side-by-side latency, throughput, and tail percentiles. It's the single most useful command for deciding whether a cache is worth shipping.

Common usage

# Default: 100 executions against each target
rdst query cache-compare a1b2c3d4e5f6 --target prod-orders

# Scale up the sample size
rdst query cache-compare a1b2c3d4e5f6 --target prod-orders --count 1000

# Time-bounded run
rdst query cache-compare a1b2c3d4e5f6 --target prod-orders --duration 60

# Sustained concurrent load (10 concurrent executions per target)
rdst query cache-compare a1b2c3d4e5f6 --target prod-orders -c 10 --duration 120

# Fixed-interval run (1 req/100ms per target)
rdst query cache-compare a1b2c3d4e5f6 --target prod-orders --interval 100 --duration 60

# Multiple queries at once (round-robin across them)
rdst query cache-compare a1b2c3d4 f6e5d4c3 9a8b7c6d --target prod-orders --count 500

You pass the original target name (prod-orders) to cache-compare, not the -cache target. RDST resolves the upstream/cache pair automatically. If no paired cache exists yet, cache-compare stops with a hint to run rdst cache deploy --target <name> --mode docker first.

What the output looks like

$ rdst query cache-compare a1b2c3d4e5f6 --target prod-orders --count 500 -c 10

Resolving targets ...
  Upstream:  prod-orders        (PostgreSQL)
  Cache:     prod-orders-cache  (Readyset)

Running 500 executions per target, concurrency 10 ...
  [==============================] 500/500  upstream
  [==============================] 500/500  cache

Results
────────────────────────────────────────────────────────────────────
  Query: SELECT * FROM orders WHERE customer_id = $1 ORDER BY ...

                          Upstream        Cache         Improvement
  ─────────────────────── ─────────── ──────────────── ──────────────
  Total executions         500         500
  Total time               00:04:12    00:00:08
  Throughput               2.0 qps     62.5 qps        31x
  Avg latency              412 ms      13 ms           32x
  p50 latency              389 ms      11 ms           35x
  p95 latency              512 ms      18 ms           28x
  p99 latency              847 ms      41 ms           21x
  Errors                   0           0

Verdict
  ✓ Cache provides substantial latency and throughput improvements.
  ✓ Tail latency (p99) also improves significantly, suggesting the cache
    absorbs variable load effectively.

Execution modes

cache-compare supports three ways to control load:

ModeFlagsWhen to use
Fixed count--count NQuick apples-to-apples comparison — each target gets exactly N requests
Duration--duration <sec>Measure sustained throughput over a fixed window
Concurrency-c <N>Simulate load: maintain N concurrent in-flight executions
Fixed interval--interval <ms>Simulate steady traffic: one request every N ms

Combine -c or --interval with --duration for load-test-style measurements.

cache-compare runs real queries against your upstream. The default is a gentle 100 executions, which is safe. Before scaling up (-c 50, --duration 600, etc.) make sure your upstream can handle the added load, or target a staging database.

Running against multiple queries

You can pass multiple queries at once. cache-compare will round-robin through them:

rdst query cache-compare \
    a1b2c3d4e5f6 \
    f6e5d4c3b2a1 \
    9a8b7c6d5e4f \
    --target prod-orders --count 1000

Useful for comparing the overall impact of a set of caches, not just one. The output includes per-query breakdowns plus an aggregate row.

From a CSV

For larger query sets, put them in a CSV:

queries.csv
hash,name
a1b2c3d4e5f6,top-orders-by-customer
f6e5d4c3b2a1,order-detail-with-customer
9a8b7c6d5e4f,session-lookup
rdst query cache-compare --file queries.csv --target prod-orders --duration 300

Flags reference

FlagPurpose
queriesOne or more hashes / names / inline SQL (positional)
-f, --file <csv>CSV of queries to compare
-t, --target <name>Upstream target (cache target resolved automatically)
--count <N>Stop after N executions per target (default 100)
--duration <sec>Stop after N seconds per target
--interval <ms>Fixed-interval mode: one request every N ms per target
-c, --concurrency <N>Concurrency mode: N concurrent executions per target
--quietMinimal output; only show the summary
--skip-warningSkip the "this will run queries against your database" warning

Interpreting the result

Questions to ask when reading a cache-compare result:

SignalWhat it means
Large latency improvement + throughput jumpCache will clearly help. Ship it.
Good avg, weak p99Cache is working but the warm-up cost is visible. Larger samples usually close the gap.
Similar latencyEither the query is already fast upstream, or Readyset can't cache the query's result set efficiently. Check rdst analyze for why.
Lower throughput on cacheUnusual — usually indicates a deployment issue (cold Readyset, under-provisioned resources). Check rdst cache show for hit rate.
Errors on cache onlyQuery became unsupported since it was added, or the Readyset instance is unhealthy.

See also