Lemit Trusts Readyset to Reach 100K QPS on MySQL, Without Changing a Single Line of Code
6 min read
•
11 days ago
When vertical scaling, read replicas, and MEMORY tables weren't enough, Readyset's SQL-level caching broke through the ceiling, delivering over 109,000 queries per second on MySQL.
Key Results
109K
Queries Per Second
0
Lines of Code Changed
<1ms
Cached Query Latency
8x
Read Throughput Increase
About Lemit
Lemit (Lemit Tecnologia da Informacao) is a Brazilian data enrichment and credit intelligence company based in Sao Paulo. Their platform enables businesses to make faster, more accurate credit decisions by enriching, qualifying, and cleansing customer data at scale, powering individual lookups, batch integrations, and real-time API queries across millions of records with 100% all-flash storage.
The Challenge: MySQL at Its Limits
Lemit's data enrichment platform powers real-time credit decisions for businesses across Brazil, processing individual lookups, batch enrichments, and API queries against millions of records. The workload is highly sensitive to latency. Every millisecond matters when clients are waiting on credit decisions. The codebase was mature with no room for application changes, and the production workload was impossible to reproduce in QA.
The team had already exhausted conventional approaches to scaling MySQL:
- Aggressive tuning of OS and MySQL parameters on Percona Server for MySQL
- Async read replicas for redundancy and OLAP offload
- MEMORY engine tables for fast in-memory lookups, but these lacked persistence, transaction support, and used table-level locking
- Massive vertical scaling to 256 virtual processors (2× AMD EPYC 7713 64-Core, 128 physical cores with hyperthreading, dual NUMA nodes), 1.4 TB of DDR4-3200 RAM, and a tiered XFS storage layout (14 TB data, 6 TB binlog, 1 TB redo) served from a Dell EMC Unity XT 380F All-Flash array over Fibre Channel
Despite this investment, read queries were still competing with writes for the same InnoDB buffer pool and CPU cycles. The database was better resourced than ever, but throughput had plateaued. Lemit had reached the ceiling of vertical scaling.
The Solution: SQL-Level Caching With Readyset
Lemit deployed Readyset as a transparent, wire-compatible caching layer in front of their MySQL primary. Unlike traditional caching solutions that require application changes, cache key management, and manual invalidation, Readyset operates at the SQL protocol level, which makes it completely invisible to the application.
SQL-Level Caching
Readyset intercepts read queries between the application and MySQL, caching results at the SQL level. No Redis, no Memcached, no application-level cache keys to manage.
Automatic Cache Freshness
Cached results are incrementally updated via the MySQL binlog as underlying data changes. No TTLs to configure, no stale data, no manual invalidation logic.
Wire-Compatible Drop-In
Readyset speaks the MySQL wire protocol natively. The application connects to it exactly as it would connect to MySQL. Zero code changes required.
Sub-Millisecond Latency
Cached queries are served from memory in under 1ms, orders of magnitude faster than even the most optimized MySQL queries hitting InnoDB.
How Readyset Integrated Into Lemit's Stack
Readyset was deployed alongside the existing MySQL primary and replica, with ProxySQL providing intelligent query routing. ProxySQL analyzed each query's digest and routed it to the optimal backend:
| Traffic Type | Routed To | Why |
|---|---|---|
| High-volume read queries | Readyset (port 3309) | Sub-millisecond response from cache |
| Writes / DML | MySQL Primary (port 3307) | Strong consistency required |
| Analytical queries | MySQL Replica (port 3312) | Offload from primary |
With 16 query rules configured in ProxySQL, the highest-volume read patterns (accounting for hundreds of millions of hits) were directed to Readyset. The cache absorbed the vast majority of read traffic before it ever reached MySQL.
Why Readyset Succeeded Where Other Approaches Didn't
Lemit had already tried multiple strategies to scale their MySQL workload. Here's how Readyset compared to each:
Readyset vs. Previous Approaches
| Approach | Limitation | How Readyset Solves It |
|---|---|---|
| Vertical Scaling | Reads still compete with writes for CPU and buffer pool; hardware has a ceiling | Readyset offloads reads entirely. They never touch MySQL |
| Read Replicas | Replication lag, operational complexity, still hitting InnoDB for every query | Readyset serves cached results from memory in <1ms with zero lag for cached queries |
| MEMORY Tables | Non-persistent, no transactions, table-level locking, requires manual sync | Readyset auto-syncs via binlog, supports concurrent reads, persists across restarts |
| Application-Level Cache (Redis/Memcached) | Requires code changes, cache key management, manual invalidation | Readyset requires zero code changes. It caches at the SQL level transparently |
| MySQL Query Cache | Deprecated, invalidates entire cache on any write, doesn't scale | Readyset incrementally updates only affected cached results. Writes don't flush the cache |
The Results: From Plateau to 100K+ QPS
The impact of deploying Readyset was immediate and dramatic:
109,120
Queries Per Second, powered by Readyset
Before Readyset
- ~13K SELECTs/sec hitting MySQL directly
- Peak threads running: 63
- Vertical scaling ceiling reached
- MEMORY tables with data loss risk
- All reads competing with 26K INSERTs/sec
After Readyset
- 109K+ total QPS across the stack
- Sub-millisecond latency on cached reads
- Primary freed for writes only
- Automatic cache sync via binlog
- Zero application code changes
What Readyset Enabled
- Broke through the 100K QPS barrier that hardware scaling alone couldn't reach
- Eliminated MEMORY table limitations. Readyset replaced non-persistent, table-locked caches with automatically-maintained, concurrent SQL-level caching
- Freed the MySQL primary for writes. Read traffic absorbed by Readyset never reaches InnoDB
- Zero code changes required. The application connects through ProxySQL exactly as before
- Reduced operational complexity. No additional replicas, no Redis cluster, no cache invalidation code
Lessons Learned
Lesson 1: Adaptation and experimentation are essential. Each phase of Lemit's journey built on learnings from the previous one. There was no single "right answer" from the start. The architecture evolved through testing and observation.
Lesson 2: Conventional scaling has a ceiling. Readyset breaks through it. Bigger servers and more replicas couldn't solve the fundamental problem of reads and writes competing for the same resources. Readyset's SQL-level caching was the architectural innovation that changed the equation.
Lesson 3: Continuous observation is critical. Durability parameters like sync_binlog and innodb_flush_log_at_trx_commit were tightened later as deliberate safety choices, trading some write throughput for stronger crash guarantees. Percona PMM monitoring the entire stack (including Readyset) was essential for quantifying the performance cost of those choices and confirming that the workload could still hit its SLAs.
Readyset now operates as a core performance layer in Lemit's stack, ensuring their MySQL infrastructure delivers the throughput and latency their application demands, today and as traffic continues to grow.
Ready to Break Through Your MySQL Scaling Ceiling?
Readyset delivers sub-millisecond query performance for MySQL and PostgreSQL, without changing your application code. See what it can do for your workload.
Authors