News

How ClickFunnels Overcame Database Scaling Challenges and Optimized MySQL Queries with Readyset

5 min read

4 months ago

How ClickFunnels Overcame Database Scaling Challenges and Optimized MySQL Queries with Readyset

Leading marketing & lead-generation platform, ClickFunnels, leverages Readyset’s database scaling solution to optimize MySQL query performance and reduce AWS Aurora costs. Let's dive into the details.

About the Customer: ClickFunnels

ClickFunnels is a leading platform that empowers businesses to build, launch, and optimize online sales funnels quickly and efficiently. By providing a comprehensive suite of tools, ClickFunnels allows users to create landing pages, sales pages, order forms, and membership sites without requiring extensive technical skills or coding knowledge. It is an all-in-one solution that is great for entrepreneurs, small businesses, and digital marketers alike who need a streamlined approach to converting website visitors into paying customers. Beyond funnel creation, ClickFunnels also offers services like email marketing, A/B testing, and upsell/downsell functionality, making it a robust platform for businesses aiming to scale their online presence and maximize revenue.

Customer Challenge: Managing Growing Database Demands

Despite its user-friendly approach, ClickFunnels’ rapid growth and increasing user interactions posed significant challenges for its underlying MySQL database infrastructure on AWS Aurora. As the platform scaled, managing millions of queries and maintaining fast response times became critical to sustaining customer satisfaction and operational efficiency. Traditional database solutions, such as read replicas, were becoming costly and inefficient, prompting ClickFunnels to explore more advanced solutions to handle the complex demands of their growing business. The need to optimize performance while controlling expenses drove the decision to explore innovative solutions for database scaling and query caching.

Customer Results: Optimizing MySQL Queries and Reducing Database Load 

To address these challenges, ClickFunnels adopted Readyset, a solution for MySQL query optimization that alleviates pressure on existing read replicas. The deployment involved multiple Readyset instances configured in a round-robin setup, integrated with ProxySQL servers to manage traffic distribution and redundancy. The entire deployment was up and running in production in less than two days. This architecture combining Readyset, ProxySQL, and AWS Aurora, enabled ClickFunnels to optimize query execution while maintaining a robust and reliable database environment, without requiring any changes to existing application code.

Percona Monitoring & Management (PMM) was utilized to continuously monitor and maintain the health and performance of the environment. During implementation, the team identified key queries contributing to high database loads and mirrored these to Readyset to build a warm cache of frequently accessed data. By progressively redirecting traffic to Readyset, ClickFunnels was able to dramatically reduce the workload on its read replicas and the upstream database - all while boosting, not compromising, on query performance. With this strategic approach, ClickFunnels is now set up to save thousands of dollars on slow and expensive database read replicas every month while supporting the growing demands of modern, data-driven applications.

Solution Overview: The Setup for Enhancing Database Scaling and Performance

ClickFunnels uses Readyset to handle the following key challenges:

1) reducing the workload on upstream instances of AWS Aurora for MySQL including its read replicas
2) providing better query performance for its applications. 

Today, Readyset is deployed in two distinct application environments at ClickFunnels. Each application environment uses a combination of two Readyset instances configured in a round-robin setup to handle traffic, ensuring efficient load distribution and improving overall redundancy. The deployment also took advantage of ProxySQL for routing queries, all without the need to bring down the applications at any point in time. The setup used three ProxySQL servers, which were responsible for routing specific query traffic to the Readyset instances. This configuration of Readyset allowed for MySQL query optimization while reducing the load on the primary databases.

Readyset Deployment Architecture at ClickFunnels
“ClickFunnels was able to introduce Readyset to its production environment in just a couple of weeks and reduce the workload on Aurora MySQL read replicas by over 85% bringing utilization down to just 6.6%. This allows us to greatly optimize our database infrastructure and get more from the same instance saving thousands needed to scale in future. - Javier Zon, Head of Platform, ClickFunnels” 

The final piece of the architecture was Percona Monitoring & Management (PMM) which allowed the ClickFunnels team to provide real-time insights for proactive monitoring for maintaining system reliability and efficiency.

Average Active Database Sessions (Before and After Readyset)

Configuring the Database Scaling Solution

The cache hostgroup in ProxySQL was configured to contain Readyset servers with higher weight and to minimize risk during deployment, the current read replicas were set up to act as failover. Once deployed, Readyset started handling all of the traffic for this hostgroup, while the read replicas moved be idle. The configuration looked as below:

Hostgroup

Server

Weight

Comment

200

readyset1.local

10000

App1 - Readyset 1

200

readyset2.local

10000

App1 - Readyset 2

200

read-replica1.local

1

App1 - Read Replica 1

200

read-replica2.local

1

App1 - Read Replica 2

202

readyset3.local

10000

App2 - Readyset 3

202

readyset4.local

10000

App2 - Readyset 4

202

read-replica3.local

1

App2 - Read Replica 3

202

read-replica4.local

1

App2 - Read Replica 4

During the analysis phase of the implementation, the team identified a subset of queries that were responsible for causing the majority of load on read replicas. Each one of those queries was given a query rule in ProxySQL, initially, the destination of the query was kept the same and mirrored to Readyset for query caching. 

Database CPU Load (Before and After Readyset)

This allowed Readyset to receive a mirror of the production traffic and populate the most frequently asked entries into cache. Next, we monitored p99 latencies for each one of these queries using PMM to ensure expected performance results. We observed a consistent drop across the board for all queries being served by Readyset, with latencies reduced to milliseconds with an order of magnitude increase in throughput.

Once the queries were warm, ClickFunnels migrated from mirror traffic to Readyset to redirect the actual queries to Readyset. From this point onwards, Readyset was responsible for serving the queries.

Results: Achieving Optimal Database Scaling and Performance with Readyset

Once deployed, Readyset significantly reduced database load by offloading queries from read replicas, showcasing exceptional database performance optimization while handling around 5,000 queries per second (QPS) with only 6.6% CPU utilization. 

“It is a testament to Readyset's configurability and ease-of-deployment that ClickFunnels was able to deploy the solution in production without involving application developers. We were up and running in the first week and in production the next. -Javier Zon, Head of Platform, ClickFunnels”
Readyset Cache hit
Cache hit latency

Cache hits were delivered in 30-40µs at the p99 latency, and cache misses (up-query) were resolved in a maximum of 800µs at the p99 latency. Using Readyset, ClickFunnels is able to efficiently handle high query volumes with minimal resource consumption, providing a substantial performance boost to the database infrastructure.

Cache miss latency
Readyset CPU Utilization

ClickFunnels was able to achieve all this in an entirely transparent way without making any changes to the application code.

Interested in Readyset, please reach out to sales@readyset.io or Book a Demo!  

Want to try ClickFunnels? Get started here!