GitHub
Blog
PreviousNext

I Scaled a Simple PostgreSQL UPDATE API to 20,000+ Requests/Second

I Scaled a Simple PostgreSQL UPDATE API to 20,000+ Requests/Second

One weekend, one UPDATE endpoint, and a question I wanted answered - how far can a single PostgreSQL endpoint be pushed? From 5-6K requests/sec to 20K+ by removing bottlenecks one at a time.

This weekend, I didn't want to build another CRUD app.

Instead, I wanted to know one thing: how far can I push a single PostgreSQL UPDATE endpoint?

The starting point

The first version handled only 5-6K requests/sec. Not bad, but I wanted to see where the ceiling really was. So I started removing bottlenecks one by one.

What I changed

  • Optimized the SQL execution path
  • Profiled PostgreSQL wait events
  • Tuned the database
  • Switched to PM2 cluster mode
  • Horizontally sharded the database into 4 PostgreSQL instances

Each step moved the bottleneck somewhere else, which is exactly what makes this kind of exercise worth doing.

The result

After several iterations, the endpoint reached:

  • 20,645 average requests/sec
  • 24,015 peak requests/sec
  • 1 million products
  • Real PostgreSQL UPDATE queries, with no caching

What I learned

Building this taught me a lot about how databases actually work:

  • The life cycle of a SQL query
  • WALs (write-ahead logs) and locks
  • How a database is actually sharded

The code is on GitHub.