Congratulations — you've made the entire journey: from a single task-tracker-api instance with SQLite, to three instances behind a load balancer, with Postgres, Redis, and full observability. In this closing article, we'll round up the common mistakes beginners make (and sometimes experienced teams too), a debugging checklist for a slow system, and where you can go from here.
Common Mistakes
Scaling without measuring first. This is the theme we've repeated throughout the series — the most expensive mistake is adding complexity (a cache, an instance, a new database) based on a guess. We ourselves nearly drew the wrong conclusion in Part 7, if we hadn't checked docker stats and found the bottleneck was actually in our own measuring tool.
Adding instances without making sure the app is genuinely stateless. If important state is stored in one instance's memory (a session, a local cache, a temporary uploaded file), horizontal scaling produces weird, hard-to-reproduce bugs — appearing randomly depending on which instance happens to answer.
Marking something async def but calling blocking code inside it. Covered in full in Part 6 — this is worse than staying with plain def, because it freezes the entire event loop, not just one thread.
Caching without a clear invalidation strategy. TTL alone isn't an invalidation strategy — it's just a safety net. Without explicit invalidation when data changes (Part 4), users will see stale data, and bugs like this often only surface after a user complains.
Indexing carelessly, or not indexing at all. Without an index on frequently filtered columns, queries scale linearly with table size (Part 5) — a problem that often only becomes seriously painful once the data is already large, and it's already too late for a quick fix. On the flip side, indexing every column isn't a solution either — there's a trade-off against write speed and storage.
High-cardinality metrics. Using a raw path/ID as a metric label (Part 8) can explode the number of time series and make the monitoring system itself slow or expensive.
Never testing failure scenarios. We built a load balancer so there'd be no single point of failure at the application layer — but if it's never actually been tested (kill one instance, see if traffic keeps flowing), that's an unproven claim, just a design assumption.
Debugging Checklist for a Slow Production System
If your system slows down and you're not sure where to start, follow the order we've practiced throughout this series:
Check the metrics first (Part 8) — which endpoint's p95 is rising? Are 5xx status codes up? Is one particular instance far busier than the others?
Check resource usage on EVERY party involved, including the measuring tool/client if you're load testing (Part 7) — don't assume the bottleneck must be the server.
Read the structured logs for the specific slow/failed requests, using request_id to trace them.
Check whether this is an I/O or a CPU problem (Part 1) — the solutions are completely different.
Check the connection pool (database, and if there is one, the HTTP client to another service) — repeated timeout errors usually signal the pool is out of slots (Part 5).
Check the thread pool if the endpoint is plain def and concurrency is high (Part 6) — an easy-to-miss bottleneck because it doesn't show up as an explicit error.
Check the query plan (EXPLAIN ANALYZE) for any query suspected of being slow — look for a Seq Scan that should be an Index Scan.
Roadmap — What's Next
This series deliberately focused on the fundamentals: caching, the database, concurrency, horizontal scaling, observability. If you want to go further, here are a few directions:
Message queues (RabbitMQ, SQS, Kafka) — for work that doesn't need to finish before the response is sent (sending an email, generating a large report), moved to a background worker via a queue, so the API endpoint stays responsive.
Read replicas & database sharding — once a single Postgres instance itself becomes the bottleneck (no longer about pool size or indexes), the next step is splitting read load off to a replica, or splitting data across multiple databases (sharding) for far larger scale.
A fully async database driver (asyncpg, SQLAlchemy async) — migrate task-tracker-api to be genuinely async def end to end, instead of a blocking endpoint with an enlarged thread pool as the "band-aid" from Part 6.
Real Prometheus + Grafana — wiring up the /metrics endpoint we already built to an actual Prometheus server and Grafana dashboard, plus automatic alerting when a metric crosses a threshold.
Rate limiting & circuit breakers — protecting the system from excessive load (whether from an abusive user or a client-side bug), and preventing one broken service from dragging the entire system down with it.
Container orchestration (Kubernetes) — once Docker Compose (a good fit for learning and a single machine) isn't enough anymore, Kubernetes provides auto-scaling, self-healing, and more sophisticated deployment strategies across many machines at once.
A CDN for static assets — if the app has static assets (images, files, a frontend build), a CDN absorbs that load long before it ever reaches the application server.
Wiring it into a CI/CD pipeline — if you haven't already, the CI/CD for Beginners series covers automating tests and deployment, so changes like the ones we made throughout this series (a new index, pool size, application code) can be deployed safely and consistently.
How This Connects to Other Series
This series continues directly from task-tracker-api in FastAPI for Beginners — if anything felt unfamiliar (path operations, dependency injection, Pydantic models), it's likely a fundamental covered there. And once your API is scalable, automating its build-test-deploy pipeline lives in CI/CD for Beginners — all three series are designed to complement each other, following one continuous journey from zero to production-ready.
Closing Thoughts
We've made the journey from "why applications stop scaling," through understanding a scalable system's anatomy, measuring a baseline, caching, database scaling, concurrency within a single process, horizontal scaling with Docker and a load balancer, all the way to observability for knowing when to scale again. Most importantly, every claim in this series is backed by real numbers we measured ourselves — including the times those numbers were surprising or didn't match our initial expectations, because that's the real heart of scalability: decisions based on data, not guesses.
Thanks for following along with this Scalability for Beginners series. Good luck applying it to your own systems — and always measure first, then optimize.