From One Server to Many

Every scaled system is the same story told in a different order: the server gets slow, you add the next thing, the new thing adds a new bottleneck, you repeat. The interesting question isn't "which component" — it's "which bottleneck are we solving now?"

The progression

One server. App + database on one box. Correct, cheap, boring — until it isn't. The first bottleneck is usually CPU or memory; the fix is usually "buy a bigger box" (vertical scaling), which works until it doesn't.

Split the database. App on one box, Postgres on another. Now the app server can die and the data survives. The trade-off: a network hop between app and data — latency goes up, but availability improves.

Load balancer + N app servers. The app is stateless (sessions in Redis or JWTs, uploads in object storage), so any server can handle any request. Horizontal scaling unlocked — add capacity by adding servers. This is the first real architecture decision.

Cache the hot reads. Most reads are repeated — the same user profile, the same product page, the same feed. Redis/Memcached in front of the database turns a 100ms query into a 1ms lookup for the hot 80%.

Queue the async work. Sending email, resizing images, processing webhooks — none of these need to happen in the request path. A queue (SQS, RabbitMQ, Postgres's own FOR UPDATE SKIP LOCKED) buffers spikes and decouples the writer from the worker.

What each step costs

• LB: a new component to configure, health checks to get right.
• Cache: invalidation bugs, stale data, cache stampedes.
• Queue: eventual consistency — "the job ran" isn't synchronous with "the request returned."
• Replicas: replication lag — a read-your-own-write guarantee needs care.

Field notes

• "Scale" isn't one thing — reads scale with cache + replicas, writes scale with queues + sharding. Know which you're solving.
• The components are commodity; the skill is knowing which problem you're solving and which new problem you're accepting.