AWS System Design

Drop into .claude/skills/ or .cursor/skills/ — the frontmatter drives when the skill loads.

---
name: aws-system-design
description: >-
  AWS system design for scaling: vertical instance sizing, horizontal
  scaling behind a load balancer, then caching, indexing, replication, and
  sharding. Use when choosing how a service should scale or reviewing a
  scaling design.
argument-hint: "[workload or tier]"
license: MIT
metadata:
  author: John Felix Lim
  version: "1.0.0"
---
# AWS System Design

Apply when deciding how a service should scale or reviewing that design. Start from the measured bottleneck. Prefer a stateless fleet behind a load balancer once one instance is the risk, and add cache, indexes, replicas, and shards only for the limit each one actually removes.

## Rules

1. **Scale the bottleneck you measured** — Adding instances, cache nodes, or shards before you know which tier is saturated spends money on the wrong layer. CloudWatch (CPU, latency, connection count, replica lag) tells you whether the limit is one machine, the app tier, or the database.
2. **Vertical scaling buys headroom and a single point of failure** — A larger instance (more vCPU and memory) is the fastest fix when one machine still fits the workload. Resizing usually means downtime, and when that instance fails the service fails with it. Treat it as a step, not the end state, once traffic keeps growing.
3. **Horizontal scaling only works when the app tier is stateless** — A load balancer can spread requests across identical instances only if any instance can serve any request. Sessions, uploads, and in-memory caches that live on one box make the next instance useless. Put shared state in a store the whole fleet can reach.
4. **A cache is a shortcut, not the source of truth** — ElastiCache (or an equivalent) absorbs repeated reads so the database does not. A hit returns immediately. A miss, an expiry, or a flush must still succeed against the database. Cache the hottest keys, set a TTL, and invalidate on writes that change those keys.
5. **Indexes, replicas, and shards solve different database limits** — An index makes a query cheap. A read replica scales reads and adds lag. Sharding scales writes by splitting rows on a shard key, and it makes cross-shard queries and rebalancing your problem. Add them in that order: a missing index is cheaper to fix than a shard you did not need.

## Patterns

### One growing instance instead of a stateless fleet

**Avoid** — Every request lands on one machine. The next resize needs downtime, and a failure takes the service down with it.

```hcl
resource "aws_instance" "app" {
  ami           = var.ami
  instance_type = "m5.2xlarge" # resized again each time CPU pins
}
```

**Prefer** — The load balancer spreads traffic across identical instances. Capacity changes by adding or removing boxes, and one failure does not take the fleet down.

```hcl
resource "aws_lb" "app" {
  load_balancer_type = "application"
  subnets            = var.public_subnet_ids
}

resource "aws_autoscaling_group" "app" {
  min_size         = 2
  max_size         = 8
  desired_capacity = 2
  target_group_arns = [aws_lb_target_group.app.arn]
}
```

### Reads that scan, with no cache in front

**Avoid** — Each request table-scans. More app instances make the database worse, because horizontal scale multiplied an unindexed read.

```sql
-- customer lookup on every request, no index, no cache
SELECT * FROM orders WHERE customer_id = $1;
```

**Prefer** — The index makes the miss cheap. Repeated reads stop at the cache, and replicas take read load off the primary. Writes still go to the primary, or to the shard that owns that key.

```sql
CREATE INDEX orders_customer_id_idx ON orders (customer_id);

-- cache key: orders:customer:{id}, TTL measured against write frequency
-- on a miss the app runs this query (on a replica when lag is acceptable)
-- and SETs the key itself; ElastiCache never reads the database
```

## Review checklist

- [ ] The saturated tier is identified from metrics (CPU, latency, connections, replica lag) before instance size, cache, or shards change.
- [ ] A vertically scaled instance has a documented resize window and is not the only copy of the service.
- [ ] App instances behind the load balancer share no local session, file, or cache state.
- [ ] Cached keys have a TTL and an invalidation path on the writes that change them.
- [ ] Hot read queries use an index, and read replicas are used only where replication lag is acceptable.
- [ ] A shard key spreads writes evenly, and any query that must cross shards is called out explicitly.

---

From the John Felix Lim Frontend Engineering Playbook — https://github.com/JohnFelixLim