Caching, queues & scaling

What caching strategies exist, and what breaks with each?

• Cache-aside (lazy loading) — the app reads cache, misses, reads the DB, writes the cache. Simple and the default. Risk: stampedes on a cold key, and the cache can go stale after a write unless you invalidate.
• Read-through / write-through — the cache layer owns loading/writing. Consistent, but every write pays cache latency.
• Write-behind — write to cache, flush to DB asynchronously. Fast, but data loss if the cache dies before flushing.
• Refresh-ahead — proactively refresh hot keys before expiry. Great for predictable hot sets, wasteful otherwise.

The invalidation rule of thumb: delete, do not update on write. Updating races with concurrent readers; deletion just forces the next reader to reload. And set a TTL on everything, even keys you invalidate explicitly — the TTL is your bug backstop.

Explain cache stampede, penetration and avalanche, and how to prevent each.

• Stampede / dogpile — a hot key expires and 10,000 requests all miss and hit the DB at once. Fix: a per-key mutex/lease (first miss wins the right to recompute, the rest wait or serve stale), or probabilistic early expiration — refresh with probability rising as the TTL nears its end.
• Penetration — requests for keys that do not exist bypass the cache entirely (often malicious). Fix: cache the negative result with a short TTL, and/or a Bloom filter of existing keys.
• Avalanche — many keys expire simultaneously (they were all populated at deploy). Fix: jitter the TTL (ttl = base + random(0, base * 0.1)).

def get_user(uid):
    key = f"user:{uid}"
    if (hit := cache.get(key)) is not None:
        return None if hit == SENTINEL_MISSING else hit
    if not cache.set(f"lock:{key}", 1, nx=True, ex=5):   # someone else is loading
        time.sleep(0.05); return get_user(uid)
    try:
        row = db.fetch_user(uid)
        cache.set(key, row if row else SENTINEL_MISSING,
                  ex=(300 + random.randint(0, 30)) if row else 30)
        return row
    finally:
        cache.delete(f"lock:{key}")

What do you cache, and where?

Layers, from the user inwards: browser cache → CDN/edge → reverse proxy → application in-process cache (per pod, sub-microsecond, but inconsistent across pods) → shared cache (Redis/Memcached) → database buffer pool/materialised views.

Cache what is expensive and read far more often than written: rendered fragments, aggregate counts, permission lookups, third-party API responses, session data. Do not cache what you cannot invalidate correctly, or anything where staleness is a correctness bug (a balance, an inventory count at checkout) — for those, cache the read model and verify at commit time.

Redis: what data structures do you actually use, and how does it persist data?

Strings (counters, cached blobs, SETNX locks), hashes (objects, partial updates), lists (simple queues, BLPOP), sets (unique membership, tags), sorted sets (leaderboards, rate limiters, delayed queues by score-as-timestamp), streams (durable append log with consumer groups — the right choice over lists for real queues), HyperLogLog (approximate uniques in 12 KB), bitmaps (daily active flags).

Persistence: RDB point-in-time snapshots (compact, fast restart, loses the window since the last snapshot) and AOF append-only log (everysec by default → ≤1s loss; always is durable but slow). Most production setups run both. The important caveat: Redis is not a database of record. Replication is asynchronous, so a failover can lose recent writes — never store data you cannot rebuild.

Explain distributed locking. Why is Redlock controversial?

A naive SETNX lock with a TTL fails when the holder pauses (GC, VM steal) past the TTL: the lock expires, another worker takes it, and now two workers act at once. Mitigations: store a unique token and delete only if it matches (a Lua compare-and-delete), keep the critical section short, and extend the lease with a watchdog.

Redlock (locking across N independent Redis nodes) is contested because it assumes bounded clock drift and bounded pauses; Martin Kleppmann's critique is that no lock with a timeout is safe for correctness without fencing tokens — a monotonically increasing number the resource itself checks, rejecting writes from an older token.

The practical senior answer: for efficiency (avoid duplicate work) a Redis lock is fine. For correctness (never double-charge) do not use a lock at all — use a unique constraint, a conditional update (UPDATE … WHERE version = $n), or a transactional outbox.

When do you use a message queue, and what does it change?

Use one to decouple producers from consumers, absorb bursts, retry failures without blocking the user, and fan out one event to many consumers. What it changes: your system becomes eventually consistent, you must handle duplicates, ordering is no longer free, and you now have a queue to monitor (depth, age of oldest message, DLQ size).

Delivery semantics: at-most-once (fire and forget, can lose), at-least-once (the practical default — you will get duplicates), exactly-once (not really achievable end-to-end; achievable effectively by making consumers idempotent). Say this: "exactly-once delivery is a myth; exactly-once processing is idempotent consumers plus at-least-once delivery."

Kafka vs RabbitMQ vs SQS — how do you choose?

• Kafka — a partitioned, replayable log. Ordering within a partition, consumers track offsets, retention lets you re-process history. For event streaming, analytics pipelines, event sourcing, high throughput. Costs: operational weight, partition-count decisions that are hard to change, no per-message ack/redelivery semantics.
• RabbitMQ — a broker with rich routing (exchanges, topics), per-message ack, priorities, delayed messages. For task distribution and complex routing. Costs: throughput ceiling is lower; queues that back up hurt.
• SQS/managed — nearly zero ops, at-least-once (or FIFO with lower throughput), visibility timeouts and DLQs built in. Costs: no replay, limited routing, per-message price.

Pick by the question "do I need to replay history?" (Kafka), "do I need complex routing and per-message control?" (Rabbit), or "do I mostly want it to not be my problem?" (SQS).

How do you handle a poison message and a failing consumer?

Bounded retries with exponential backoff and jitter, then route to a dead-letter queue with the original message, the error and the attempt count. Alert on DLQ depth — a silent DLQ is a silent data-loss bug. Make consumers idempotent so retries are safe. Distinguish retryable (timeout, 503, deadlock) from non-retryable (validation failure, 404) and DLQ the second kind immediately instead of burning 10 attempts. Keep a runbook for replaying a DLQ after the fix ships.

What is backpressure, and what happens without it?

Backpressure is the signal that a consumer cannot keep up, propagated back to the producer. Without it, queues grow unbounded, memory fills, latency climbs, and the system fails all at once instead of degrading. Mechanisms: bounded queues that block or reject, 429/503 with Retry-After, TCP flow control, concurrency limits per dependency, and load shedding — dropping low-priority work to protect the core. Related patterns: the circuit breaker (stop calling a failing dependency, fail fast, probe periodically) and bulkheads (separate connection pools per dependency so one slow service cannot consume every thread).

How do you keep a cache and a database consistent when writing?

The honest answer: you cannot make them perfectly consistent without distributed transactions, so you choose an acceptable staleness. Practical patterns, in order of robustness:

1. Write DB, then delete cache key. Simple; a small race window exists where a concurrent reader repopulates stale data. Mitigate with short TTLs and, if it matters, a delayed second delete.
2. Transactional outbox + CDC. Write the row and an outbox record in one transaction; a consumer (Debezium/log tailing) invalidates caches and publishes events. This is the correct answer when cache/search-index/downstream consistency actually matters.
3. Versioned keys. Include a version or updated_at in the cache key so a write naturally makes old entries unreachable — no invalidation at all, just garbage that expires.

The anti-pattern to name: writing to the cache and the DB in application code as if they were one transaction. A crash between them leaves you inconsistent with no way to detect it.