Non-Functional Requirements

Don't draw boxes until you know what the system demands. For each NFR this doc covers what it means, how the answer changes your architecture layer by layer, key terms, and which real systems make it their top priority. Pick the ones most relevant to the system and let them drive your design.

Scale

How big is the system, and where does the load actually hit? Scale affects every layer — not just the database.

Ask:

• How many daily active users?
• What's the read/write ratio?
• Any bursty traffic patterns (holidays, events)?

DAU → QPS conversion:

A day has 86,400 seconds. In interviews, round that to 100,000 — the 16% error is irrelevant at estimation scale and the mental math becomes trivial.

CopyQPS = DAU × requests_per_user_per_day ÷ 100,000

Worked example with 1M DAU and 10 requests/user/day:

CopyQPS = 1,000,000 × 10 ÷ 100,000
    = 10,000,000 ÷ 100,000
    = 100 QPS

For peak QPS, multiply by 2–3× (traffic isn't uniform — mornings and evenings are heavier):

CopyPeak QPS = 100 × 3 = 300 QPS

How scale changes tech choices per layer:

• App servers: Single instance → horizontal auto-scaling group → stateless containers (AWS ECS/EKS, GCP Cloud Run/GKE, or self-managed K8s)
• Database: PostgreSQL single → read replicas → sharding → NoSQL or distributed SQL
• Cache: No cache → Redis single → Redis Cluster (partitioned across nodes)
• LB: Not needed → single regional LB (AWS ALB, GCP Cloud Load Balancing) → multi-AZ → global load balancing (AWS Route53 latency routing, GCP global anycast Cloud Load Balancing)
• CDN: Not needed → CDN for static (AWS CloudFront, GCP Cloud CDN, or Cloudflare) → full edge caching with dynamic content

Read/Write ratio shapes your architecture:

• Read-heavy (100:1) → cache aggressively (Redis, CDN), DB read replicas. Twitter feed, Reddit homepage.
• Write-heavy (1:10) → message queues (Kafka) to absorb bursts, append-only logs, async consumers. Consider CQRS (separate write model from read model) to prevent reads from competing with writes. Logging pipeline, analytics ingestion.
• Balanced → general-purpose horizontal scaling.

Burst traffic: If traffic spikes at predictable times (Black Friday, live events), design for auto-scaling and queue-based buffering, not steady-state peak capacity.

Storage estimate: DAU × avg_event_size × events_per_day × retention_days

Most critical for: Twitter/X (read-heavy feed), YouTube (video storage + CDN), Uber (surge traffic), ticketing systems (flash sales).

Latency

How fast must the system respond? This determines where you place compute, what stays synchronous, and what you offload.

Ask:

• What's the acceptable p99 response time?
• Are there specific operations that must be fast?

What is p99? Sort all requests by response time. P99 is the 99th one — 99% of requests completed faster than this. P50 is the median. P99.9 is 1 in 1,000. At 1,000 QPS, p99 means 10 requests every second are slower than that number.

Why p99 and not average — average hides pain: 99 requests complete in 50ms. One hangs for 10,000ms.

CopyAverage = (99×50 + 10,000) / 100 = 149ms  ← looks healthy
P99     = 10,000ms                          ← system is on fire

At Walmart's scale — 260 million customers a week — 1% is 2.6 million people experiencing that hang. Average would never surface it.

Why p99 specifically:

P99 is the SLA standard because it catches real problems early without false-alarming on one-off outliers.

P99 spike = early overload warning: When a system gets busy, it fails at the tail first — not uniformly.

By the time P50 degrades, you're already in serious trouble. P99 gives you the window to act — shed load, scale out, open a circuit — before the majority of users feel it.

Setting the threshold — from your SLA, not arbitrary: Set the shedding trigger at a fraction of your SLA, not after you've already breached it.

CopyInventory reservation SLA = 500ms → shed when p99 > 400ms  (80% of SLA)
Homepage SLA = 2,000ms           → shed when p99 > 1,600ms (80% of SLA)

The gap between threshold and SLA is your response window. Trigger too late and you're already breaking the promise.

How it's measured: Rolling window over the last N seconds (typically 10s), recomputed every 5 seconds. In practice, use Micrometer or Prometheus histograms rather than sorting raw request lists — same concept, far more efficient.

Latency vs throughput: Latency is how fast one request completes. Throughput is how many requests per second the system handles. Optimizing for one can hurt the other — batching increases throughput but adds per-request latency. Know which the interviewer cares about.

p99 matters more than average. If the interviewer says "users complain it feels slow", think tail latency, not mean. One unindexed query or missing cache can blow your p99 while your average looks fine.

Per-component latency costs: For exact numbers per hop (LB, Redis, DB, Kafka, S3, etc.) and worked end-to-end request path examples, see the Latency Reference Table in System Design Layers.

Most critical for: Search/autocomplete (Yelp, Google — < 100ms), stock trading (HFT — microseconds), multiplayer gaming, ride-matching (Uber — driver must get request fast).

Availability

How much downtime is acceptable? This drives redundancy, replication topology, and failover strategy across all layers.

Ask:

• What's the uptime requirement?
• What happens to users if this goes down?

How each layer achieves redundancy:

• Load Balancer: Health checks drop unhealthy app servers from rotation in seconds. Multi-AZ deployment so one AZ outage doesn't take the LB down.
• App Servers: Stateless (no local state) so any instance can handle any request. Auto-scaling group replaces failed instances automatically.
• Cache (Redis): Redis Sentinel (monitors, auto-promotes replica on primary failure). Redis Cluster (shards + replicas, handles node loss).
• Database: Primary + read replica → automatic failover on primary crash. Multi-region → async or sync replication depending on consistency needs.
• Message Queue: Kafka replication factor ≥ 3 means 2 broker deaths don't lose messages.
• CDN: CDN providers are globally redundant by design (AWS CloudFront, GCP Cloud CDN, Cloudflare).

CAP Theorem tradeoff: During a network partition, choose availability (keep serving, possibly stale) or consistency (stop serving until consistent). Most consumer apps choose availability. Payment systems choose consistency.

Graceful degradation: Netflix shows cached thumbnails when recommendations service is down. Don't fail completely — fail partially.

SLI / SLO / SLA — know the difference:

• SLI (Service Level Indicator) — the actual measured metric. E.g., "97.8% of requests completed in < 200ms this week."
• SLO (Service Level Objective) — your internal target. E.g., "99.9% of requests must complete in < 200ms." This is what your team is held to.
• SLA (Service Level Agreement) — the contractual commitment to customers, with financial penalties for breach. Always looser than your SLO (you'd go out of business otherwise).

Error budget: 1 − SLO. A 99.9% SLO gives you ~43 min/month to spend on incidents and deploys. When the budget is gone, freeze non-critical changes until the window resets.

Each extra 9 is roughly 10× harder and more expensive. Push back if the requirement seems over-engineered.

Most critical for: Payment processors (Stripe, Visa — 99.999%), AWS infrastructure, healthcare systems, any system where downtime = revenue loss or safety risk.

Consistency

When a write happens, when do all nodes and users see it? This is the core CAP tradeoff in practice.

Ask:

• Can users see slightly stale data?
• If two users write at the same time, does it matter which one wins?

ACID vs BASE: Relational databases give you ACID (Atomic, Consistent, Isolated, Durable) — all or nothing, always correct. Most NoSQL databases give you BASE (Basically Available, Soft state, Eventually consistent) — always up, eventually right. Choosing a DB is often choosing between these two philosophies.

Idempotency: A write operation is idempotent if calling it multiple times produces the same result as calling it once. Critical when clients retry on network failure — without it, a retried payment charges the user twice.

Causal consistency explained: If Alice posts "I'm going to the store" and Bob replies "I'll come with you", causal consistency guarantees Carol always sees Alice's post before Bob's reply — because Bob's reply causally depends on Alice's post. But Carol might see Alice's post before or after Dave's unrelated status update — that's fine, they're not causally linked.

This is stronger than eventual (which could show Bob's reply before Alice's post) but weaker than strong (which globally orders every single write). It's the right choice when order matters within a thread or conversation, but not globally.

Interview signal: "It's fine if the like count is off by a few seconds" → eventual consistency, scale horizontally. "Double-charging a user is unacceptable" → strong consistency, accept the latency cost.

Most critical for: Banking and payments (double-spend prevention), inventory systems (Amazon — can't oversell), booking systems (airline seats, hotel rooms).

Idempotency

If a client retries a request, will it cause duplicate side effects? This shapes how you design APIs and payment flows.

Ask:

• Can clients retry failed requests safely?
• Are there operations where duplicates are catastrophic (charges, transfers, order submissions)?

The problem: A client sends a payment request. The server processes it, but the response is lost in transit. The client retries. Without idempotency, the user gets charged twice.

The solution — idempotency keys: Client generates a unique key per logical operation (e.g., UUID) and sends it with the request. Server stores (idempotency_key → result) in Redis or DB. On duplicate request: return the stored result, skip re-execution.

Idempotency in queues: A Kafka consumer that crashes mid-processing will re-receive the same message. Design consumers to be idempotent — check if the event was already processed (by storing the event ID) before acting on it.

Most critical for: Payment APIs (Stripe uses idempotency keys on every charge endpoint), order submission, booking systems, any POST that creates or transfers.

Durability

How much data loss is acceptable if the system crashes or a node goes down?

Ask:

• If we lose a server right now, what's the worst acceptable outcome?
• Can we replay events from a log?

RPO = Recovery Point Objective — how much data can we lose? Measured in time (0ms, 1s, 1hr means you lose up to that much data). RTO = Recovery Time Objective — how long can the system be down during recovery?

RPO=0 adds latency at every layer that writes:

• DB: Synchronous replication adds 5–20ms (waiting for replica in another AZ to confirm).
• App layer: If using distributed transactions (two-phase commit), add 10–50ms.
• Queue: Kafka with acks=all (wait for all in-sync replicas) adds 2–5ms vs acks=1.

This is the direct tradeoff: stronger durability = higher write latency.

Real examples:

• Banking: RPO = 0. Every transaction written synchronously to multiple replicas before confirmation.
• Social media posts: RPO = seconds is fine. Async replication acceptable.
• Object storage: 11 nines of durability via cross-AZ redundant storage (AWS S3, GCP Cloud Storage).

Most critical for: Banking and financial systems, medical records (Epic, FHIR), legal document storage, payment transaction logs — any system where lost data = legal or financial liability.

Fault Tolerance

How well does the system handle partial failures without going fully down?

Ask:

• What happens when one server crashes?
• What happens when a whole datacenter goes down?
• What if a dependency is slow or unavailable?

Circuit breaker pattern: If a downstream service fails N times in a row, stop calling it for a period. Return a cached/default response. Let the dependency recover before retrying.

Retry with exponential backoff + jitter: When a request fails, wait before retrying — and double the wait each attempt (backoff). Add random jitter so all retrying clients don't slam the service at the same moment (thundering herd). A common sequence: retry after 1s, 2s, 4s, 8s with ±30% jitter, then give up and dead-letter.

Dead Letter Queue (DLQ): When a message fails processing repeatedly (after N retries), route it to a DLQ instead of blocking the queue. The DLQ holds poisoned messages for inspection and manual replay. Without a DLQ, one bad message can stall an entire consumer group indefinitely. (AWS SQS dead-letter queues, GCP Pub/Sub dead-letter topics, or a separate Kafka topic).

Most critical for: Microservices architectures (each service can fail independently), distributed databases, any system with SLA > 99.9%.

Security

What data does the system handle and who should access it? Drives auth, encryption, and regulatory design.

Ask:

• Does this handle PII, payments, or health data?
• Who are the users — public, internal, B2B?

Key terms:

• PII (Personally Identifiable Information) — any data that can identify a person: name, email, phone, SSN, IP address. Triggers GDPR/HIPAA obligations.
• TLS (Transport Layer Security) — encrypts data in transit (the "S" in HTTPS). Prevents interception.
• AES-256 (Advanced Encryption Standard) — standard algorithm for encrypting data at rest. Used in S3, databases, filesystems.
• JWT (JSON Web Token) — signed token the client sends with each request to prove identity. Stateless, server doesn't store sessions.
• OAuth2 — standard for delegated auth. "Sign in with Google" is OAuth2. Separates identity from your app.
• mTLS (Mutual TLS) — both sides verify certificates. Used for service-to-service auth inside your system.
• RBAC (Role-Based Access Control) — users get roles (admin, editor, viewer), roles get permissions. Simpler than per-user rules.
• ACL (Access Control List) — per-resource list of who can do what. More granular than RBAC (e.g. S3 bucket policies).

Security by layer:

Most critical for: Healthcare (HIPAA — audit every access to patient records), financial systems (PCI-DSS — card data tokenized immediately), auth systems (OAuth provider), any multi-tenant SaaS.

Compliance

Are there legal or regulatory constraints that shape the architecture?

Ask:

• What region are users in?
• Does this handle health, financial, or personal data?

Key terms:

• GDPR (General Data Protection Regulation) — EU law. Applies to any system with EU users, regardless of where the company is located.
• HIPAA (Health Insurance Portability and Accountability Act) — US law governing health data. Applies to any app handling patient records.
• PCI-DSS (Payment Card Industry Data Security Standard) — required for any system that stores, processes, or transmits card data.
• SOC 2 — US auditing standard for SaaS companies. Type I = point-in-time assessment. Type II = 6 months of continuous evidence. Required by enterprise buyers.

GDPR complicates event-sourcing: Append-only logs make "right to delete" hard — you can't erase a past event. Solve with tombstone records or keep PII in a separate deletable store and only store user IDs in the event log.

Most critical for: Healthcare apps, payment processors, social platforms with EU users, any enterprise B2B SaaS sold to regulated industries.

Monitoring & Observability

How do you know the system is healthy in production? Drives logging, metrics, and alerting design.

Ask:

• Do you need real-time alerting?
• How quickly must the team detect and diagnose production issues?

The four golden signals (Google SRE): Latency, Traffic, Errors, Saturation. Build monitoring around these first.

• Latency — how long requests take (track p99, not average)
• Traffic — how much load the system is under (QPS, requests/sec)
• Errors — rate of failed requests (5xx errors, timeouts, exceptions)
• Saturation — how "full" a resource is. A CPU at 95% is saturated. A disk at 98% capacity is saturated. Saturation predicts future failure — a resource approaching 100% will soon become a bottleneck and cause latency spikes or crashes. Monitor: CPU %, memory %, disk I/O utilization, DB connection pool usage, queue depth.

Most critical for: Any system with a strict SLA, microservices (failures are hard to trace), Netflix-style chaos engineering, financial systems where bugs cost real money.

Environment Constraints

Are there non-standard constraints on the environment the system runs in?

Ask:

• Are clients on mobile or constrained devices?
• Are there low-bandwidth or offline scenarios to handle?

Most critical for: Uber driver app (poor network in some cities), Google Maps offline, WhatsApp (works on 2G), IoT sensor pipelines, healthcare apps in hospitals with restricted networks.

Quick Reference — Which NFR Matters Most