Skip to content

Case: High-Throughput Public API

Problem

Design a public REST API for a SaaS providing weather forecasts. Targets: 100K req/s peak, p99 < 100ms, 99.95% availability, multi-region.

Walkthrough

Clarify

  • Read-heavy (99% reads, 1% writes from internal admin).
  • Forecasts updated every 15 min.
  • Cache TTL ~5 min OK (data freshness vs cost).
  • Public; auth via API key + rate limit.
  • Multi-region (US, EU, APAC).
  • Cost-sensitive at this scale.

Capacity

100K req/s peak * 86400s ≈ 8.6B req/day
Storage: 200K cities * 1 KB forecast = 200 MB; small.
Compute: 100K rps / 1000 rps per instance = 100 instances per region.
Bandwidth: 100K * 5KB = 500 MB/s = 4 Gbps; CDN-served.

Architecture

[Client]
  │ HTTPS
  ▼
[CDN — Cloudflare / Azure Front Door] ← caches public responses
  │  cache miss
  ▼
[Regional Anycast → Container Apps]
  │
  ├──→ [HybridCache: L1 memory + L2 Redis]
  │       │ cache miss
  │       ▼
  └──→ [Read replica: Postgres or Cosmos]

Admin (1% writes):
[Admin] → [Write API → Master DB] → [event] → [Cache invalidation]

API design

GET /v1/weather/{city}
GET /v1/weather/{city}/hourly?days=3
GET /v1/weather/forecast?lat=&lon=

Headers:
  Authorization: Bearer <api-key>
  X-RateLimit-Remaining: 99
  ETag: "abc"
  Cache-Control: public, max-age=300

Errors: ProblemDetails

Caching layers

Browser → 5 min Cache-Control
CDN → 1 min edge cache
HybridCache L1 → 30 sec memory
HybridCache L2 → 5 min Redis
DB → ground truth

Cache hit at edge → no origin call. Massive cost saving.

Code structure

app.MapGet("/v1/weather/{city}",
    async (string city, HybridCache cache, IWeatherRepo repo, CancellationToken ct) =>
    {
        var fc = await cache.GetOrCreateAsync(
            $"weather:v1:{city}",
            (state, ct) => state.repo.GetAsync(state.city, ct),
            (city, repo),
            new HybridCacheEntryOptions { Expiration = TimeSpan.FromMinutes(5) },
            cancellationToken: ct);

        return Results.Ok(fc, cacheControl: "public, max-age=300");
    })
    .CacheOutput(p => p.Expire(TimeSpan.FromMinutes(1)).VaryByValue(c => c.Request.Path))
    .WithName("GetWeather");

builder.Services.AddRateLimiter(o =>
{
    o.AddPolicy("api", c => RateLimitPartition.GetTokenBucketLimiter(
        c.Request.Headers["X-API-Key"].ToString(),
        _ => new TokenBucketRateLimiterOptions { TokenLimit = 100, ReplenishmentPeriod = TimeSpan.FromSeconds(1), TokensPerPeriod = 100 }));
});

app.UseRateLimiter();

Resilience

builder.Services.AddHttpClient<UpstreamWeatherProvider>()
    .AddStandardResilienceHandler(o =>
    {
        o.AttemptTimeout = TimeSpan.FromSeconds(5);
        o.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(15);
        o.Retry.MaxRetryAttempts = 3;
        o.CircuitBreaker.FailureRatio = 0.3;
    });

Observability

builder.Services.AddOpenTelemetry()
    .WithTracing(t => t.AddAspNetCoreInstrumentation().AddHttpClientInstrumentation().AddOtlpExporter())
    .WithMetrics(m => m.AddAspNetCoreInstrumentation().AddRuntimeInstrumentation().AddOtlpExporter());

Custom metrics: cache hit rate, upstream latency, RPS by API key.

Multi-region

  • Active-active.
  • Regional Postgres read replicas (or Cosmos with multi-region writes).
  • Front Door for routing (latency-based).
  • Replicate cache invalidations cross-region (Service Bus topic → regional consumers).

Cost optimization

  • CDN absorbs 90%+ of traffic → only 10% hits origin.
  • Cosmos vs Postgres cost depends on RPS profile; Postgres cheaper at this scale typically.
  • Container Apps scale-to-zero NOT useful (constant traffic).
  • Reserved instances; right-sized.

Failure modes

  • DB outage → serve from cache (stale OK; alarm).
  • Cache outage → fall through to DB; capacity hit; circuit breaker.
  • Region outage → Front Door routes to healthy regions.
  • Bad data: ETag invalidation; cache busting via version key (weather:v2:{city}).

Trade-offs

Choice Why Trade-off
Postgres + replicas Cost-effective at scale Regional write latency
HybridCache L1+L2; stampede protection Two layers complexity
5-min cache TTL Hits 90%+ Stale up to 5 min
API key + rate limit Simple; per-customer Less granular than OAuth
CDN-fronted Massive offload Cache invalidation lag

What we'd skip

  • Kafka: overkill for this read-heavy use case.
  • CQRS: read/write asymmetry not extreme.
  • Saga: no multi-step workflow.
  • Service mesh: a few services; YARP / direct LB enough.

What we'd add for higher scale

  • Read replicas across more regions.
  • Multiple Front Door tiers.
  • Write throttling and queue for the admin write side.
  • Per-tier (free, paid, premium) rate limits.

HTTP version

  • HTTP/2 minimum.
  • HTTP/3 (QUIC) for mobile / poor network.
  • Both supported by ASP.NET Core 9 + Kestrel; CDN handles transparent fallback.

Senior interview signals

  • Mentioning CDN before designing the API code.
  • Right-sized DB (Postgres, not Kafka).
  • Capacity math first.
  • Failure modes explicitly.
  • Trade-offs named.
  • Cost considered.
  • Skipping fancy tech we don't need.

Cross-references