Rate Limiting

Key Points

• Built-in Microsoft.AspNetCore.RateLimiting middleware (.NET 7+) with four algorithms: fixed window, sliding window, token bucket, concurrency.
• Partition by key (per-IP, per-tenant, per-API-key) via RateLimitPartition.GetXxxLimiter.
• Apply to specific endpoints with RequireRateLimiting("policy") or [EnableRateLimiting].
• Returns 429 Too Many Requests with optional Retry-After header.
• For multi-instance services, distributed rate limiting requires Redis-backed implementation (community packages — built-in is per-instance).
• Output rate-limiting headers (X-RateLimit-Limit, X-RateLimit-Remaining) help clients self-throttle.

Concepts (deep dive)

Algorithms

Fixed window
  ┌────10 req/min ────┬────10 req/min ────┐
  0                  60                 120 sec
  Counter resets at boundary; spike at boundary possible.

Sliding window
  Approximates rolling 60-sec window via N segments. Smoother than fixed.

Token bucket
  Bucket fills at rate R; each request takes a token. Allows bursts up to bucket size.

Concurrency limiter
  At most N concurrent in-flight requests. Pure "no more than N at once".

Setup

builder.Services.AddRateLimiter(o =>
{
    o.RejectionStatusCode = 429;

    // Global default policy
    o.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(ctx =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: ctx.Connection.RemoteIpAddress?.ToString() ?? "anon",
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 100,
                Window = TimeSpan.FromMinutes(1)
            }));

    // Named policies
    o.AddPolicy("api", ctx =>
        RateLimitPartition.GetTokenBucketLimiter(
            partitionKey: ctx.User.Identity?.Name ?? ctx.Connection.RemoteIpAddress!.ToString(),
            factory: _ => new TokenBucketRateLimiterOptions
            {
                TokenLimit = 20,
                ReplenishmentPeriod = TimeSpan.FromSeconds(1),
                TokensPerPeriod = 5,
                AutoReplenishment = true
            }));

    o.AddPolicy("tier-pro", ctx => /* per-tier policy */);

    // Custom rejection
    o.OnRejected = async (ctx, ct) =>
    {
        if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retry))
            ctx.HttpContext.Response.Headers.RetryAfter = retry.TotalSeconds.ToString("F0");
        await ctx.HttpContext.Response.WriteAsync("Too many requests; retry later.", ct);
    };
});

app.UseRateLimiter();

app.MapGet("/api/items", GetItems).RequireRateLimiting("api");

Per-tenant / per-API-key partitioning

o.AddPolicy("per-key", ctx =>
{
    var apiKey = ctx.Request.Headers["X-API-Key"].FirstOrDefault();
    if (apiKey is null)
        return RateLimitPartition.GetNoLimiter("anon");   // no limit for unkeyed (or block earlier)

    return RateLimitPartition.GetTokenBucketLimiter(apiKey,
        _ => new TokenBucketRateLimiterOptions
        {
            TokenLimit = 1000,
            ReplenishmentPeriod = TimeSpan.FromSeconds(1),
            TokensPerPeriod = 100
        });
});

Concurrency limiter for expensive endpoints

o.AddPolicy("expensive", ctx =>
    RateLimitPartition.GetConcurrencyLimiter("global",
        _ => new ConcurrencyLimiterOptions
        {
            PermitLimit = 5,
            QueueProcessingOrder = QueueProcessingOrder.OldestFirst,
            QueueLimit = 20
        }));

app.MapPost("/heavy-job", DoHeavy).RequireRateLimiting("expensive");

Useful when CPU/memory are the limit, not request rate.

Disabled per-endpoint

app.MapGet("/health", () => "OK").DisableRateLimiting();

Outbound rate-limit headers

The middleware doesn't set rate-limit headers by default. Add manually or via OnRejected:

o.OnRejected = async (ctx, ct) =>
{
    ctx.HttpContext.Response.StatusCode = 429;
    if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retry))
    {
        ctx.HttpContext.Response.Headers.RetryAfter = retry.TotalSeconds.ToString("F0");
        ctx.HttpContext.Response.Headers["X-RateLimit-Reset"] =
            DateTimeOffset.UtcNow.Add(retry).ToUnixTimeSeconds().ToString();
    }
    await ctx.HttpContext.Response.WriteAsync("Rate limited", ct);
};

For per-request remaining count, you'd need to read Lease.GetMetadata after acquiring (more involved).

Distributed rate limiting

Built-in is per-instance — N replicas mean N times the limit. Solutions:

1. Redis-backed (community: RateLimiting.Redis, etc.).
2. API gateway (APIM, Kong, Cloudflare) — enforce at the edge.
3. CDN-level (Cloudflare rate-limiting) — even further out.

For most apps, edge enforcement (API gateway / WAF) is more accurate than per-replica .NET limits. Use .NET's built-in for backstop protection per replica.

Queue vs reject

new TokenBucketRateLimiterOptions
{
    QueueLimit = 10,                  // queue up to 10 requests when bucket empty
    QueueProcessingOrder = QueueProcessingOrder.OldestFirst
}

QueueLimit > 0 means: when limit hit, queue the request and wait. Latency increases for queued requests but no immediate 429.

QueueLimit = 0 means: reject immediately when over limit.

Reject status code

Default 503 (Service Unavailable). Most APIs prefer 429 Too Many Requests:

o.RejectionStatusCode = 429;

Code: correct vs wrong

❌ Wrong: rate-limiting at the application but not at the edge

// Single replica enforces 100 RPS; with 5 replicas, total is 500 RPS — not what you wanted

✅ Correct: enforce at edge AND in-app

Edge (APIM / Cloudflare) for accurate rate limiting; in-app as backstop.

❌ Wrong: partitioning by IP behind proxy

ctx => RateLimitPartition.GetFixedWindowLimiter(
    ctx.Connection.RemoteIpAddress!.ToString(),
    /* ... */);

If proxy is in front, all requests have the same IP. Combine with UseForwardedHeaders first.

❌ Wrong: forgetting Retry-After

Client doesn't know when to retry. Add via OnRejected.

✅ Correct

o.OnRejected = async (ctx, ct) =>
{
    if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retry))
        ctx.HttpContext.Response.Headers.RetryAfter = retry.TotalSeconds.ToString("F0");
    /* ... */
};

Design patterns for this topic

Pattern 1 — "Token bucket for typical APIs"

• Intent: allow short bursts; sustain rate over time.

Pattern 2 — "Concurrency limiter for expensive endpoints"

• Intent: protect resources, not request count.

Pattern 3 — "Per-tenant partitioning via API key"

• Intent: fair share across tenants.

Pattern 4 — "Edge + in-app rate limiting"

• Intent: accurate at edge; backstop in-app.

Pattern 5 — "OnRejected to set Retry-After"

• Intent: client-friendly response.

Pros & cons / trade-offs

When to use / when to avoid

• Use built-in for in-process rate-limiting backstop.
• Use API gateway for accurate cross-replica limits.
• Avoid IP partitioning without ForwardedHeaders.
• Avoid relying on in-process for total rate across replicas.

Interview Q&A

Q1. Four built-in algorithms? Fixed window, sliding window, token bucket, concurrency.

Q2. When token bucket vs fixed window? Token bucket allows bursts up to bucket size; fixed window has hard boundary spikes. Bucket is generally more user-friendly.

Q3. What's a concurrency limiter for? Limits in-flight requests, not request rate. Use for expensive endpoints (large queries, ML inference).

Q4. How do you partition? RateLimitPartition.GetXxxLimiter(key, factory). Common keys: IP, user id, API key, tenant.

Q5. Why might built-in rate limiting be wrong across replicas? Per-instance state. 5 replicas × 100 RPS each = 500 RPS total. Use Redis-backed or edge enforcement.

Q6. What status code does the middleware return? Default 503; configure RejectionStatusCode = 429.

Q7. What's Retry-After for? Tells clients when to retry. Set via OnRejected from Lease.GetMetadata(MetadataName.RetryAfter).

Q8. Queue vs immediate reject? QueueLimit > 0 queues when over limit (latency cost). QueueLimit = 0 rejects immediately (429 cost).

Q9. How do you exclude an endpoint? .DisableRateLimiting().

Q10. Why might rate-limit headers be missing? Built-in doesn't add them. Set in OnRejected or a custom middleware.

Gotchas / common mistakes

• ⚠️ IP partition behind proxy — all requests look the same.
• ⚠️ Forgetting Retry-After — clients hammer.
• ⚠️ Per-replica limit assumed global — wrong totals.
• ⚠️ No queue — sudden 429s; with queue, latency spike.
• ⚠️ Partition without : separator — collisions if using composite keys.

Further reading

• Rate limiting middleware
• RateLimitPartition
• Token bucket