Caching Strategies

Key Points

• Three layers in .NET: in-memory (IMemoryCache), distributed (IDistributedCache — Redis), HTTP output (UseOutputCache).
• HybridCache (.NET 9+) combines L1 (memory) + L2 (distributed) automatically. Replaces hand-rolled two-level cache patterns. Stampede protection built-in.
• Patterns: cache-aside (read-through), write-through, write-behind, refresh-ahead.
• Invalidation is hard: TTL is the simplest; tag-based invalidation (HybridCache, Redis) for grouped invalidation; events for distributed.
• Stampede protection: when cache expires, only one caller should rebuild — others wait. HybridCache has it; manual via SemaphoreSlim.

Concepts (deep dive)

IMemoryCache

builder.Services.AddMemoryCache();

public class C(IMemoryCache cache, AppDb db)
{
    public async Task<User> GetAsync(int id)
        => await cache.GetOrCreateAsync($"user:{id}", async e =>
        {
            e.SlidingExpiration = TimeSpan.FromMinutes(5);
            e.AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1);
            e.Size = 1;   // for size-bounded cache
            return await db.Users.FindAsync(id);
        });
}

In-process only. Per-instance. Fast (~ns). Lost on restart.

IDistributedCache

builder.Services.AddStackExchangeRedisCache(o =>
{
    o.Configuration = "redis:6379";
    o.InstanceName = "myapp:";
});

public class C(IDistributedCache cache)
{
    public async Task<User?> GetAsync(int id)
    {
        var json = await cache.GetStringAsync($"user:{id}");
        if (json is not null) return JsonSerializer.Deserialize<User>(json);
        var user = await /* load */;
        await cache.SetStringAsync($"user:{id}", JsonSerializer.Serialize(user),
            new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1) });
        return user;
    }
}

Redis (or SQL Server, Cosmos via providers). Shared across instances. Network hop.

HybridCache (.NET 9+)

builder.Services.AddHybridCache(o =>
{
    o.MaximumPayloadBytes = 1024 * 1024;
    o.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromHours(1),
        LocalCacheExpiration = TimeSpan.FromMinutes(5)
    };
});

public class C(HybridCache cache)
{
    public ValueTask<User> GetAsync(int id, CancellationToken ct)
        => cache.GetOrCreateAsync($"user:{id}", async (state, ct) =>
        {
            return await db.Users.FindAsync(state, ct);
        }, id, cancellationToken: ct);
}

Built-in features: - L1 (memory) + L2 (distributed) automatic. - Stampede protection — one rebuild per key. - Tag-based invalidation. - Configurable serializer. - Strongly-typed.

This replaces hand-rolled two-level patterns. Use it for new code.

Cache-aside (read-through)

Read: cache.Get → if miss: DB → cache.Set
Write: DB → cache.Invalidate (or Set)

Most common pattern. Cache lazy-fills on read.

Write-through

Write: cache.Set + DB.Update simultaneously

Cache always up-to-date. Slower writes (two operations).

Write-behind

Write: cache.Set; queue async DB write

Fast writes. Risk of data loss on cache failure. Rarely used in real systems — fragile.

Refresh-ahead

Proactive refresh before expiration. Eliminates expiration latency at cost of refresh load.

// HybridCache pattern: short LocalCacheExpiration triggers re-fetch from L2
o.LocalCacheExpiration = TimeSpan.FromSeconds(30);
o.Expiration = TimeSpan.FromMinutes(10);
// L1 expires often → re-pulls from L2 (or builds if also expired)

Stampede protection

Without protection: cache expires → 1000 simultaneous requests → 1000 DB hits. The "thundering herd".

// Manual stampede protection
private readonly SemaphoreSlim _lock = new(1);
public async Task<T> GetAsync(string key)
{
    if (_cache.TryGetValue(key, out T value)) return value;
    await _lock.WaitAsync();
    try
    {
        if (_cache.TryGetValue(key, out value)) return value;   // double-check
        value = await Load();
        _cache.Set(key, value);
        return value;
    }
    finally { _lock.Release(); }
}

HybridCache does this automatically.

Tag-based invalidation

// HybridCache
var user = await cache.GetOrCreateAsync(
    $"user:{id}",
    factory,
    tags: new[] { "users", $"user:{id}" });

await cache.RemoveByTagAsync("users");   // invalidate all users

For "invalidate everything user-related when policy changes" scenarios.

TTL strategies

For most apps: absolute + reasonable TTL + explicit invalidation on writes.

Cache warming

On startup, populate cache for known hot keys. Prevents cold-start latency cliff.

public class CacheWarmer : IHostedService
{
    public async Task StartAsync(CancellationToken ct)
    {
        foreach (var id in topUserIds) await cache.GetAsync($"user:{id}");
    }
}

Output caching (.NET 7+)

builder.Services.AddOutputCache(o =>
{
    o.AddPolicy("public", p => p.Expire(TimeSpan.FromMinutes(5)));
});
app.UseOutputCache();

app.MapGet("/products", () => /* ... */).CacheOutput("public");

Caches HTTP responses. Fastest cache — bypasses controller execution entirely.

Response caching (older)

[ResponseCache(Duration = 60, Location = ResponseCacheLocation.Any)]

Sets HTTP cache headers (Cache-Control). Browser/CDN caches.

OutputCache is preferred over ResponseCache for new code — server-side and configurable.

Cache size bounds

builder.Services.AddMemoryCache(o => o.SizeLimit = 1024);
// each entry: e.Size = 1

Otherwise unbounded → OOM. Always bound memory caches.

Negative caching

Cache misses (e.g., "user not found") to avoid repeated DB lookups for non-existent IDs:

await cache.SetStringAsync($"user:{id}:notfound", "1", TimeSpan.FromMinutes(1));

Short TTL — eventual consistency.

Cache key design

• Include the schema version: user:v2:{id}. Bumping v2→v3 invalidates everything atomically.
• Include the tenant: tenant:{t}:user:{id} for multi-tenant isolation.
• Use stable, deterministic keys.

Serialization

HybridCache supports pluggable serializers — JSON default, MessagePack faster.

builder.Services.AddHybridCache().AddSerializer<MyType, MyTypeSerializer>();

Stats / monitoring

• Hit rate (>90% = healthy).
• Memory usage.
• Eviction rate.

OpenTelemetry instrumentation exposes these.

Distributed cache failure modes

// ❌ Hard fail on Redis down
var cached = await _redis.GetStringAsync(...);
if (cached == null) throw new();

// ✅ Fall back to DB
try { return JsonSerializer.Deserialize<User>(await _redis.GetStringAsync(key)); }
catch (RedisException) { /* log, fall through */ }
return await _db.Users.FindAsync(id);

Treat distributed cache as best-effort. Don't make app down when Redis is.

CDN as cache

For public data, CDN (Cloudflare, Azure Front Door) is the fastest cache. Set Cache-Control: public, max-age=.... Pair with OutputCache middleware for origin caching.

Code: correct vs wrong

❌ Wrong: unbounded memory cache

builder.Services.AddMemoryCache();   // no SizeLimit

✅ Correct: bounded

builder.Services.AddMemoryCache(o => o.SizeLimit = 10_000);

❌ Wrong: stampede on expire

public async Task<T> Get(string key) =>
    cache.TryGetValue(key, out T v) ? v : (cache.Set(key, await Load()) /* parallel callers all hit DB */);

✅ Correct: HybridCache

return await cache.GetOrCreateAsync(key, factory);   // built-in stampede protection

❌ Wrong: hard fail on Redis outage

var u = JsonSerializer.Deserialize<User>(await _redis.GetStringAsync(key))
    ?? throw new("redis empty");

✅ Correct: fall through

try { /* redis */ } catch { /* fallback to DB */ }

Design patterns for this topic

Pattern 1 — "HybridCache for two-level"

• Intent: L1+L2 with stampede protection.

Pattern 2 — "Cache-aside default"

• Intent: simple, well-understood.

Pattern 3 — "Tag-based invalidation"

• Intent: group invalidate.

Pattern 4 — "Output caching for HTTP"

• Intent: bypass controller for hot reads.

Pattern 5 — "Versioned keys"

• Intent: atomic invalidation on schema change.

Pros & cons / trade-offs

When to use / when to avoid

• Use HybridCache for new code with multi-instance deployment.
• Use OutputCache for public read-heavy endpoints.
• Avoid write-behind unless you accept data loss.
• Avoid unbounded memory caches.

Interview Q&A

Q1. IMemoryCache vs IDistributedCache? In-memory: per-instance; fast. Distributed: shared via Redis; slower.

Q2. What does HybridCache add? L1 (memory) + L2 (distributed) automatic. Stampede protection. Tag invalidation.

Q3. Cache-aside vs write-through? Cache-aside: lazy-fill on read. Write-through: write to both at once. Cache-aside more common.

Q4. Stampede protection? Only one caller rebuilds expired key; others wait. SemaphoreSlim manual or HybridCache built-in.

Q5. TTL strategies? Short for hot/volatile; long + explicit invalidation for stable.

Q6. Tag-based invalidation? Tag entries on write; invalidate by tag (RemoveByTag). Group invalidation.

Q7. Output caching? Caches HTTP responses server-side. Bypasses controller. Fastest.

Q8. Why bound memory caches? Otherwise OOM. Always set SizeLimit + per-entry Size.

Q9. Cache key versioning? Include version (user:v2:{id}). Bump version = atomic global invalidation.

Q10. Distributed cache failure? Don't crash. Treat as best-effort; fall back to source.

Q11. CDN vs OutputCache? CDN: edge; closer to user. OutputCache: origin; controllable. Both layered.

Q12. Negative caching? Cache "not found" results to avoid repeated lookups. Short TTL.

Gotchas / common mistakes

• ⚠️ Unbounded memory cache — OOM.
• ⚠️ Cache-aside without stampede protection — thundering herd.
• ⚠️ Hard fail on Redis — coupled availability.
• ⚠️ No invalidation on writes — stale forever.
• ⚠️ Long TTL + complex data — diverges.

Further reading

• MS Learn: HybridCache
• MS Learn: Caching in ASP.NET Core
• MS Learn: Output caching
• Andrew Lock: Distributed cache