Memory Allocations & Pooling

Key Points

• Allocation isn't free even when it's fast. A bump-pointer alloc is sub-microsecond, but each alloc adds GC work eventually.
• ArrayPool<T>.Shared rents arrays from a process-wide pool. Best for short-lived buffers in hot paths.
• ObjectPool<T> (from Microsoft.Extensions.ObjectPool) reuses arbitrary objects (StringBuilder, custom DTOs) — useful when construction is expensive.
• stackalloc allocates on the stack. Zero GC cost. Cap the size; never put stackalloc inside a loop without strict bounds.
• Span<T> / Memory<T> are the views that let pooled or stack memory feel like idiomatic .NET. Without them, pooling is awkward.
• Avoid the LOH cliff (85 KB) — either stay below or pool above. Don't repeatedly allocate 90 KB arrays.
• MemoryPool<T> is the abstraction for pluggable pools that produce IMemoryOwner<T> (buffers with using semantics).
• Profile first. BenchmarkDotNet with [MemoryDiagnoser] shows you allocations per call. Without that, you're guessing.

Concepts (deep dive)

The full allocation map

┌──────────────────────────────────────────────────────────────┐
│                      Process memory                           │
├──────────────────────────────────────────────────────────────┤
│ Stack (per thread, ~1 MB default)                              │
│   └─ stackalloc, locals, struct values                         │
├──────────────────────────────────────────────────────────────┤
│ Managed heap                                                   │
│   ├─ Gen 0 / Gen 1 / Gen 2  ← `new`, most allocations          │
│   ├─ LOH                    ← `new` for ≥85 KB                 │
│   └─ POH                    ← GC.AllocateArray(pinned: true)   │
├──────────────────────────────────────────────────────────────┤
│ Native (unmanaged)                                             │
│   ├─ Marshal.AllocHGlobal / AllocCoTaskMem                     │
│   ├─ NativeMemory.Alloc (.NET 6+)                              │
│   └─ MemoryMappedFile, native libs                             │
└──────────────────────────────────────────────────────────────┘

The senior-bar question is always: does this allocation matter for my workload? A 10-byte allocation in a method that runs once is irrelevant. The same allocation in a method that runs 10⁸ times per second is a PR-priority issue.

ArrayPool<T>.Shared

byte[] buffer = ArrayPool<byte>.Shared.Rent(minimumLength: 4096);
try
{
    int read = stream.Read(buffer, 0, 4096);
    ProcessSpan(buffer.AsSpan(0, read));
}
finally
{
    ArrayPool<byte>.Shared.Return(buffer, clearArray: false);
}

Key facts:

1. Rent(n) returns an array of size ≥ n — possibly larger. Always pass the actual length to consumers via Span<T>.
2. clearArray: false is the default for performance; set true if the buffer held sensitive data (PII, secrets) before returning.
3. Shared is process-wide and bucketized by power-of-two sizes. Asking for 5,000 bytes gets you a bucket of 8,192.
4. Return is not strictly required — the pool tolerates leaks (the array just gets GC'd) — but you lose the perf benefit.

💡 Senior insight: rent into a Span<T> view, not a raw array, to avoid accidentally overflowing the requested length:

byte[] arr = ArrayPool<byte>.Shared.Rent(size);
Span<byte> buffer = arr.AsSpan(0, size);    // ← only the asked-for portion

IMemoryOwner<T> and MemoryPool<T>

MemoryPool<T>.Shared returns IMemoryOwner<T> — a using-friendly handle that owns a memory region:

using IMemoryOwner<byte> owner = MemoryPool<byte>.Shared.Rent(size);
Memory<byte> memory = owner.Memory;
// ... use memory; auto-returned on Dispose ...

This wraps ArrayPool<T>.Shared under the hood (in the default impl). Use this when you want the using semantics or you want a pluggable pool — for example, a streaming pipeline accepting any MemoryPool<byte>.

ObjectPool<T>

// Setup (DI):
services.AddSingleton<ObjectPoolProvider, DefaultObjectPoolProvider>();
services.AddSingleton(provider =>
    provider.GetRequiredService<ObjectPoolProvider>().CreateStringBuilderPool());

// Use:
StringBuilder sb = _pool.Get();
try
{
    sb.Append("...");
    return sb.ToString();
}
finally
{
    _pool.Return(sb);   // returns to pool, not freed
}

Built-in helpers: CreateStringBuilderPool(), Create<T>() for any class, new(). Custom pooled types implement IPooledObjectPolicy<T> (Create() and Return(T) → bool so you can reject "too big" instances and let them be GC'd).

💡 Senior insight: the default StringBuilder pool keeps small builders. If your code grows them to megabytes, the policy may reject them on return — defaulting to GC. That's correct behavior but worth knowing.

stackalloc — when and how

const int Threshold = 256;
Span<byte> buffer = size <= Threshold
    ? stackalloc byte[size]
    : new byte[size];

Process(buffer);

Rules:

1. stackalloc size must be small. The thread stack is ~1 MB. Allocate 100 KB and you've used 10% of it.
2. stackalloc in a loop does not free between iterations. The stack frame holds them all until the method returns.
3. Always assign to Span<T> (or ReadOnlySpan<T>) to get bounds checking. Raw pointer stackalloc byte* p = stackalloc byte[N] is unsafe.
4. C# 7.2+ removed the unsafe requirement for stackalloc when assigned to Span<T>. Modern code rarely uses pointers.

Span<T> / Memory<T> / ref struct

Span<T> is a ref struct — stack-only. Cannot be a field of a class, cannot be boxed, cannot be captured by an async lambda (because it would need to be lifted to the heap).

Memory<T> is the heap-friendly alternative: a struct (not ref struct) wrapping the same kind of contiguous-memory view. Use Memory<T> when you need to pass through awaits.

// ✅ Span — sync hot paths
public int Process(ReadOnlySpan<byte> data) { /* ... */ }

// ✅ Memory — async pipelines
public async Task ProcessAsync(ReadOnlyMemory<byte> data, CancellationToken ct)
{
    await Pipeline.WriteAsync(data, ct);
}

Convert: memory.Span returns the Span<T> view (valid as long as the underlying buffer is alive).

NativeMemory.Alloc — the unmanaged option

unsafe
{
    void* ptr = NativeMemory.Alloc(byteCount: 1024);
    try
    {
        Span<byte> buffer = new(ptr, 1024);
        // ... no GC pressure at all ...
    }
    finally
    {
        NativeMemory.Free(ptr);
    }
}

NativeMemory (since .NET 6) is the modern, runtime-aware native allocator. Faster and safer than Marshal.AllocHGlobal. Use when:

1. You need a long-lived buffer that GC has no business touching.
2. You're interop with native APIs that take ownership of allocations.

⚠️ Manual lifetime management. Forget Free() and you have a real, OS-level leak.

Avoiding the LOH cliff

// ❌ Routinely allocates onto LOH; no compaction; fragmentation builds up.
public byte[] Encode(string message)
    => Encoding.UTF8.GetBytes(LongJsonHeader + message);

// ✅ Pool the buffer; encode into it.
public ArraySegment<byte> Encode(string message, ArrayPool<byte> pool)
{
    int max = Encoding.UTF8.GetMaxByteCount(LongJsonHeader.Length + message.Length);
    byte[] buffer = pool.Rent(max);
    int written = Encoding.UTF8.GetBytes(LongJsonHeader + message, buffer);
    return new ArraySegment<byte>(buffer, 0, written);
    // Caller is responsible for returning the buffer.
}

Allocation hot-spot patterns

How it works under the hood

ArrayPool<T>.Shared is a configurable pool with two implementations:

• SharedArrayPool (default) — per-CPU buckets to reduce contention. Bucket sizes power-of-two from 16 up to 2²⁰ bytes. Larger requests bypass the pool.
• Custom impls can be substituted via ArrayPool<T>.Create(maxArrayLength, maxArraysPerBucket).

// Default
var arr = ArrayPool<byte>.Shared.Rent(1024);

// Custom (for, say, predictable behavior in a test)
var pool = ArrayPool<byte>.Create(maxArrayLength: 4096, maxArraysPerBucket: 16);
var arr = pool.Rent(1024);

Source: runtime/src/libraries/System.Buffers.ArrayPool/....

ObjectPool<T> is a thin wrapper around a per-CPU ConcurrentQueue<T> with a configurable maximum count. The default policy reuses objects up to a small cap.

Code: correct vs wrong

❌ Wrong: not returning rented arrays

public string ReadAll(Stream s)
{
    var buf = ArrayPool<byte>.Shared.Rent(64 * 1024);
    s.Read(buf, 0, 64 * 1024);
    return Encoding.UTF8.GetString(buf);   // ❌ size used = full bucket size
                                           // ❌ buf never returned
}

Two bugs: returning the entire bucket size as decoded text, and leaking the buffer.

✅ Correct

public string ReadAll(Stream s)
{
    byte[] buf = ArrayPool<byte>.Shared.Rent(64 * 1024);
    try
    {
        int read = s.Read(buf, 0, buf.Length);
        return Encoding.UTF8.GetString(buf, 0, read);
    }
    finally
    {
        ArrayPool<byte>.Shared.Return(buf);
    }
}

❌ Wrong: stackalloc in a loop

for (int i = 0; i < 100_000; i++)
{
    Span<byte> tmp = stackalloc byte[256];   // ❌ accumulates on stack
    Process(tmp);
}
// Stack overflow after enough iterations.

✅ Correct: hoist outside the loop

Span<byte> tmp = stackalloc byte[256];
for (int i = 0; i < 100_000; i++)
{
    tmp.Clear();
    Process(tmp);
}

❌ Wrong: capturing in a hot lambda

public int Sum(int[] data, int threshold)
    => data.Where(x => x > threshold).Sum();   // closure allocated per call

✅ Correct: avoid the closure

public int Sum(ReadOnlySpan<int> data, int threshold)
{
    int sum = 0;
    foreach (var x in data)
        if (x > threshold) sum += x;
    return sum;
}

❌ Wrong: string concatenation in a tight loop

public string Build(IEnumerable<string> parts)
{
    string s = "";
    foreach (var p in parts) s += p;   // ❌ O(n²) and lots of allocations
    return s;
}

✅ Correct: StringBuilder (and pool it if hot)

private static readonly ObjectPool<StringBuilder> _sbPool =
    new DefaultObjectPoolProvider().CreateStringBuilderPool();

public string Build(IEnumerable<string> parts)
{
    var sb = _sbPool.Get();
    try
    {
        foreach (var p in parts) sb.Append(p);
        return sb.ToString();
    }
    finally
    {
        _sbPool.Return(sb);
    }
}

Design patterns for this topic

Pattern 1 — "Rent, use, return"

• Intent: acquire a pooled buffer for the duration of a method, return on exit.
• When to use it: any short-lived buffer larger than ~1 KB on a hot path.
• Code sketch: see "correct vs wrong" #1.
• Trade-offs: the try/finally adds a few lines; small price for the GC savings.

Pattern 2 — "Stack-allocate small, heap-allocate large"

• Intent: zero-allocation for small inputs; correct for arbitrary sizes.
• When to use it: parsing, formatting, inner loops with bounded inputs.
• Code sketch:

const int Threshold = 1024;
Span<byte> buf = size <= Threshold
    ? stackalloc byte[size]
    : new byte[size];

Pattern 3 — "Pooled StringBuilder for repeated string building"

• Intent: reuse builders to avoid allocation churn.
• When to use it: logging hot paths (where structured logging isn't appropriate), serialization, large concatenation in services.

Pattern 4 — "Span everywhere on the synchronous path; Memory at the async boundary"

• Intent: avoid copying as long as possible; transition to heap-friendly type only when crossing async.
• Code sketch:

public ReadOnlyMemory<byte> Encrypt(ReadOnlyMemory<byte> input) // can be awaited
{
    // ... convert to span at the inner sync core ...
    var span = input.Span;
    Span<byte> output = stackalloc byte[span.Length + 16];
    Cipher.Encode(span, output);
    return output.ToArray();   // materialize for return — final allocation
}

Pattern 5 — "NativeMemory for very-long-lived buffers"

• Intent: keep a multi-megabyte buffer entirely off the GC heap.
• When to use it: caching layers, columnar data, custom database internals.
• Trade-offs: manual Free. Use SafeHandle to make ownership disposable-friendly.

Pros & cons / trade-offs

When to use / when to avoid

• Use ArrayPool<T>.Shared for transient buffers ≥ 1 KB on hot paths.
• Use pooled StringBuilder when serializing/formatting in hot paths.
• Use stackalloc for buffers that fit in a few hundred bytes and don't escape the method.
• Avoid pooling tiny objects — the pool's overhead exceeds the GC savings.
• Avoid allocating exactly at 85 KB threshold — either stay below or pool above.
• Avoid premature pooling — measure first. Pooling adds complexity; only pay the price where the benefit is real.

Interview Q&A

Q1. What does ArrayPool<T>.Shared.Rent(n) actually return? An array of length at least n — typically a power-of-two bucket size. You must track the actual usable length yourself (often via Span<T> slicing).

Q2. When is ArrayPool worse than new T[n]? For very small or very large allocations. Tiny: pool overhead exceeds GC savings. Very large: Shared doesn't pool above ~1 MB by default, and Rent falls back to new. Also: when you'd hold the buffer for a long time — pooling a long-lived buffer defeats the purpose.

Q3. What's the LOH and why is it dangerous? The Large Object Heap, for objects ≥85 KB. Not compacted by default — fragmenting it produces apparent free space the allocator can't use. Avoid by pooling large buffers.

Q4. When would you choose Memory<T> over Span<T>? Across await boundaries. Span<T> is a ref struct; the async state machine can't lift it to the heap. Memory<T> can cross awaits.

Q5. Why is stackalloc in a loop dangerous? The stack frame keeps every allocation alive until the method returns. A loop with stackalloc accumulates space — eventually overflowing the thread stack.

Q6. How does ObjectPool<T> differ from ArrayPool<T>? ArrayPool is specialized for arrays with bucketed sizes. ObjectPool is generic over any class, new() (or with custom policy). Use ObjectPool for StringBuilder, custom DTOs, or anything expensive to construct.

Q7. What does clearArray do on ArrayPool.Return? If true, the array is zeroed before being placed back in the pool. Set to true for buffers that held sensitive data; otherwise leave false for performance.

Q8. What's MemoryPool<T> and when do you reach for it? The pluggable abstraction returning IMemoryOwner<T> (a using-disposable). Default impl wraps ArrayPool. Use when you want pluggable pools or using semantics.

Q9. What's NativeMemory.Alloc good for? Off-heap allocation that the GC never scans. Good for very-long-lived buffers (cache layers, columnar data) where GC scanning overhead matters. Manual Free required.

Q10. How would you find the allocation hotspot in a service? 1) [MemoryDiagnoser] in BenchmarkDotNet for unit benchmarks. 2) dotnet-trace with the GC and AllocationTick events for production-like trace. 3) PerfView "Allocation Sampling" view, sort by exclusive allocations. 4) dotMemory for heap-and-allocations together.

Q11. Why is iterating a Span<int> faster than iterating an int[]? Both bounds-check, but the JIT special-cases Span<T> indexers and (especially in .NET 9) eliminates bounds checks aggressively when the loop bound matches Span.Length. Net effect: identical or better than the array loop.

Q12. Walk through a hidden allocation in string.Split. Split returns string[], which means: (a) one heap allocation for the array; (b) one heap allocation per substring (each string is allocated separately); © on entry, the input string was already heap-resident. For one call, irrelevant; in a hot loop parsing 10⁶ records, it's the GC's #1 source.

Q13. What's [InterpolatedStringHandler] and how does it help allocations? A C# 10+ feature that lets the compiler emit a struct-based builder for a method-call's interpolated argument. The classic example is ILogger.LogInformation($"User {userId} did X"): the framework's handler captures the format string + args without allocating a runtime string, and only formats if the level is enabled. Free elimination of allocations when the log is disabled.

Gotchas / common mistakes

• ⚠️ Forgetting to Return rented arrays — pool degrades to GC over time.
• ⚠️ Using rented array's Length as the data length — you got a larger bucket than requested.
• ⚠️ stackalloc in loops — eventual stack overflow.
• ⚠️ Boxing structs by storing in object-typed collections — heap allocation per add. Use generic List<T> etc.
• ⚠️ Closure captures in hot LINQ chains — measure with [MemoryDiagnoser].
• ⚠️ StringBuilder not pooled — every .ToString() materializes; every new StringBuilder() allocates.
• ⚠️ Span<T> stored in a field — compile error, but people try.
• ⚠️ Memory<T> from pooled buffer outliving the rental — the underlying array goes back to the pool while another part of the code still holds a Memory<T> view of it. Race condition.
• ⚠️ NativeMemory leak via thrown exception — wrap in try/finally or use a SafeHandle.

Further reading

• ArrayPool<T>
• MemoryPool<T>
• Microsoft.Extensions.ObjectPool
• Span<T> and Memory<T> guidance
• stackalloc reference
• NativeMemory