JIT & Tiered Compilation

Key Points

The .NET JIT (RyuJIT) compiles IL to native code at runtime. Two tiers: tier-0 ("quick JIT") prioritizes startup; tier-1 prioritizes steady-state.
Dynamic PGO (Profile-Guided Optimization) — enabled by default since .NET 6 — instruments tier-0 code and uses the profile to drive better tier-1 decisions (devirtualization, inlining, type-check elimination).
R2R (ReadyToRun) pre-compiles assemblies to native at publish time. Faster startup; tier-1 JIT can later replace hot R2R methods.
NativeAOT is a different beast — no JIT at all. See NativeAOT & Trimming.
.NET 9 brought significant loop optimizations (cloning, induction-variable widening, bounds-check elimination) and type-check PGO that can devirtualize is-checks and virtual calls based on observed types.
[MethodImpl(MethodImplOptions.AggressiveInlining)] is rarely the right answer. The JIT inlines aggressively when it can. Use it as a tie-breaker hint, not a blanket policy.

Concepts (deep dive)

Why tiered?

A non-tiered JIT has to choose: optimize hard (slow startup, fast steady-state) or optimize lightly (fast startup, slow steady-state). Tiered compilation gives you both:

                 First call to method M
                          │
                          ▼
                 ┌────────────────────┐
                 │  Tier 0 ("Quick")  │   ◄── small budget, fast compile
                 │  - few opts        │      - less inlining
                 │  - instrumentation │      - no aggressive specialization
                 │    counters        │
                 └─────────┬──────────┘
                           │
                           ▼ (M called many times)
                 ┌────────────────────┐
                 │  Tier 1 ("Opt")    │   ◄── full optimizer
                 │  - inlining        │      - PGO-guided devirtualization
                 │  - loop opts       │      - bounds-check elimination
                 │  - SIMD            │      - register allocation tuning
                 └─────────┬──────────┘
                           │
                           ▼
                 Tier-1 native code
                 replaces tier-0 in the
                 method-table slot

The runtime uses an internal call counter per method to decide when to recompile. Tier-1 takes longer but only happens for methods that actually run a lot.

💡 Senior insight: Steady-state benchmarks need a warm-up phase. BenchmarkDotNet does this automatically. If you measure cold tier-0 code, you measure noise.

Dynamic PGO

Tier-0 instruments — collects edge counts on branches, type observations on virtual calls, etc. By the time tier-1 fires, the runtime has actual profile data. PGO uses it to:

Devirtualize virtual / interface calls when one type dominates ("guarded devirtualization": specialize for the common case, fall back if surprised).
Inline based on actual hot-callee, not heuristics.
Move uncommon branches off the hot path.

public abstract class Animal { public abstract string Speak(); }
public sealed class Dog : Animal { public override string Speak() => "Woof"; }

public string MakeSpeak(Animal a) => a.Speak();

Without PGO, MakeSpeak is forced to do a virtual dispatch — table lookup, indirect call. With PGO, if 99% of calls pass Dog, the JIT generates:

if (a.GetType() == typeof(Dog)) {
    return "Woof";          // inlined! direct return!
} else {
    return a.Speak();       // fall back to virtual dispatch
}

The "guarded devirtualization" pays one type check for the win of an inlined fast path.

.NET 9 loop optimizations (selected)

Loop cloning — duplicate a loop, generating an optimized version for the common case (e.g., array.Length > 0) and a guard. Eliminates per-iteration bounds checks in the cloned loop.
Induction variable widening — when a loop counter is int but indexes a long-shaped array, widening the IV avoids per-iteration extension.
Bounds-check elimination in more shapes — e.g., for (int i = 0; i < arr.Length; i++) arr[i] = ... can drop checks entirely.

These are why a for loop over an array often emits the same code as you'd hand-write in C, in modern .NET.

R2R and Crossgen2

R2R (ReadyToRun) is pre-compiled native code embedded alongside IL in an assembly. At load time the runtime uses the native code immediately, no JIT needed. Tier-1 may later replace hot R2R methods with PGO-driven code.

Generate it via:

<PropertyGroup>
  <PublishReadyToRun>true</PublishReadyToRun>
  <PublishReadyToRunComposite>true</PublishReadyToRunComposite>  <!-- one big native unit -->
</PropertyGroup>

Crossgen2 is the tool that produces R2R images, written in C# (replacing the older C++ Crossgen). Supports cross-architecture compilation: build R2R for linux-arm64 from your win-x64 dev machine.

💡 Trade-off: R2R increases binary size (IL + native). For containerized services where startup matters, the size cost is usually acceptable.

Inlining heuristics

The JIT inlines aggressively when: - Callee is small (default budget ~16 IL instructions, more for "simple" methods). - Callee is non-virtual (or PGO has devirtualized it). - Callee is in the same assembly (no version-resilience cost).

The JIT does not inline when: - Callee has SEH (try/catch/finally) — historically a hard rule, increasingly relaxed. - Callee uses Calli, varargs, or unmanaged exports. - Caller's emit budget is exhausted.

[MethodImpl(MethodImplOptions.AggressiveInlining)] raises the budget but doesn't override hard rules. Use sparingly:

// Good case: tiny accessor in a hot loop where the JIT was reluctant.
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static int Square(int x) => x * x;

Don't sprinkle it. The JIT is generally smart; over-inlining bloats code, increases I-cache pressure, and slows things down.

How tier-0 → tier-1 happens

┌─────────────────────────────────────────────────────────────┐
│ Method call count exceeds tier-1 threshold (~30 by default) │
└────────────────────────────┬────────────────────────────────┘
                             │
                             ▼
                  Tier-1 compile queued
                             │
                             ▼
                  Background tier-1 thread JITs the method
                             │
                             ▼
                  Method-table slot atomically updated
                  to point at the new code
                             │
                             ▼
                  Existing in-flight tier-0 calls finish
                  Future calls go to tier-1

You can watch this with the dotnet-counters jit-tiered group — see how many methods are at each tier.

On-Stack Replacement (OSR) — the loop-stuck problem

Without OSR, a long-running loop in a method that started in tier-0 stays at tier-0 forever (the method never "returns" to trigger recompilation). OSR solves this: at a safe point inside the loop, the runtime can replace the running tier-0 frame with a tier-1 frame, mid-execution. Default in .NET 7+. Massive win for hot loops in long-running methods.

TieredCompilationQuickJitForLoops (default true) lets methods with loops go through tier-0. Older default was to skip tier-0 for those (because they couldn't recompile). OSR makes the default safe.

.NET 9 type-check PGO

public static int Process(object obj)
{
    if (obj is string s) return s.Length;     // type-check
    if (obj is int[] arr) return arr.Length;  // another type-check
    return -1;
}

In .NET 9, the JIT observes which types actually arrive (via PGO) and reorders / specializes the type checks. If string is the common case, it goes first; if it can prove the others are rare, it can devirtualize aggressively. The whole Process may JIT down to a fast-path direct check + a fallback.

How it works under the hood

The JIT lives in src/coreclr/jit/ in dotnet/runtime. The pipeline:

Importer — IL → JIT IR (an SSA-form intermediate representation).
Morph — high-level rewrites (e.g., struct copies, virtual call lowering).
Optimizer phases — inlining, copy-propagation, loop optimizations, bounds-check elimination, devirtualization.
Lowering — convert to a target-specific representation.
Register allocation — LSRA (Linear Scan).
Code emission — write native bytes.

Tier-0 skips most of (3); tier-1 runs the full pipeline.

The runtime tracks per-method state in an MethodDesc. The native code pointer in MethodDesc is what the JIT updates atomically when tier-1 compilation completes. Calls go indirect through this slot, which is what makes hot-swap possible.

Code: correct vs wrong

❌ Wrong: micro-benchmarking without warmup

var sw = Stopwatch.StartNew();
for (int i = 0; i < 1_000_000; i++)
    DoWork(i);
Console.WriteLine($"Elapsed: {sw.ElapsedMilliseconds} ms");
// You measured a mix of tier-0 and tier-1, plus possible OSR transitions.
// Numbers are unreliable.

✅ Correct: BenchmarkDotNet handles warmup, tiering, and statistical rigor

[MemoryDiagnoser]
public class WorkBench
{
    [Benchmark] public int DoWorkBench() => DoWork(42);
}

// Run via: dotnet run -c Release

❌ Wrong: spamming `[MethodImpl(MethodImplOptions.AggressiveInlining)]`

public class Calculator
{
    [MethodImpl(MethodImplOptions.AggressiveInlining)] public int Add(int a, int b) => a + b;
    [MethodImpl(MethodImplOptions.AggressiveInlining)] public int Sub(int a, int b) => a - b;
    [MethodImpl(MethodImplOptions.AggressiveInlining)] public int Mul(int a, int b) => a * b;
    // … etc.
}

The JIT inlines these without help. AggressiveInlining bloats your IL and gives no benefit in tier-1.

✅ Correct: use AggressiveInlining only after measurement showed the JIT didn't inline a hot path

// BenchmarkDotNet showed this method NOT inlined and on a hot loop:
[MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static int FastHash(ReadOnlySpan<byte> data) { /* ... */ }

❌ Wrong: assuming abstract calls cost nothing because "the JIT will figure it out"

public interface IProcessor { int Process(int x); }
// Lots of implementations.

public int Loop(IProcessor[] procs, int x)
{
    int sum = 0;
    foreach (var p in procs) sum += p.Process(x);
    return sum;
}

If procs contains many distinct types, PGO can't devirtualize — every call is an interface dispatch with type-table lookups. For very hot polymorphic loops, consider a closed-set switch:

public int Loop(ProcessorKind[] kinds, int x)
{
    int sum = 0;
    foreach (var k in kinds)
    {
        sum += k switch
        {
            ProcessorKind.A => ProcessA(x),
            ProcessorKind.B => ProcessB(x),
            _ => 0
        };
    }
    return sum;
}

Trade-off: less extensible but inlinable.

Design patterns for this topic

Pattern 1 — "Let PGO do its work"

Intent: write idiomatic abstractions; trust the JIT + PGO to specialize.
When to use it: virtually all application code.
When NOT to use it: kernels of inner loops profiled to be JIT-bound.

Pattern 2 — "Closed-set polymorphism via switch"

Intent: avoid virtual dispatch in tight loops where the dispatch overhead matters and the set of types is closed.
When to use it: parser dispatchers, message handlers, opcode interpreters.
Code sketch: see "correct vs wrong" #3.
Trade-offs: less extensible — every new type requires an edit. Acceptable when the types are also closed by domain.

Pattern 3 — "R2R for fast cold start"

Intent: compile the entire app to native at publish time so startup doesn't pay JIT cost.
When to use it: containerized services where cold-start latency matters.
When NOT to use it: desktop apps you ship via auto-update (size matters more than 200 ms cold start).

Pattern 4 — "AggressiveInlining only after measuring"

Intent: force the JIT to inline a small method that benchmarking showed it wouldn't.
When to use it: rarely; after a [MethodImpl(MethodImplOptions.NoInlining)] baseline benchmark proves a difference.

Pros & cons / trade-offs

Aspect	Pros	Cons
Tiered JIT	Best of both worlds: fast startup, fast steady-state	Brief tier-0 → tier-1 transition can show measurement noise
Dynamic PGO	Devirtualization based on actual workload	Adds tier-0 instrumentation overhead (small)
R2R	Faster startup; tier-1 can still replace hot methods	Larger binaries
OSR	Long-running loops escape tier-0	None significant; just on by default
AggressiveInlining	Force inline	Easy to misuse; bloats IL

When to use / when to avoid

Use R2R for services where startup latency matters and binary size doesn't.
Avoid disabling tiered compilation (COMPlus_TieredCompilation=0) unless benchmarking specifically requires it.
Avoid [MethodImpl(MethodImplOptions.NoOptimization)] outside of debugging.
Use PGO data files (<TieredPGO>true</TieredPGO> for static PGO, default dynamic) when you have a representative workload trace.

Interview Q&A

Q1. What is tiered compilation? A two-tier JIT strategy. Tier-0 ("quick JIT") emits less-optimized code fast — minimizing startup latency. After a method is called enough times, it's recompiled at tier-1 ("optimizing JIT") with full optimizations including PGO.

Q2. What is dynamic PGO? Profile-Guided Optimization driven by tier-0 instrumentation. The runtime collects edge counts and type observations during tier-0 execution; tier-1 uses that profile for inlining, devirtualization, and branch ordering.

Q3. What's "guarded devirtualization"? A specific PGO output: when one type dominates a virtual or interface call, the JIT generates if (obj is ExpectedType) { /* inlined call */ } else { /* virtual dispatch */ }. The check is cheap; the inlined fast path is the win.

Q4. What's OSR (On-Stack Replacement)? Replacing a running tier-0 stack frame with tier-1 code. Necessary for long-running loops that never return to retrigger compilation. Default since .NET 7.

Q5. What is R2R? ReadyToRun: pre-compiled native code shipped alongside IL in an assembly. Startup uses the native code immediately. Tier-1 may later replace hot methods with PGO-driven code.

Q6. Difference between R2R and NativeAOT? R2R = pre-compiled native code + IL + JIT still present. Tier-1 can replace methods. NativeAOT = no JIT at all; everything is statically compiled at publish time. NativeAOT has stricter rules (no reflection-emit, full trimming).

Q7. Why does [MethodImpl(MethodImplOptions.AggressiveInlining)] rarely help? Because the JIT inlines aggressively in tier-1. Most candidates are inlined without help. AggressiveInlining only matters when a method is just over the budget threshold; identify those by measurement, not by sprinkling.

Q8. How would you tell if a method is at tier-0 vs tier-1? dotnet-counters exposes jit-tiered-up counts. For more detail, dotnet-trace with the JIT keyword. In a debugger, you can see the method's native code change shape after enough calls.

Q9. What's crossgen2 for? The tool (written in C#) that produces R2R images. Supports cross-architecture compilation. Replaces the older C++-based crossgen.

Q10. Why should I prefer Span<T> over T[] in hot paths even when both are bounds-checked? Span<T> indexers are special-cased by the JIT for aggressive bounds-check elimination. Combined with .NET 9's loop opts, idiomatic for over a Span<T> often compiles to bounds-check-free code.

Q11. When does the JIT not inline? Callees with structured exception handling, calli, varargs, methods marked MethodImplOptions.NoInlining, or methods exceeding the inlining budget. Some of these have softened in recent .NET versions.

Q12. What's <TieredPGO>true</TieredPGO> in csproj? Enables static PGO using a profile file you provide (typically generated from a representative run). Static PGO is in addition to dynamic PGO; static benefits cold-start methods that don't run long enough for dynamic PGO to kick in.

Q13. Why might disabling tiered compilation slow down a benchmark? Without tiered, the runtime starts methods directly at tier-1 — slower startup, more JIT cost. If your benchmark has lots of one-shot methods, you pay the full optimizer cost for each. Counterintuitively, tier-0 + tier-1 is usually faster overall.

Gotchas / common mistakes

⚠️ Measuring before warmup — first calls hit tier-0 + JIT compilation cost. Always warm up.
⚠️ Disabling tiered compilation for "more consistent benchmarks" — usually the wrong move; just warm up.
⚠️ Spamming [MethodImpl(MethodImplOptions.AggressiveInlining)] without measurement.
⚠️ Polymorphic call sites with too many types — PGO can't help; consider closing the set.
⚠️ Trusting old JIT folklore — "the JIT can't optimize X" is often outdated by the time you read it. Re-test on the current .NET version.
⚠️ R2R'ing your library expecting tier-1 perf — R2R is closer to tier-0 quality. Tier-1 still happens at runtime for hot methods.