Performance & Load Testing

Key Points

• Load testing ≠ microbenchmarking. BenchmarkDotNet (covered in Performance) measures method-level perf in nanoseconds. Load testing exercises the full system end-to-end at production-like throughput.
• Tools (.NET 2026): NBomber (.NET-native, scenario API), k6 (JS-scripted, modern, by Grafana), Azure Load Testing (managed JMeter/k6, App Insights integration), Artillery, Gatling (Scala/Java).
• Load model matters: open-loop (constant arrival rate — realistic web traffic) vs closed-loop (constant VUs/threads — legacy JMeter default). Senior teams default to open-loop.
• Test types in order: smoke (1 user, sanity) → load (target sustained) → stress (overload to find breakpoint) → spike (sudden ramp) → soak (hours, find leaks).
• Read percentiles, not averages: P50/P95/P99/P99.9. Average masks tail latency, the actual user pain.
• Coordinated omission is the trap: closed-loop tools systematically under-report tail latency by skipping requests during slow periods.
• Where it fits in CI/CD: smoke per PR (60s, 1 user), load nightly, soak before production deploys. Gate on percentile thresholds and error rate.

Concepts (deep dive)

Load testing vs microbenchmarking

Both matter. They answer different questions.

Tool landscape

NBomber — .NET-native

using NBomber.CSharp;
using NBomber.Http.CSharp;

var http = HttpClientFactory.Create();

var scenario = Scenario.Create("get_users", async ctx =>
{
    var req = Http.CreateRequest("GET", "https://api.example.com/users")
        .WithHeader("Accept", "application/json");
    var response = await Http.Send(http, req);
    return response;
})
.WithLoadSimulations(
    Simulation.RampingInject(rate: 100, interval: TimeSpan.FromSeconds(1), during: TimeSpan.FromMinutes(2)),
    Simulation.Inject(rate: 100, interval: TimeSpan.FromSeconds(1), during: TimeSpan.FromMinutes(5))
);

NBomberRunner
    .RegisterScenarios(scenario)
    .WithReportFolder("./reports")
    .WithReportFormats(ReportFormat.Html, ReportFormat.Csv)
    .Run();

Strengths: scenarios in C#, full .NET ecosystem (DI, config, logging), open-loop injectors built in (Inject, RampingInject), pluggable for non-HTTP (gRPC, MQTT, custom).

Weaknesses: smaller community than k6; reports less polished than Gatling/Grafana.

k6 — modern, JS-scripted

import http from 'k6/http';
import { check } from 'k6';

export const options = {
    scenarios: {
        constant_load: {
            executor: 'constant-arrival-rate',
            rate: 100,                 // 100 requests per second
            timeUnit: '1s',
            duration: '5m',
            preAllocatedVUs: 50,
            maxVUs: 200,
        },
    },
    thresholds: {
        http_req_duration: ['p(95)<300', 'p(99)<800'],
        http_req_failed:   ['rate<0.01'],
    },
};

export default function () {
    const res = http.get('https://api.example.com/users');
    check(res, { 'status is 200': r => r.status === 200 });
}

Strengths: scenario syntax is clean, thresholds gate the run, Grafana Cloud Tests SaaS option, very active dev. Open-loop (constant-arrival-rate) is a first-class executor.

Weaknesses: JS not C# — context-switch for .NET teams; debugging custom logic inside k6 isn't great.

Azure Load Testing

Managed service in Azure. Upload a JMeter .jmx script or k6 .js file; runs at scale on Azure agents; integrates with App Insights to correlate client-side latency with server-side telemetry.

# azure-pipelines.yml snippet
- task: AzureLoadTest@1
  inputs:
    azureSubscription: 'My-Sub'
    loadTestConfigFile: 'loadtest.yaml'
    loadTestResource: 'my-loadtest-resource'

Strengths: zero infra, integrates with Azure Monitor / App Insights, scaled distributed runs, criteria-based pass/fail.

Weaknesses: cost (per virtual-user-minute), Azure lock-in, JMeter scripts feel dated.

Artillery / Gatling — also valid

• Artillery: Node.js, YAML scenarios, simple onramp. Fine for moderate loads; Pro tier needed for higher scale.
• Gatling: Scala-based, very mature, beautiful HTML reports. Heavier learning curve for Scala.

For .NET shops in 2026: NBomber if you want C#, k6 if you want best-in-class tooling and don't mind JS.

Open-loop vs closed-loop

This is the senior-level distinction.

Closed-loop (a.k.a. "concurrent users"):

N virtual users; each sits in a loop:
   request → wait for response → think for X ms → repeat

Open-loop (a.k.a. "arrival rate"):

Generate R requests/second regardless of how slow responses are.

              [ Closed-loop ]                [ Open-loop ]
              VU1 → req → wait                arrival_rate = 100/s
              VU2 → req → wait                |
              VU3 → req → wait                ↓
                                              queue ──→ workers
              if server is slow,              if server is slow,
              fewer requests fired            queue grows, true latency exposed

Closed-loop systematically underestimates tail latency under load because slow responses throttle further requests. Open-loop reflects the truth.

JMeter classic mode is closed-loop. NBomber's Inject and k6's constant-arrival-rate are open-loop. Default to open-loop unless you're explicitly modeling fixed concurrency (e.g., a thread-pool-bounded internal service).

Test types — pyramid by intent

Throughput
    ↑
    │     /\        ← Spike
    │    /  \
    │ ___    ___    ← Stress (ramping)
    │/         /
    │         /     ← Load (target sustained)
    │________________→ time

Reading results — percentiles

Latency:
  P50:  85 ms
  P95: 280 ms
  P99: 750 ms
  P99.9: 4200 ms
  Max: 18s
Avg: 110 ms

If you only saw Avg (110 ms) you'd think things are fine. P99.9 says 1 in 1000 users waits 4 seconds. At 1M requests/day that's 1000 furious users.

Senior rule: publish at minimum P50, P95, P99, error rate, throughput. Average is not a percentile and tells you nothing about tail behavior.

Coordinated omission

Coined by Gil Tene. Closed-loop tools that "wait for response then send next request" miss latency in slow periods.

Imagine sending 1 req/sec.
At t=0s: sent. Response at t=10s (slow!).
While waiting, the closed-loop tool didn't send t=1, t=2, ..., t=9.
Those 9 missed requests would have measured very high latency
(they would have been queued). They're omitted.
The reported latency reads like ONE request was slow.
The truth: ten requests would have been slow.

Tools that correct for coordinated omission: - HdrHistogram-based tools (wrk2, NBomber's HdrHistogram reporter). - Open-loop generators by definition don't have the problem — they keep firing.

If a tool reports incredibly clean P99 numbers under heavy load, suspect coordinated omission.

Where load tests live in CI/CD

Per-PR:    [ Smoke ] 60s, 1 VU. Catches API broken at runtime.
Nightly:   [ Load + Stress ] 30 min. Catches regressions.
Pre-prod:  [ Soak ] 4 hr. Catches memory leaks before deploy.
On-demand: [ Spike, Capacity ] before launches.

Don't gate every PR on a 30-minute load test — developer flow dies. Gate on smoke; track load/stress/soak on a schedule with percentile-based thresholds.

Threshold gating example (k6)

thresholds: {
    http_req_duration: ['p(95)<300', 'p(99)<800'],
    http_req_failed:   ['rate<0.01'],
    http_reqs:         ['rate>=95'],     // throughput floor
}

Each threshold becomes a pass/fail gate. The run fails the build if any breach. Senior practice: review thresholds quarterly; don't let them drift up silently.

Realistic test data

Synthetic load that hits the same record 1M times caches everything; results don't reflect production. Mix data:

const users = new SharedArray('users', function () {
    return JSON.parse(open('./users.json'));   // 10000 users
});

export default function () {
    const u = users[Math.floor(Math.random() * users.length)];
    http.get(`https://api.example.com/users/${u.id}`);
}

Or use parameterized data sources (NBomber DataFeed). Avoid trivial constants.

Network and infrastructure realism

Test from where users live. Local laptop → staging in another region adds 100 ms of WAN you can't change. Real load tests run from cloud agents in the same region as production.

For internal services, bias toward representative network (same VNet, same load balancer, same SSL termination).

Correlating with telemetry

Load test results alone tell you "P95 is 500 ms". They don't tell you why. Pair with:

• APM: App Insights, OpenTelemetry traces, Datadog APM.
• Server metrics: CPU, memory, GC pauses, thread pool starvation.
• Database metrics: query latency, connection pool waits, lock waits.

A great load test report links to the time-window in your APM tool. Reviewers see the latency spike and the GC chart side by side.

How it works under the hood

A load generator is a request-rate scheduler with a worker pool:

[ scheduler ]   tick = 1/rate seconds
     ↓
[ work queue ]  enqueue request descriptors
     ↓ pop
[ worker pool ] async HTTP clients fire requests
     ↓
[ collector ]   record start_ts, end_ts, status, bytes
     ↓
[ aggregator ]  HdrHistogram, percentiles, throughput
     ↓
[ report ]      HTML/JSON/CSV

For open-loop, the scheduler pushes regardless of worker availability — workers must keep up or the queue grows (true latency exposed).

For closed-loop, workers drive scheduling — for VU in 1..N: loop { req; wait response } — naturally throttling.

HdrHistogram records latency in a logarithmic bucket scheme that's O(1) memory — essential for hour-long runs that produce billions of samples. Lossless percentiles up to a configurable max value (typically 1 hr).

NBomber on top of HttpClient/Channel orchestrates this in idiomatic .NET. k6 uses a Go runtime executing JS via the goja interpreter (not Node.js).

Code: correct vs wrong

❌ Wrong: closed-loop test reporting fast P99 under stress

// Old JMeter-style: 100 threads, each loops
options = { vus: 100, duration: '5m' };
// Server slows to 2 sec/request → only 50 req/s actually fired
// P99 reads 2.1s — doesn't reflect production queue backlog

✅ Correct: open-loop arrival rate

options = {
    scenarios: {
        load: {
            executor: 'constant-arrival-rate',
            rate: 200, timeUnit: '1s',
            duration: '5m',
            preAllocatedVUs: 50, maxVUs: 500,
        }
    }
};
// 200/s fired regardless. If server slows, true tail latency exposed.

❌ Wrong: reporting only the average

Average response time: 145 ms. Looks great!

✅ Correct: percentile suite

P50: 90 ms | P95: 350 ms | P99: 1.2s | P99.9: 4.8s | Errors: 0.4%

❌ Wrong: hammering one record

http.get('/api/orders/42');   // every request, same id

Caches everything. Untrue throughput.

✅ Correct: distribution of inputs

const id = ids[Math.floor(Math.random() * ids.length)];
http.get(`/api/orders/${id}`);

❌ Wrong: gating every PR on a 30-minute load test

Pipeline waits 30 min for every commit; developers context-switch.

✅ Correct: smoke per PR, load nightly

60-second smoke gates correctness; nightly load test posts a percentile diff to the team channel.

Design patterns for this topic

Pattern 1 — "Smoke per PR, load nightly, soak pre-deploy"

• Intent: match cadence to test cost.

Pattern 2 — "Open-loop by default"

• Intent: model real arrival traffic, not concurrency.

Pattern 3 — "Percentile thresholds as gates"

• Intent: P95/P99/error rate fail the build when breached.

Pattern 4 — "Realistic data distribution"

• Intent: parameterize inputs to avoid cache-eaten results.

Pattern 5 — "Correlate with APM"

• Intent: every load test report links to APM time window.

Pattern 6 — "Soak for leaks"

• Intent: memory leaks and thread pool drift only show under hours, not minutes.

Pros & cons / trade-offs

When to use / when to avoid

• ✅ Use before any major launch or capacity decision.
• ✅ Use to baseline percentile budgets and gate on regressions.
• ✅ Use soak tests before any deploy with significant new state (caches, pools).
• ❌ Avoid as a substitute for monitoring real production traffic.
• ❌ Avoid closed-loop tools when modeling user traffic.
• ❌ Avoid averaging-only reports.
• ❌ Avoid running load tests against shared dev environments — flake everyone else.

Interview Q&A

Q1. Difference between BenchmarkDotNet and load testing? BenchmarkDotNet measures method-level performance (ns/op). Load testing measures full-system performance (req/s, P95 latency) over network.

Q2. Open-loop vs closed-loop generators? Open-loop: fixed arrival rate, queue grows under slowness — models web traffic. Closed-loop: fixed VUs that wait for response — under-reports tail latency under stress.

Q3. What is coordinated omission? Closed-loop tools skip requests during slow periods — those would-have-been-slow requests are omitted. Tail latency is systematically under-reported.

Q4. Why are percentiles required, not averages? Average masks the tail. P99 / P99.9 reveal the slow requests real users experience. Always publish a percentile suite.

Q5. List the test types. Smoke (1 VU sanity), load (sustained target), stress (find breakpoint), spike (sudden surge), soak (long endurance for leaks).

Q6. Where do load tests fit in CI/CD? Smoke per PR; load/stress nightly; soak pre-prod. Don't gate every PR on long tests — gate on smoke and nightly trends.

Q7. NBomber vs k6 — when to choose which? NBomber if your team is C# and wants .NET ecosystem (DI, logging, custom protocols). k6 if you want top-tier scenario syntax, Grafana integration, and accept JS.

Q8. How do you correlate load test results with server-side data? Run during a known time window; link the report to that window in App Insights / OpenTelemetry / Datadog. APM shows GC, CPU, query latency that explain the percentile spike.

Q9. How do you avoid trivial result skew from caching? Parameterize inputs: random user IDs, varied query parameters, real distribution of payload sizes. Trivial constants make the cache do all the work.

Q10. What's a soak test? Long-running test (hours) at sustained moderate load. Reveals memory leaks, connection pool drift, file handle leaks — issues invisible in 10-minute runs.

Q11. Why test from cloud agents in the production region? Network latency is real. Local laptop adds WAN that distorts results. Test where users live; production region is the closest realistic vantage.

Q12. How do you set percentile thresholds? Start from product SLO (e.g., P95 < 300 ms). Set the threshold slightly above current observed prod. Review quarterly; don't let it drift up silently.

Gotchas / common mistakes

• ⚠️ Closed-loop tools in 2026 → underestimate tail latency.
• ⚠️ Reporting averages → hides real user pain.
• ⚠️ Hammering one URL → cache eats it; results unrealistic.
• ⚠️ Local-machine load → flaky; can't sustain load.
• ⚠️ No correlation with APM → percentiles without root cause.
• ⚠️ Forgotten warmup phase → first-request latency skews P99.
• ⚠️ Running against shared dev env → flakes for everyone.
• ⚠️ Thresholds drifting upward → silent regression normalization.
• ⚠️ Soak too short → leaks not yet visible.
• ⚠️ Real-credentials in test scripts → secrets leak in artifacts.

Further reading

• NBomber docs
• k6 docs
• Azure Load Testing
• Gil Tene: How NOT to Measure Latency (talk) — coordinated omission explained
• HdrHistogram
• Brendan Gregg: USE method