Property-Based Testing with FsCheck
Key Points
- Property-based testing (PBT) generates many random inputs and asserts invariants that must hold for all inputs — not just hand-picked examples.
- FsCheck is the canonical .NET library; works in C# and F#. Integrates with xUnit via
FsCheck.Xunitand the[Property]attribute. - Shrinking is the killer feature: when a property fails, FsCheck simplifies the input toward a minimal failing case — often surfaces the bug in two lines.
- Common property categories: round-trip (
decode(encode(x)) == x), commutativity, associativity, idempotence, model-based (compare to a simpler reference impl), invariant-after-N-operations. - Generators drive input creation.
Arb.Defaultcovers built-ins; for domain types you writeArbitrary<T>instances or use generator combinators. - PBT vs example-based: example tests prove "code works for the cases I thought of"; properties prove "code works for the input space I described". Different angle of attack.
- Watch out: nondeterministic CI failures (use a recorded seed for repro), trivial generators (always small ints), exponentially expensive shrinking on large inputs.
- Best fits: numerical code, parsers/serializers, state machines, domain invariants. Worst fits: CRUD plumbing, UI flows.
Concepts (deep dive)
Why property-based testing
Example-based testing:
[Theory]
[InlineData(1, 2, 3)]
[InlineData(0, 0, 0)]
[InlineData(-1, 1, 0)]
public void Add_works(int a, int b, int expected) => Assert.Equal(expected, Add(a, b));
You verify the cases you thought of. The bug typically lives in the case you didn't.
Property-based:
FsCheck runs this 100 times (configurable) with random (a, b) pairs. If any pair fails, FsCheck shrinks to find the minimum repro. You assert a property (commutativity); the framework explores the input space.
Install (FsCheck 3.x)
dotnet add package FsCheck.Xunit # xUnit integration with [Property] attribute
# or:
dotnet add package FsCheck.NUnit
dotnet add package FsCheck # core library
For 2026, FsCheck 3.x is current. The API stabilized around Prop/Gen/Arb namespaces.
First property
using FsCheck;
using FsCheck.Xunit;
public class StringProperties
{
[Property]
public bool Reverse_twice_is_identity(string s) =>
new string(s.Reverse().Reverse().ToArray()) == s;
[Property]
public bool Concat_length_equals_sum_of_lengths(string a, string b) =>
(a + b).Length == a.Length + b.Length;
}
Each [Property] runs N random invocations. Fails on the first counter-example. The xUnit runner shows the failing input + the shrunk version.
Shrinking — the magic
Given a failing test on input [5, 3, 9, 2, 7, 8, 1, 4, 6], FsCheck shrinks toward the minimum input that still fails. It might land on [2, 1] — a much clearer signal.
Falsifiable, after 17 tests (3 shrinks):
Original: Sort produces [1, 2, 5, 4, 7, 9, 3, 6, 8] for input [5, 3, 9, 2, 7, 8, 1, 4, 6]
Shrunk: Sort produces [1, 1] for input [1, 1] // bug: duplicates dropped
Without shrinking, debugging from a 1000-element failing input is brutal. Shrinking is what makes PBT practical.
Common property patterns
1. Round-trip
[Property]
public bool Json_round_trip<T>(T value) =>
JsonSerializer.Deserialize<T>(JsonSerializer.Serialize(value))!.Equals(value);
[Property]
public bool Base64_round_trip(byte[] input) =>
Convert.FromBase64String(Convert.ToBase64String(input)).SequenceEqual(input);
Universal for encode/decode, parse/print, serialize/deserialize pairs.
2. Commutativity
3. Associativity
[Property]
public bool Add_is_associative(int a, int b, int c) =>
Add(Add(a, b), c) == Add(a, Add(b, c));
4. Idempotence
[Property]
public bool Trim_is_idempotent(string s) => s.Trim().Trim() == s.Trim();
[Property]
public bool Sort_is_idempotent(int[] xs) =>
Sort(Sort(xs)).SequenceEqual(Sort(xs));
5. Model-based (oracle)
// Test optimized impl against a simple reference
[Property]
public bool Custom_sort_matches_LINQ(int[] xs) =>
MyCustomSort(xs).SequenceEqual(xs.OrderBy(x => x));
Powerful when you have a slow but obviously-correct reference implementation.
6. Invariant after N operations
[Property]
public bool Stack_count_matches_pushes_minus_pops(int[] ops)
{
var stack = new MyStack<int>();
var pushes = 0;
var pops = 0;
foreach (var op in ops)
{
if (op > 0) { stack.Push(op); pushes++; }
else if (stack.Count > 0) { stack.Pop(); pops++; }
}
return stack.Count == pushes - pops;
}
State-machine testing: random sequences of operations, then assert a structural invariant.
Generators and Arbitrary<T>
Arb.Default covers primitives and basic collections. For domain types:
public class EmailGenerator
{
public static Arbitrary<Email> Arb() =>
FsCheck.Arb.From(
from local in Gen.Choose(1, 20).SelectMany(n => Gen.ArrayOf(n, Gen.Elements("abc...".ToCharArray())))
from domain in Gen.Elements("example.com", "test.org", "x.io")
select new Email($"{new string(local)}@{domain}")
);
}
[Property(Arbitrary = new[] { typeof(EmailGenerator) })]
public bool Email_round_trip(Email e) =>
Email.Parse(e.ToString()) == e;
Gen.Choose, Gen.Elements, Gen.ArrayOf, LINQ syntax over Gen<T> — composable like LINQ over IEnumerable.
Conditional properties
Filter inputs that don't apply:
[Property]
public Property Divide_inverts_multiply(int a, int b)
{
return (b != 0).Implies(() => Divide(Multiply(a, b), b) == a);
}
Implies discards inputs failing the precondition. Watch out: aggressive filters may cause "rejected too many tests" errors — write a generator instead.
Configuration
[Property(MaxTest = 1000)] // run 1000 cases
public bool MyProperty(int x) => /* ... */;
[Property(Replay = "12345,67890")] // reproduce a specific failure
public bool MyProperty(int x) => /* ... */;
[Property(QuietOnSuccess = true)] // less console noise
When CI fails, FsCheck prints the seed (Replay = "..."). Capture it, paste it into the attribute, you reproduce locally deterministically.
Combining with xUnit Theory
public class CartProperties
{
[Property]
public bool Total_is_sum_of_lines(decimal[] prices)
{
var cart = new Cart();
foreach (var p in prices.Where(p => p > 0))
cart.Add(new Item(p));
return cart.Total() == prices.Where(p => p > 0).Sum();
}
}
Mixes naturally with regular xUnit [Fact] and [Theory] tests in the same class.
Alternative .NET PBT libraries
- Hedgehog.NET — port of the F# Hedgehog. Smaller community; integrated shrinking (vs FsCheck's separate shrinkers).
- CsCheck — C#-first, designed by Anthony Lloyd. Newer, fast, model-based testing primitives, less ceremony for C# generators.
// CsCheck flavor
Check.Sample(Gen.Int.Array, xs =>
{
var sorted = MySort(xs);
Assert.True(IsSorted(sorted));
Assert.Equal(xs.Length, sorted.Length);
});
For pure C# shops, CsCheck is increasingly attractive. FsCheck remains canonical and is very stable.
How it works under the hood
A property test is a function (seed, size) -> bool (conceptually). FsCheck:
- Pulls a random seed.
- Generates an input of "size" S (e.g., a string of length S, a tree of depth S).
- Runs your property; if
true, increments S and repeats. - If
false, shrinks: tries simpler variants of the failing input (smaller numbers, shorter strings, fewer nodes) and reports the smallest still-failing case.
The shrinker walks a search tree: each Arbitrary<T> provides a Shrink(T) -> IEnumerable<T> function — "here are simpler versions of this value." FsCheck does BFS/DFS, finding the minimum failing leaf.
For collections, the default shrinker tries: removing elements, shrinking each element, halving the length. For numeric types: toward zero. For strings: removing characters, simplifying chars.
This is why custom domain types need both a generator and a shrinker to get the full PBT experience.
Code: correct vs wrong
❌ Wrong: tautology property
True for any implementation. No actual assertion of behavior.
✅ Correct: real invariant
❌ Wrong: implies-everything filter
[Property]
public Property Divide_works(int a, int b)
{
return (b != 0 && a % b == 0 && a > 0).Implies(() => /* ... */);
}
Most random (a, b) pairs fail the filter. FsCheck rejects too many cases. Solution: write a generator that produces valid inputs directly.
✅ Correct: generator-driven valid inputs
public class DivisiblePairs
{
public static Arbitrary<(int, int)> Arb() =>
FsCheck.Arb.From(
from b in Gen.Choose(1, 100)
from k in Gen.Choose(1, 100)
select (a: b * k, b)
);
}
[Property(Arbitrary = new[] { typeof(DivisiblePairs) })]
public bool Divide_works((int a, int b) p) => /* ... */;
❌ Wrong: nondeterministic property
Fails sometimes, passes sometimes. CI nightmare.
✅ Correct: deterministic property + recorded seed
When CI prints a failing seed, paste it back, reproduce locally.
❌ Wrong: trivial generator giving up
[Property]
public bool Sort_works(int[] xs)
{
var sorted = Sort(xs);
return sorted.Length == xs.Length; // weak property
}
Verifies length only. Doesn't catch wrong-order bugs.
✅ Correct: stronger property
[Property]
public bool Sort_works(int[] xs)
{
var sorted = Sort(xs);
return sorted.Length == xs.Length
&& IsSorted(sorted)
&& sorted.OrderBy(x => x).SequenceEqual(xs.OrderBy(x => x)); // permutation
}
Three conjoined invariants: length preserved, output sorted, output is a permutation of input.
Design patterns for this topic
Pattern 1 — "Round-trip everywhere"
- Intent: for every encode/decode pair, write
decode(encode(x)) == x.
Pattern 2 — "Algebraic laws as tests"
- Intent: commutativity, associativity, identity, idempotence are free invariants.
Pattern 3 — "Reference oracle"
- Intent: test optimized impl against a slow obvious one.
Pattern 4 — "State-machine model"
- Intent: generate a random sequence of operations; assert a structural invariant after each.
Pattern 5 — "Custom generator > heavy filter"
- Intent: when most random inputs fail preconditions, build a generator that produces valid inputs.
Pattern 6 — "Recorded seed for repro"
- Intent: capture seed from CI failure, embed in
[Property(Replay = ...)].
Pros & cons / trade-offs
| Aspect | Pros | Cons |
|---|---|---|
| Bug discovery | Finds edge cases humans miss | Random; no specific guarantee |
| Coverage | Explores input space, not just examples | Slower than [Theory] |
| Confidence | Strong invariants verified at scale | Specifying invariants is hard |
| Maintenance | One property = many cases | Generator code overhead |
| Debugging | Shrinking gives minimal repros | Custom types need shrinkers too |
When to use / when to avoid
- ✅ Use for parsers, serializers, encoders.
- ✅ Use for numerical / algorithmic code.
- ✅ Use for state machines (random op sequences).
- ✅ Use for domain invariants (totals, balances, conservation laws).
- ❌ Avoid for CRUD plumbing — examples cover the shape.
- ❌ Avoid for UI flows — too dynamic.
- ❌ Avoid when invariants are unclear — write examples first, derive properties later.
Interview Q&A
Q1. What is property-based testing? A technique where the framework generates many random inputs and the test asserts an invariant that must hold for all of them — instead of hand-picking specific examples.
Q2. What does "shrinking" mean in PBT? When a property fails, the framework simplifies the failing input toward the minimum that still fails. Turns a sprawling counter-example into a tiny readable repro.
Q3. Name common property categories. Round-trip, commutativity, associativity, idempotence, model-based / oracle, invariant-after-N-operations.
Q4. FsCheck vs CsCheck vs Hedgehog? FsCheck — canonical .NET PBT, mature, separate generators/shrinkers. CsCheck — C#-first, fast, integrated. Hedgehog — generators carry shrinkers.
Q5. How do you reproduce a failing PBT in CI deterministically? Capture the seed printed in the failure output and embed via [Property(Replay = "seedA,seedB")]. FsCheck reruns with the same generator state.
Q6. What's the trap with Implies filtering? If most random inputs fail the predicate, FsCheck rejects too many tests and fails. Fix: write a generator that produces only valid inputs.
Q7. Why are custom shrinkers important for domain types? Default shrinkers don't know your domain. Without a custom shrinker, a failing complex object stays complex; shrinking is the practical edge of PBT.
Q8. Where does PBT pay off? Parsers/serializers, numerical code, state machines, anything with algebraic structure or strong invariants.
Q9. Where does PBT not pay off? CRUD plumbing, UI flows, code without clear invariants. Use example-based tests there.
Q10. PBT vs mutation testing — same goal? Different angles. PBT explores the input space to find bugs. Mutation testing explores the test-quality space by perturbing production code. Complementary.
Q11. How do you avoid trivial generated values dominating? Use size-aware generators (Gen.Sized), bias toward boundary values, write custom generators that explore the interesting parts of the input space.
Q12. How does PBT integrate with xUnit? FsCheck.Xunit provides [Property] attribute. Each property is discovered as a test; FsCheck handles invocation. Mixes with [Fact]/[Theory] in the same class.
Gotchas / common mistakes
- ⚠️ Tautological properties (
x is int) → no real check. - ⚠️ Heavy
Impliesfiltering → "rejected too many tests". - ⚠️ No custom shrinker for domain types → unhelpful counter-examples.
- ⚠️ Time/random side effects in property body → nondeterministic CI.
- ⚠️ Over-specified properties (e.g., asserting exact element ordering when only set equality matters).
- ⚠️ Forgetting to record seed when CI fails → can't reproduce.
- ⚠️
MaxTest = 100for cheap properties → too few cases for rare bugs; bump up. - ⚠️ Mixing PBT with side-effecting state → flaky; keep properties pure.
- ⚠️ Generator producing only trivial values (e.g.,
Gen.Choose(0, 5)for "any int") → coverage illusion. - ⚠️ Skipping example tests entirely → properties are abstract; examples document intent.