Strings, Encoding & Interpolation Internals

Key Points

• string in .NET is UTF-16 internally — sequence of char (16-bit code units). One Unicode codepoint may take 1 or 2 chars (surrogate pair). Don't index into "characters."
• Encoding.UTF8 for I/O — bytes on disk and on the wire are virtually always UTF-8 in 2026.
• [InterpolatedStringHandler] (C# 10) makes $"..." allocation-free in hot paths (logging, string.Create, custom formatters).
• string.Create<TState> / string.Concat / StringBuilder — three perf tiers; pick by predicted length and number of fragments.
• String interning is a CLR table for compile-time literals. string.Intern exists but is rarely a win in modern .NET.

Concepts (deep dive)

UTF-16 and surrogate pairs

string s = "𝕏";                  // mathematical X — codepoint U+1D54F, outside BMP
Console.WriteLine(s.Length);      // 2 — two UTF-16 code units (high + low surrogate)
Console.WriteLine(s[0]);          // unprintable surrogate
foreach (var rune in s.EnumerateRunes()) Console.WriteLine($"{rune.Value:X}"); // 1D54F

Length is in code units, not codepoints. To iterate codepoints, use EnumerateRunes(). To iterate grapheme clusters (user-perceived characters incl. emoji + skin-tone modifiers), use StringInfo.GetTextElementEnumerator.

string is immutable

Every concatenation allocates. s += "x" in a loop is O(n²) memory.

// Bad
string s = "";
for (int i = 0; i < 1_000; i++) s += "x";

// Good
var sb = new StringBuilder(1_000);
for (int i = 0; i < 1_000; i++) sb.Append('x');
string s = sb.ToString();

// Best for known short list:
string s = string.Concat(parts);
// Or for templates:
string s = $"order={id} total={total:F2}";

Encoding: UTF-8 vs UTF-16 vs ASCII

byte[] utf8  = Encoding.UTF8.GetBytes(s);
byte[] utf16 = Encoding.Unicode.GetBytes(s);     // BOM-less LE on .NET
byte[] utf16Bom = Encoding.UTF8.GetPreamble();   // [0xEF,0xBB,0xBF]
string back = Encoding.UTF8.GetString(utf8);

• UTF-8 for files, HTTP, JSON, logs.
• UTF-16 LE internal CLR layout, Windows APIs (the W variants).
• ASCII never — use UTF-8 (ASCII is a subset, no downside).
• Encoding.Default is platform-dependent; rarely what you want in 2026 (it's UTF-8 on .NET 5+).

BOM (byte-order mark)

UTF-8 BOM (EF BB BF) is rare and undesirable on Linux/macOS. .NET writes UTF-8 without BOM by default since .NET 5. Watch for tools that prepend it (Excel CSV exporter is notorious).

Interpolated string handlers (C# 10+)

$"..." is rewritten to a DefaultInterpolatedStringHandler, which uses a stack-allocated Span<char> buffer when the result fits, falling back to ArrayPool<char> when not. Far better than the old string.Format.

[InterpolatedStringHandler]
public ref struct LogHandler
{
    public LogHandler(int literalLen, int formattedCount, ILogger logger, LogLevel level, out bool enabled)
    { enabled = logger.IsEnabled(level); /* ... */ }
    public void AppendLiteral(string s) { /* ... */ }
    public void AppendFormatted<T>(T value) { /* ... */ }
    public string ToStringAndClear() { /* ... */ }
}

public void Log(LogLevel level, [InterpolatedStringHandlerArgument(/* logger, level */)] LogHandler msg)
{
    if (level >= MinLevel) _logger.Log(level, msg.ToStringAndClear());
}

// Caller:
Log(LogLevel.Trace, $"order={id} total={total}");   // when Trace is off, args never formatted

ASP.NET Core's LoggerMessage source generator and Microsoft.Extensions.Logging use this pattern to skip work when a log level is disabled.

string.Create<TState>

Allocate exactly the final length and write into a Span<char> — single allocation, no intermediate buffer:

string Format(int n)
    => string.Create(11, n, (span, value) =>
       {
           value.TryFormat(span, out int written);
           " items".AsSpan().CopyTo(span.Slice(written));
       });

Comparison APIs (recap from equality topic)

"a".Equals("A", StringComparison.OrdinalIgnoreCase)     // true
"file.TXT".EndsWith(".txt", StringComparison.OrdinalIgnoreCase)
"über".CompareTo("uber")                                 // culture-dependent — avoid
"über".AsSpan().CompareTo("uber", StringComparison.Ordinal) // explicit

Utf8 (UTF-8 strings as ReadOnlySpan<byte>)

ReadOnlySpan<byte> json = """{"hello":"world"}"""u8;     // u8 literal, C# 11+
Utf8.TryWrite(buffer, $"id={id}", out int written);      // .NET 8+ Utf8 formatter

For wire protocols, working in UTF-8 bytes directly avoids the UTF-16 round-trip.

Interning

string a = "hello";
string b = "hello";
ReferenceEquals(a, b);   // True — both point to the same interned literal
string c = string.Intern(new string('h', 1) + "ello");   // forces interning

Compile-time literals are auto-interned. Manual Intern rarely pays off in modern code (cost: lifetime is process-wide, can never be GC'd).

Globalization mode

<RuntimeHostConfigurationOption Include="System.Globalization.Invariant" Value="true" />

Drops culture data from the runtime — smaller deployment, deterministic behavior. Good for backend services that never localize. Bad for anything customer-facing that needs proper sorting/casing.

Code: correct vs wrong

❌ Wrong: indexing emoji

string flag = "🇺🇸";
Console.WriteLine(flag.Length);         // 4 — two surrogate pairs
char first = flag[0];                   // unprintable surrogate; not "the U letter"

✅ Correct: iterate runes or grapheme clusters

foreach (var r in flag.EnumerateRunes()) Console.WriteLine(r.Value);
var enumerator = StringInfo.GetTextElementEnumerator(flag);
while (enumerator.MoveNext()) Console.WriteLine(enumerator.Current); // single 🇺🇸

❌ Wrong: building strings in a loop

string result = "";
foreach (var item in items) result += item.ToString() + ",";

✅ Correct: StringBuilder or string.Join

string result = string.Join(",", items);
// Or
var sb = new StringBuilder();
foreach (var item in items) sb.Append(item).Append(',');

❌ Wrong: hardcoding encoding

File.WriteAllText("out.txt", text, Encoding.GetEncoding(1252));   // Windows-1252; mojibake on Linux

✅ Correct: explicit UTF-8 (or default UTF-8 on .NET 5+)

File.WriteAllText("out.txt", text, new UTF8Encoding(encoderShouldEmitUTF8Identifier: false));

Design patterns for this topic

Pattern 1 — Pre-sized StringBuilder

var sb = new StringBuilder(estimatedFinalLength);

Avoids reallocation as the buffer grows. Estimate from input length.

Pattern 2 — Custom interpolated handler for hot logging

Skip formatting when the logger isn't listening — the handler queries IsEnabled first and short-circuits.

Pattern 3 — UTF-8 literal + Utf8.TryWrite

ReadOnlySpan<byte> prefix = "Bearer "u8;
Span<byte> buffer = stackalloc byte[256];
prefix.CopyTo(buffer);
Utf8.TryWrite(buffer.Slice(prefix.Length), $"{token}", out int written);

For HTTP header construction, JSON serialization shortcuts.

Pattern 4 — CompositeFormat for repeated templates

private static readonly CompositeFormat Template = CompositeFormat.Parse("Hello, {0}! You have {1} messages.");
string result = string.Format(null, Template, name, count);

Parses the template once, reuses across calls. Replaces hot string.Format paths.

Pros & cons / trade-offs

When to use / when to avoid

• Default: $"" interpolation. The compiler picks the right path.
• Hot loops: StringBuilder with capacity hint, or string.Create.
• Wire formats: stay in UTF-8 spans; avoid Encoding.GetString round-trips.
• Avoid culture-sensitive string comparison for identifiers (use Ordinal).
• Avoid string.Intern unless profiling proves it.

Interview Q&A

Q1. Why is string.Length not the codepoint count? A. string is UTF-16; Length counts 16-bit code units. Codepoints outside the BMP take two units (surrogate pair). Use EnumerateRunes() for codepoints.

Q2. When does $"..." allocate? A. The interpolated string handler stack-allocates a small buffer; rents from ArrayPool<char> if larger; allocates the final string once. Far better than the old string.Format path.

Q3. What's the difference between EnumerateRunes and StringInfo.GetTextElementEnumerator? A. Runes = Unicode codepoints. Text elements = grapheme clusters (user-perceived characters, including combining marks and emoji sequences). For "characters as users see them," use text elements.

Q4. Why is Encoding.UTF8.GetBytes("hello") faster than Encoding.Default.GetBytes? A. They're typically the same on modern .NET (default = UTF-8 since .NET 5). On old code or different platforms Default could be Windows-1252 or similar. Always be explicit.

Q5. What does the u8 suffix do? A. Creates a ReadOnlySpan<byte> of UTF-8 bytes baked into the assembly metadata — zero-allocation literal, ideal for headers and protocol fragments.

Q6. Why prefer StringComparison.Ordinal for identifiers? A. Ordinal is byte-wise, fastest, deterministic across locales; culture-sensitive comparison (the default for Compare) varies (Turkish "I", German "ß") and is much slower.

Q7. What's string.Create good for? A. Building a string of known final length in a single allocation by writing directly into the destination Span<char>. Avoids the intermediate buffer that StringBuilder uses.

Q8. How does [InterpolatedStringHandler] enable conditional formatting? A. The handler's constructor receives an out bool indicating whether to format at all. If false, all AppendLiteral/AppendFormatted calls become no-ops, so the arguments aren't even evaluated when (e.g.) the log level is off.

Gotchas / common mistakes

• Indexing into a string assuming char == codepoint == grapheme.
• Hardcoding Encoding.GetEncoding(1252) and getting mojibake everywhere else.
• Building strings with += in loops.
• Using string.Format instead of interpolation in modern code.
• Comparing strings with default culture-sensitive semantics for identifiers.
• BOM in CSV files breaking downstream parsers.
• Encoding.UTF8 constructor variants — passing true for BOM emission silently corrupts.
• Iterating string and ignoring grapheme clusters when measuring "user-perceived length."

Further reading

• Strings in .NET
• Interpolated string handlers
• System.Text.Rune
• Encoding class
• String performance best practices
• u8 literal