P/Invoke & Native Interop with [LibraryImport]

Key Points

• [DllImport] is the legacy P/Invoke attribute — runtime-generated marshalling stubs, AOT-hostile.
• [LibraryImport] (C# 11 / .NET 7+) is the source-generated replacement — compile-time stubs, AOT-friendly, faster, fewer surprises.
• Blittable types (primitives + structs of primitives, no marshalling) move across the boundary as bit copies. Non-blittable (string, bool, arrays) need explicit marshallers.
• Marshal class still essential for manual buffer manipulation, GCHandle pinning, COM HRESULT translation.
• Function pointers (delegate*) enable callbacks without delegate allocations — the modern way to pass C-style callbacks across the boundary.

Concepts (deep dive)

DllImport (legacy)

[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
public static extern bool DeleteFile(string path);

The runtime generates a marshalling stub via reflection on first call: argument conversion, GC pinning, transition to/from native. Stubs are slow to JIT and hostile to NativeAOT trimming.

LibraryImport (modern)

public partial class FileNative
{
    [LibraryImport("kernel32.dll", SetLastError = true, StringMarshalling = StringMarshalling.Utf16)]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static partial bool DeleteFile(string path);
}

A Roslyn source generator emits the marshalling code at compile time. The method must be static partial. AOT-friendly — no reflection, no runtime stub generation.

Blittable types

Move byte-for-byte across the boundary:

• byte, sbyte, short, ushort, int, uint, long, ulong
• float, double
• IntPtr, UIntPtr, nint, nuint
• Pointers (int*)
• structs containing only blittable fields with [StructLayout(LayoutKind.Sequential)] or Explicit

Non-blittable: bool (size differs by ABI), char, string, arrays — all need marshalling hints.

Common marshalling

[LibraryImport("user32.dll", StringMarshalling = StringMarshalling.Utf16)]
public static partial int MessageBoxW(IntPtr hwnd, string text, string caption, uint type);

[LibraryImport("native.dll")]
public static partial int Sum(
    [MarshalUsing(CountElementName = nameof(count))] int[] values,
    int count);

StringMarshalling: - Utf16 — Windows wide strings - Utf8 — POSIX, modern APIs - Custom — provide your own marshaller

Layout control

[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Unicode)]
public struct WIN32_FIND_DATAW
{
    public uint dwFileAttributes;
    public FILETIME ftCreationTime;
    public FILETIME ftLastAccessTime;
    public FILETIME ftLastWriteTime;
    public uint nFileSizeHigh;
    public uint nFileSizeLow;
    public uint dwReserved0;
    public uint dwReserved1;
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 260)]
    public string cFileName;
    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 14)]
    public string cAlternateFileName;
}

Pack = 1 disables padding when the native side packs tightly. Mismatched layout = silent data corruption.

Function pointers

C# 9+ feature; pass a managed method as a delegate*:

[UnmanagedCallersOnly(CallConvs = [typeof(CallConvCdecl)])]
static int Compare(int a, int b) => a.CompareTo(b);

[LibraryImport("native.dll")]
public static partial void QSort(IntPtr data, int count, int size, delegate* unmanaged[Cdecl]<int, int, int> cmp);

QSort(buffer, n, sizeof(int), &Compare);

No Delegate allocation; matches C function-pointer ABI directly. Required for AOT scenarios with callbacks.

GCHandle for pinning

var bytes = new byte[1024];
var h = GCHandle.Alloc(bytes, GCHandleType.Pinned);
try
{
    nint ptr = h.AddrOfPinnedObject();
    NativeRead(ptr, bytes.Length);
}
finally { h.Free(); }

Or, simpler, use fixed:

fixed (byte* p = bytes) { NativeRead((nint)p, bytes.Length); }

fixed only pins for the scope of the block; GCHandle for cross-call pinning.

Marshal essentials

nint p = Marshal.AllocHGlobal(1024);
try { /* native code */ }
finally { Marshal.FreeHGlobal(p); }

string s = Marshal.PtrToStringUTF8(nativeStringPtr)!;
int err = Marshal.GetLastWin32Error();           // pair with SetLastError = true
Marshal.ThrowExceptionForHR(hr);                  // COM HRESULT → exception

COM interop (sketch)

[ComImport, Guid("…"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface ISomeCom { int DoWork(); }

var obj = (ISomeCom)Activator.CreateInstance(Type.GetTypeFromCLSID(new Guid("…"))!)!;

Microsoft.Windows.CsWin32 source generator (NuGet) generates [LibraryImport] and COM bindings from Win32 metadata — a much better path than hand-rolling.

Code: correct vs wrong

❌ Wrong: DllImport in AOT app

[DllImport("native.dll")]
public static extern int Process(string input);   // runtime stub generation, breaks NativeAOT

✅ Correct: LibraryImport

public partial class Native
{
    [LibraryImport("native.dll", StringMarshalling = StringMarshalling.Utf8)]
    public static partial int Process(string input);
}

❌ Wrong: passing managed array without pinning

[LibraryImport("native.dll")]
public static partial void Read(byte[] buffer);
// If native side stores the pointer and uses it after return, GC moves the array → corruption.

✅ Correct: pin for cross-call use

var bytes = new byte[1024];
var h = GCHandle.Alloc(bytes, GCHandleType.Pinned);
try { NativeRegisterBuffer(h.AddrOfPinnedObject(), bytes.Length); /* ... */ }
finally { h.Free(); }

Design patterns for this topic

Pattern 1 — partial class for native functions

Group native methods by library/area in a partial class. Source generator emits stubs alongside.

Pattern 2 — SafeHandle over raw IntPtr

public sealed class FileSafeHandle : SafeHandleZeroOrMinusOneIsInvalid
{
    public FileSafeHandle() : base(true) { }
    protected override bool ReleaseHandle() => CloseHandle(handle);
    [LibraryImport("kernel32.dll")] private static partial bool CloseHandle(nint h);
}

SafeHandle ties handle lifetime to GC + finalizer — essential for resource leaks across exceptions and async.

Pattern 3 — Use CsWin32 source generator

<PackageReference Include="Microsoft.Windows.CsWin32" Version="..." />

NativeMethods.txt lists APIs you want. The generator emits [LibraryImport] declarations from official Win32 metadata — type-safe, AOT-friendly.

Pattern 4 — [UnmanagedCallersOnly] for C-callable exports

[UnmanagedCallersOnly(EntryPoint = "myplugin_init")]
public static int Init() => 0;

Required when exposing managed functions as native exports (e.g., NativeAOT shared libraries consumed by C/C++).

Pros & cons / trade-offs

When to use / when to avoid

• Use LibraryImport for new code. Migrate DllImport when convenient.
• Use SafeHandle for any unmanaged resource exposed through P/Invoke.
• Avoid hand-rolled COM if CsWin32 or generator-based options exist.
• Avoid P/Invoke entirely when a managed alternative exists — fewer ABI surprises.

Interview Q&A

Q1. Why is [LibraryImport] faster than [DllImport]? A. The marshalling stub is generated at compile time by a Roslyn source generator instead of at runtime by the JIT/CLR. Faster startup, smaller working set, and AOT-compatible.

Q2. What does "blittable" mean? A. A type whose managed and unmanaged representations are bitwise identical, so it can cross the boundary by direct copy without conversion. Primitives, pointers, and structs of blittable fields qualify.

Q3. How does bool cross the boundary? A. Default is Win32 BOOL (4 bytes); native APIs may expect 1 byte. Use [MarshalAs(UnmanagedType.U1)] or [MarshalAs(UnmanagedType.I1)] to control width.

Q4. When do you need GCHandle instead of fixed? A. When the native code retains the pointer beyond the call's scope. fixed only pins within the block; GCHandle.Alloc(GCHandleType.Pinned) survives until you Free().

Q5. What's [UnmanagedCallersOnly]? A. Marks a method as callable only from native code with a specified calling convention. Required for native function-pointer scenarios and NativeAOT exports.

Q6. Why use SafeHandle instead of IntPtr? A. GC-tracked, finalizable, exception-safe. The runtime guarantees ReleaseHandle runs on cleanup, even if the consuming code forgets to dispose. Raw IntPtr leaks on exception.

Q7. What does SetLastError = true do? A. After the call, the runtime captures GetLastError() and exposes it via Marshal.GetLastWin32Error() — without it, the value can be clobbered by intervening calls.

Q8. How do you call a varargs C function (int printf(const char* fmt, ...))? A. Mark the P/Invoke as [CallingConvention.Cdecl] and pass arguments as object[] via Marshal-friendly types, or use multiple [LibraryImport] overloads with explicit signatures. Varargs are awkward; prefer wrapper functions.

Gotchas / common mistakes

• Wrong CallingConvention → stack corruption (Cdecl vs StdCall).
• Mismatched struct layout (Pack, padding) → silent data corruption.
• Forgetting SetLastError = true and reading stale errno.
• Using string parameter without specifying StringMarshalling.
• Passing a managed array to a callback and expecting it to stay put across callback returns.
• Calling P/Invoke on a finalizer thread — finalizer must not block on native calls that take locks.
• Forgetting [UnmanagedCallersOnly] → callback won't bind correctly in AOT.
• Marshal.AllocHGlobal without matching FreeHGlobal → native leak invisible to GC.

Further reading

• LibraryImport source generator
• P/Invoke fundamentals
• Marshalling rules
• SafeHandle class
• Microsoft.Windows.CsWin32
• Function pointers in C#