Brokers Comparison

Key Points

• Azure Service Bus — managed; queues + topics; sessions for ordering; first-class .NET support. Azure shop default.
• RabbitMQ — open-source AMQP; flexible exchanges; clustered. Mature, dependable. Lower throughput than Kafka.
• Apache Kafka — log-based; high throughput; replay; ordered per partition. Streaming + event-sourcing.
• Azure Event Hubs — Kafka-compatible (mostly); managed; for telemetry/event streams.
• AWS SQS / SNS / MSK — AWS equivalents. SQS = simple queue; SNS = pub/sub; MSK = managed Kafka.
• Choosing: queues for work distribution; topics for fan-out; logs (Kafka/EH) for streaming + replay; managed > self-hosted unless you need maximum control.

Concepts (deep dive)

Broker categories

Azure Service Bus

builder.Services.AddAzureClients(c =>
{
    c.AddServiceBusClient(connStr);
});

// Send
var sender = client.CreateSender("orders");
await sender.SendMessageAsync(new ServiceBusMessage(json) { SessionId = customerId.ToString() });

// Receive (with sessions for ordering)
var sessionReceiver = await client.AcceptNextSessionAsync("orders");
var msg = await sessionReceiver.ReceiveMessageAsync();

Strengths: managed, premium tier offers low latency, RBAC via managed identity, sessions for ordering, dead-letter built-in, scheduled messages, message-time-to-live, transactions across queues/topics.

Weaknesses: cost; Azure-only.

Tier picks: Standard for most; Premium for low-latency, dedicated capacity; Geo-DR for disaster recovery.

RabbitMQ

Open-source. AMQP 0.9.1 protocol (AMQP 1.0 plugin available).

var factory = new ConnectionFactory { HostName = "rabbitmq" };
using var conn = await factory.CreateConnectionAsync();
using var ch = await conn.CreateChannelAsync();
await ch.ExchangeDeclareAsync("orders", ExchangeType.Topic);
await ch.QueueDeclareAsync("processor");
await ch.QueueBindAsync("processor", "orders", "order.#");

await ch.BasicPublishAsync("orders", "order.placed", body: payload);

Strengths: flexible exchange types (direct, topic, fanout, headers); mature; clustered; on-prem.

Weaknesses: throughput ceiling (~50K msg/s/broker); no replay; ordering only per-queue with single consumer.

Plugins: management UI, federation, shovel, streams.

Apache Kafka

Distributed log. Topics partitioned; consumers read at their own offset.

var producer = new ProducerBuilder<string, string>(new ProducerConfig { BootstrapServers = "kafka:9092" }).Build();
await producer.ProduceAsync("orders", new Message<string, string> { Key = orderId, Value = json });

var consumer = new ConsumerBuilder<string, string>(new ConsumerConfig
{
    BootstrapServers = "kafka:9092", GroupId = "processor", EnableAutoCommit = false
}).Build();
consumer.Subscribe("orders");
while (!ct.IsCancellationRequested)
{
    var r = consumer.Consume(ct);
    await Process(r.Message);
    consumer.Commit(r);
}

Strengths: massive throughput (millions msg/s); replay (read from offset 0); ordered per-partition; ecosystem (Kafka Streams, Connect, Schema Registry); event sourcing.

Weaknesses: operational complexity (was Zookeeper, now KRaft); steep learning curve; no priority queue.

Use cases: clickstream, IoT, change-data-capture, event sourcing.

Azure Event Hubs

Managed log. Kafka-compatible endpoint (use Confluent.Kafka client). Optimized for high-throughput ingestion.

// Same Kafka code; different bootstrap servers
new ProducerConfig { BootstrapServers = "myhub.servicebus.windows.net:9093" /* + SASL/SSL */ }

Strengths: managed; auto-scale; Capture (auto-archive to blob).

Weaknesses: not 100% Kafka API; cost.

AWS SQS / SNS

• SQS: simple queue. Standard (at-least-once, no order) or FIFO (exactly-once-ish, ordered). Visibility timeout for in-flight messages.
• SNS: pub/sub. Fanout to SQS, Lambda, HTTP, email.
• SQS + SNS: typical pub/sub topology — SNS topic → multiple SQS queues.

// AWS SDK
var client = new AmazonSQSClient();
await client.SendMessageAsync(new SendMessageRequest { QueueUrl = url, MessageBody = body });

AWS MSK / MSK Serverless

Managed Kafka on AWS. Less ecosystem polish than Confluent Cloud.

Confluent Cloud

Hosted Kafka by the Kafka company. Most ecosystem features. Multi-cloud.

Other

• Apache Pulsar: log + queue hybrid; geo-replication first-class.
• NATS / NATS JetStream: lightweight; cloud-native.
• Redis Streams: when you already have Redis; small workloads.

Choosing matrix

Latency

Service Bus Premium: ~5-10ms
RabbitMQ in-DC:       ~1-5ms
Kafka:               ~5-50ms (batching)
SQS:                 ~10-50ms

For sub-ms, you're using HTTP/gRPC, not messaging.

Throughput

RabbitMQ:   ~50K msg/s per broker
Service Bus: ~100K msg/s per namespace
Kafka:      ~millions msg/s per cluster
Event Hubs: ~millions events/s

Cost

Pay-per-message (cloud) varies wildly. For high-volume, self-hosted Kafka often cheaper. For business apps with modest volume, managed wins.

Operational complexity

Easiest →  SQS, SNS, Service Bus (managed)
           Event Hubs, Confluent Cloud (managed Kafka)
           RabbitMQ (clustered)
           Kafka self-hosted
Hardest

Common features comparison

.NET client maturity

Abstractions

MassTransit and Wolverine abstract over brokers — switch with config. Worth using even with one broker for the saga + outbox + retry features.

Code: correct vs wrong

❌ Wrong: ignoring DLQ

Failed messages stay in retry forever; consumer can't make progress.

✅ Correct: configure DLQ + max delivery count

// Service Bus
new ServiceBusReceiverOptions { ReceiveMode = ServiceBusReceiveMode.PeekLock }
// queue config: MaxDeliveryCount = 5; auto-DLQ on exceed

❌ Wrong: Kafka without partition key

producer.ProduceAsync("orders", new Message { Value = json });   // no key → random partition

Order events for one customer scattered across partitions.

✅ Correct: key by entity

producer.ProduceAsync("orders", new Message { Key = customerId, Value = json });

Design patterns for this topic

Pattern 1 — "Match topology to use case"

• Intent: queue for work; topic for fanout; log for replay.

Pattern 2 — "Managed broker default"

• Intent: ops cost dominates broker cost.

Pattern 3 — "Abstract via MassTransit/Wolverine"

• Intent: broker-agnostic apps.

Pattern 4 — "Schema registry for events"

• Intent: evolution without breakage.

Pattern 5 — "DLQ + monitoring + replay"

• Intent: poison-message recovery.

Pros & cons / trade-offs

When to use / when to avoid

• Use Service Bus for Azure business apps.
• Use Kafka/Event Hubs for streaming/event-sourcing.
• Use RabbitMQ for on-prem with complex routing.
• Avoid self-hosted Kafka unless you have ops capacity.

Interview Q&A

Q1. Service Bus vs RabbitMQ? SB managed, Azure-only, rich features. RabbitMQ open-source, flexible exchanges, on-prem-friendly.

Q2. Kafka vs Event Hubs? EH is Kafka-compatible managed offering. Same API for most uses; managed by Microsoft.

Q3. Why Kafka over queues? Replay (read from any offset); high throughput; partitioned ordering; streaming ecosystem.

Q4. What's a partition? Kafka topic = collection of ordered logs (partitions). Order preserved per partition; not across.

Q5. SQS Standard vs FIFO? Standard: at-least-once, no order. FIFO: ordered + dedup; lower throughput.

Q6. Service Bus sessions? Group messages by session ID; same session always to same consumer; ordered.

Q7. Why DLQ? Failed messages don't block live queue. Manual fix + replay.

Q8. Why partition key? Ensures related events stay ordered (same customer's order events on same partition).

Q9. RabbitMQ exchange types? Direct (exact key), Topic (wildcard), Fanout (broadcast), Headers (header match).

Q10. Confluent Cloud? Hosted Kafka by Kafka authors. Best ecosystem; cross-cloud.

Q11. MassTransit benefit even with one broker? Saga, outbox, retry policies, request-reply, test harness.

Q12. Cost optimization in cloud brokers? Batching, premium vs standard tiers, partition counts (EH/Kafka).

Gotchas / common mistakes

• ⚠️ Kafka without partition key — random distribution; ordering lost.
• ⚠️ No DLQ — poison messages block.
• ⚠️ Self-hosted broker without ops capacity.
• ⚠️ Service Bus sessions without sticky consumers.
• ⚠️ Mixing event-sourcing in queue brokers — no replay.

Further reading

• Azure Service Bus docs
• Apache Kafka docs
• Azure Event Hubs
• RabbitMQ docs
• AWS SQS docs