Skip to content

Orchestration: Concurrent

Key Points

  • Multiple agents run in parallel on the same input. Outputs aggregated.
  • Use for independent perspectives: multiple reviewers, multiple search strategies, multiple expert opinions.
  • An aggregator agent typically synthesizes the final answer.
  • Faster than sequential; same total token cost.

Concept

                       [User input]
                            │
        ┌───────────────────┼───────────────────┐
        ▼                   ▼                   ▼
 [Code Reviewer]    [Security Reviewer]  [Perf Reviewer]
        │                   │                   │
        └───────────┬───────┴───────────────────┘
                    ▼
            [Aggregator Agent]
                    │
                    ▼
            [Final synthesis]

Code

var codeReviewer = new ChatClientAgent(chat) { Instructions = "Review for code quality." };
var securityReviewer = new ChatClientAgent(chat) { Instructions = "Review for security issues." };
var perfReviewer = new ChatClientAgent(chat) { Instructions = "Review for performance issues." };

var aggregator = new ChatClientAgent(chat)
{
    Instructions = "Combine reviewer feedback into a single prioritized list of issues."
};

var orchestration = new ConcurrentOrchestration(
    parallel: [codeReviewer, securityReviewer, perfReviewer],
    aggregator: aggregator);

var resp = await orchestration.InvokeAsync(prDiff);

Implementation pattern

public class ConcurrentOrchestration(IList<Agent> parallelAgents, Agent aggregator)
{
    public async Task<AgentResponse> InvokeAsync(string input, CancellationToken ct = default)
    {
        var messages = new List<ChatMessage> { new(ChatRole.User, input) };

        var tasks = parallelAgents.Select(a => a.InvokeAsync(messages, ct));
        var results = await Task.WhenAll(tasks);

        var combined = new List<ChatMessage>(messages);
        foreach (var r in results)
            combined.Add(new ChatMessage(ChatRole.Assistant, r.Messages.Last().Text!) { AuthorName = r.AgentName });

        return await aggregator.InvokeAsync(combined, ct);
    }
}

Use cases

  • Multi-reviewer code: different perspectives.
  • Multi-strategy search: vector + keyword + graph.
  • Multi-expert decisions: legal + finance + operations review.
  • Brainstorming: multiple agents propose; aggregator dedupes.

Aggregator strategies

// Strategy 1: Best-of
"Pick the most useful reviewer's response."

// Strategy 2: Synthesize
"Combine into one coherent response."

// Strategy 3: Vote
"For each issue, count reviewer agreement. Surface those with 2+ agreement."

// Strategy 4: Tag and group
"Group by category; show all issues per category."

Cost

N parallel agents × token usage + aggregator

Same as sequential but wall-clock faster (parallel calls).

Bounded parallelism

If you have many agents:

var sem = new SemaphoreSlim(5);
var results = await Task.WhenAll(agents.Select(async a =>
{
    await sem.WaitAsync(ct);
    try { return await a.InvokeAsync(messages, ct); }
    finally { sem.Release(); }
}));

Avoid hammering rate limits.

Streaming

Each parallel agent streams independently; aggregator runs after all complete:

await foreach (var update in orchestration.InvokeStreamingAsync(input))
{
    Console.WriteLine($"[{update.AgentName}] {update.Text}");
}

When NOT concurrent

  • Agents depend on each other → sequential or handoff.
  • Single-step task → just one agent.
  • Cost-sensitive → may not need N opinions.

Comparison

Pattern Use
Sequential Linear pipeline
Concurrent Independent perspectives
Group chat Conversational consensus
Handoff Routing-based

Senior considerations

  • Diversify perspectives: don't have 5 agents with similar instructions.
  • Aggregator is critical: weak aggregator wastes the parallelism.
  • Cost control: budget; not every request needs 5 opinions.
  • Consistency: parallel calls may give conflicting answers; aggregator must handle.

Cross-references