Streaming Chat — IAsyncEnumerable, Minimal APIs, and SignalR

🗓️ Last updated: August 2026

It is 23:47 on a Thursday. A senior platform engineer types why is the warehouse webhook 502-ing into the internal copilot built on top of a .NET 10 + Microsoft Agent Framework service. The cursor blinks. Two hundred and eighty milliseconds later the first token lands on the page. The answer streams in, word by word, for forty-one seconds. By the time the model finishes the engineer has already opened a second tab on the webhook log, read three lines, and stopped reading the assistant output. The shape of that conversation has almost nothing to do with the model's intelligence and almost everything to do with the transport layer underneath it. The first token arrived early enough to hold attention. The connection stayed open without a proxy buffering the chunks. The cancellation token fired the moment the engineer's tab lost focus and the model stopped spending budget on a paragraph nobody would ever read.

The architect's job on the .NET side of that interaction is not the model. It is the discipline of the pipe. .NET has had the right streaming primitive — IAsyncEnumerable<T> — since C# 8 and .NET Core 3.0, more than three years before streaming-chat became the default UX. The architect's choice in 2026 is the transport above it, and the cost of getting it wrong is paid in first-token latency, in dropped cancellations, in stalled spinners behind misconfigured proxies, and in billing line items for paragraphs the user closed the tab on at the second token.

What this article is NOT

This is not a tutorial on JavaScript EventSource, the WHATWG fetch streams API, or the Blazor server-side rendering pipeline. It is not a benchmark of SSE versus WebSockets versus HTTP/2 server push — the honest answer is "depends on direction, depends on proxy, and you almost certainly want SSE for one-way." It is not a SignalR primer.

It is an architect's read on IAsyncEnumerable<T> as the language-level streaming primitive on .NET, on Microsoft.Extensions.AI's and the Microsoft Agent Framework's streaming surface as the LLM-side adapter, and on the two transport surfaces ASP.NET Core ships out of the box — Minimal APIs with Server-Sent Events and SignalR streaming hubs — together with the failure modes that make a streaming endpoint quietly cost more than the model call it wraps.

Thesis

Token-by-token streaming is not a UX flourish. It is the contract between the model's slowest-to-emit byte and the user's willingness to wait. On .NET the streaming primitive is IAsyncEnumerable<T>. The LLM surface is IChatClient.GetStreamingResponseAsync returning IAsyncEnumerable<ChatResponseUpdate>. The agent surface is AIAgent.RunStreamingAsync returning IAsyncEnumerable<AgentResponseUpdate>. The transport above is either Server-Sent Events on a Minimal API endpoint via TypedResults.ServerSentEvents, or a SignalR streaming hub method that returns the same IAsyncEnumerable<T> straight to a typed client. The architect's job is to pick the right transport for the direction of the stream, propagate the cancellation token honestly from the browser down through the model call, keep the framing readable end-to-end without an intermediate proxy buffering it, and put a cost cap on the producer so a reconnect storm does not become a billing event.

What the classical .NET version looked like

The pre-streaming-chat .NET pattern was Task<string>. You sent a request, you awaited a complete response, you rendered it. Latency was hidden behind a spinner. If the answer took twelve seconds, the user stared for twelve seconds and then saw the whole answer at once. The streaming workaround in that world was a polling loop — the client polled /api/chat/status?id=... every five hundred milliseconds and the server stored partial output in a cache.

IAsyncEnumerable<T> shipped with C# 8 in September 2019, alongside await foreach and the WithCancellation extension. The [EnumeratorCancellation] attribute that closes the cancellation contract came shortly after with .NET 5 and C# 9 in November 2020. Microsoft.Extensions.AI exposes IChatClient.GetStreamingResponseAsync as a first-class member of the chat interface, alongside the non-streaming GetResponseAsync — the package's 10.x line ships with the .NET 10 wave. The Microsoft Agent Framework, when it consolidated Semantic Kernel and AutoGen in April 2026, kept the same shape one altitude up — AIAgent.RunStreamingAsync returns IAsyncEnumerable<AgentResponseUpdate>. The classical Task<string> surface still exists, but treating it as the primary surface for an interactive agent in 2026 is a decision to ship a polling architecture.

The streaming primitive on .NET

IAsyncEnumerable<T> is the language-level contract for an asynchronous sequence. Three pieces of the contract carry the whole article.

The first is [EnumeratorCancellation], the attribute from System.Runtime.CompilerServices that marks one CancellationToken parameter on an async IAsyncEnumerable<T> method as the merge point. The compiler then routes the token that the caller passes to WithCancellation(ct) into that parameter, on top of whatever the method already received. Without the attribute, a caller doing await foreach (var x in source.WithCancellation(ct)) sees no signal from ct reach the body of the iterator.

The second is ConfigureAwait(false) on the awaits inside a library-level streaming method. The synchronisation context capture is the wrong default for library code that does not need to marshal back to a UI thread, and on a streaming endpoint that emits hundreds of updates per response the cost is real. The Agent Framework's own AIAgent.RunStreamingAsync uses ConfigureAwait(false) on the inner await foreach. Library code should match.

The third is the shape of the unit you yield. Microsoft.Extensions.AI's ChatResponseUpdate is a delta — a single chunk that carries Role, Text, Contents, MessageId, ResponseId, ConversationId, CreatedAt, FinishReason, ModelId, and a ContinuationToken for stream resumption. Updates are not snapshots; they are slices. The extension ToChatResponseAsync collapses them into a single ChatResponse when server-side code needs the assembled message. The agent altitude is the same shape one level up — AgentResponseUpdate carries tool-call events, intermediate steps, and the final message.

The trap that catches juniors here is treating updates as snapshots and overwriting the rendered message on every chunk. They are deltas. The renderer concatenates.

The LLM streaming surface, two altitudes

IChatClient.GetStreamingResponseAsync(IEnumerable<ChatMessage> messages, ChatOptions? options, CancellationToken cancellationToken) returns IAsyncEnumerable<ChatResponseUpdate>. This is the M.E.AI substrate the Series 4 Article 2 covered as the unified client layer. The middleware pipeline on ChatClientBuilder — UseLogging, UseOpenTelemetry, UseDistributedCache, UseFunctionInvocation — wraps the streaming method the same way it wraps GetResponseAsync. The OpenTelemetry middleware emits a span whose lifetime spans the entire enumeration, not just the call, which is correct because the streaming call is the enumeration.

AIAgent.RunStreamingAsync(IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) returns IAsyncEnumerable<AgentResponseUpdate>. The agent altitude adds the session parameter Article 4 introduced — the same AgentSession that survives the pod restart can be the streaming context for the next response. The signature on RunStreamingAsync carries [EnumeratorCancellation] on the cancellation parameter, the way every library-grade async iterator should.

The architect picks the altitude by whether the streaming endpoint needs the tool-call surface and the agent-session boundary, or whether it is happy with a chat-level delta. An interactive copilot endpoint that exposes one model and no tools can target IChatClient directly. A multi-step agent endpoint that runs tools between updates targets AIAgent.

Minimal API SSE in .NET 10

ASP.NET Core 10 shipped a typed Server-Sent Events result on TypedResults and Results. The shape — straight from the public API surface — is one of two overloads, ServerSentEvents<T>(IAsyncEnumerable<SseItem<T>> values) for full control over event type and event id, or ServerSentEvents<T>(IAsyncEnumerable<T> values, string? eventType = null) for the common case of plain typed events. The result wires up the HTTP framing the way an SSE consumer expects — Content-Type: text/event-stream, Cache-Control: no-cache,no-store, Pragma: no-cache, Content-Encoding: identity — and it disables output buffering through the response-body feature so chunks are flushed as they arrive. Cancellation flows through HttpContext.RequestAborted, which the framework passes to the underlying SseFormatter.WriteAsync. For non-string item types the result JSON-serialises with the application's configured JsonOptions; for IAsyncEnumerable<SseItem<string>> it skips JSON and writes the string straight.

The endpoint shape is small enough to read in one breath.

app.MapPost("/chat/stream", (
    [FromBody] ChatRequest request,
    IChatClient chat,
    HttpContext http,
    CancellationToken ct) =>
{
    var messages = new List<ChatMessage>
    {
        new(ChatRole.System, "You are an internal platform copilot."),
        new(ChatRole.User, request.Prompt),
    };

    async IAsyncEnumerable<SseItem<string>> Stream(
        [EnumeratorCancellation] CancellationToken token)
    {
        await foreach (var update in chat
            .GetStreamingResponseAsync(messages, options: null, token)
            .ConfigureAwait(false))
        {
            if (update.Text is { Length: > 0 } text)
            {
                yield return new SseItem<string>(text, eventType: "token");
            }

            if (update.FinishReason is { } reason)
            {
                yield return new SseItem<string>(reason.Value, eventType: "done");
            }
        }
    }

    return TypedResults.ServerSentEvents(Stream(http.RequestAborted));
});

Four things in that block carry the article's architecture. HttpContext.RequestAborted is the canonical cancellation source for an HTTP request — when the browser closes the tab, the proxy drops the connection, or the client cancels the fetch, this token fires. [EnumeratorCancellation] on the iterator's token parameter merges it with whatever the framework passes through. ConfigureAwait(false) is on the inner await foreach because the iterator is library-grade code with no business marshalling back to a sync context. And the iterator emits two event types — token for the chunks and done for the final reason — which lets the browser EventSource route them through different handlers without parsing the body.

The non-obvious detail is what happens after the model finishes generating but the user has already closed the tab. The token from RequestAborted fires. The await foreach throws OperationCanceledException. The streaming method on the chat client is contractually required to honour the token and stop pulling from the provider. If the underlying provider implementation does not honour cancellation — and a real number of community provider packages did not in 2024 and early 2025 — the model keeps generating tokens the user will never see, and the bill keeps ticking. The architect's responsibility is to verify the provider's cancellation behaviour by integration test, not by reading the docs.

SignalR streaming hub methods

Server-Sent Events is one-way, server-to-client. The moment the streaming endpoint needs the user to send messages mid-stream — a stop button, a clarification, an interrupt — SSE is no longer enough. SignalR is the other transport surface ASP.NET Core ships, and a streaming hub method on SignalR is a hub method that returns IAsyncEnumerable<T>, ChannelReader<T>, Task<IAsyncEnumerable<T>>, or Task<ChannelReader<T>>. The framework recognises the return type and turns the method into a streaming invocation automatically.

public sealed class ChatHub(IChatClient chat) : Hub
{
    public async IAsyncEnumerable<string> StreamReply(
        string prompt,
        [EnumeratorCancellation] CancellationToken cancellationToken)
    {
        var messages = new List<ChatMessage>
        {
            new(ChatRole.System, "You are an internal platform copilot."),
            new(ChatRole.User, prompt),
        };

        await foreach (var update in chat
            .GetStreamingResponseAsync(messages, options: null, cancellationToken)
            .ConfigureAwait(false))
        {
            if (update.Text is { Length: > 0 } text)
            {
                yield return text;
            }
        }
    }
}

The CancellationToken parameter is not the connection's lifetime token — it is the per-invocation token that SignalR fires when the client unsubscribes from this particular stream. On the .NET client, the call is hubConnection.StreamAsync<string>("StreamReply", prompt, ct) returning an IAsyncEnumerable<string> the consuming code reads with await foreach. The browser JavaScript client uses connection.stream("StreamReply", prompt).subscribe({ next, error, complete }) and disposes the subscription to send the cancellation upstream.

The architectural choice between SSE and SignalR comes down to two questions. First, does the stream need to carry messages from the client back to the server while it is open, or does it only need to deliver tokens? Second, does the deployment need to traverse a proxy or CDN that aggressively buffers text/event-stream with no way to set X-Accel-Buffering: no? SignalR's WebSocket transport sidesteps both. SSE pays the proxy tax every time.

Cancellation propagation, end to end

The cancellation contract on a streaming chat endpoint is one token, flowed through five layers without being dropped at any of them. The browser closes the tab. The proxy notices and closes the connection. The framework fires HttpContext.RequestAborted or the SignalR per-invocation token. The endpoint passes that token to [EnumeratorCancellation]. The await foreach flows it to IChatClient.GetStreamingResponseAsync, which passes it to the underlying provider HTTP call. The provider stops the model response if its API supports it.

Any layer that drops the token leaks money. Three common drops: a try/catch (Exception) that swallows OperationCanceledException and continues iterating; a hand-rolled SSE endpoint with Response.WriteAsync that does not pass the request-aborted token down; a SignalR hub method whose CancellationToken parameter is missing [EnumeratorCancellation]. All three look correct on review. All three quietly burn provider budget on responses nobody is watching.

Buffering, backpressure, and the proxy tax

ServerSentEventsResult<T> disables output buffering on the response body. The browser flushes its receive buffer on each chunk. The middle layer — proxy, gateway, WAF, CDN — is where the streaming contract breaks in production.

Nginx with default settings buffers text/event-stream and delivers in eight-kilobyte chunks. The fix is X-Accel-Buffering: no on the response header. AWS Application Load Balancer streams correctly on HTTP/1.1 with keep-alive on. Cloudflare respects SSE when Cache-Control: no-cache is set, which ServerSentEventsResult<T> already does. Azure Front Door requires the origin to send Cache-Control explicitly. The architect verifies this with an end-to-end test from a real browser through the production edge, not from a curl on the cluster.

Backpressure on a streaming method is implicit in IAsyncEnumerable<T> — the producer's yield return does not return until the consumer's MoveNextAsync completes. The producer cannot outrun the consumer. The trap is the channel-based variant — when the hub method writes to Channel.CreateUnbounded<T> and returns the reader, the producer outruns the consumer indefinitely and the channel grows. The fix is Channel.CreateBounded<T> with BoundedChannelFullMode.Wait, which makes WriteAsync await capacity and reintroduces backpressure.

The cost surface

Streaming changes the cost shape of an LLM endpoint. A non-streaming call is paid for once — the full response, billed at the end. A streaming call is paid for the same number of output tokens, but the cost is committed as the response generates, and the failure mode is asymmetric. If the client closes the tab at the second token of a five-hundred-token reply and the cancellation is honoured at every layer, the architect pays for two tokens. If cancellation is dropped at any layer, the architect pays for five hundred. Multiply by a reconnect storm — a flaky network that drops and reopens the SSE connection every five seconds — and the bill on a copilot endpoint can run an order of magnitude over the steady-state number for the same number of useful answers delivered.

This is where Series 3 Article 6 — Cost Guardrails for LLM Systems — earns its keep. The streaming endpoint sits behind the same per-request token budget, the same per-tenant rate limit, the same per-session daily cap as the non-streaming endpoint. The ChatClientBuilder middleware pipeline is the right place to enforce them, because the budget enforcement wraps the streaming method exactly the way it wraps the non-streaming one.

Failure modes

The streaming-chat failure modes are not the model's. They are the pipe's.

The first and worst is dropped cancellation. Any layer between the browser and the provider that swallows the token leaks money. The damage is silent — the response generated successfully, the user closed the tab, the bill ticked up. There is no error in the log to grep.

The second is the proxy buffer. The model emitted the first token at 280 ms; nginx held it until the eight-kilobyte buffer filled; the user saw nothing for thirty seconds and reloaded the page; the reload kicked off a second streaming call; both calls billed.

The third is the swallowed enumerator exception. The iterator throws on a transient provider 502; the outer try/catch (Exception) logs and continues; the next iteration calls the disposed enumerator and the connection drops; the client retries; the cycle repeats until a circuit breaker fires several minutes later.

The fourth is the reconnect storm. A flaky network on the client closes and reopens the SignalR connection every few seconds; each reconnect re-subscribes and starts a new model call; cost runs ten times steady state. The fix is an idempotency key on the streaming invocation so the server can detect "this is the same stream the client was just on" and resume from a continuation token rather than start fresh.

The fifth is the unbounded channel. The hub method writes to Channel.CreateUnbounded<T>; the producer outruns the consumer; memory grows linearly with stream length; the pod OOMs on a long conversation.

The sixth is JSON serialisation on the hot path. TypedResults.ServerSentEvents JSON-serialises every non-string event using the configured JsonOptions. The per-event cost is small individually and adds up over a hundred-thousand-event day. Using IAsyncEnumerable<SseItem<string>> and pre-encoding at the source skips the serialiser.

The seventh is the cancellation token at the wrong altitude. A hub method that declares CancellationToken without [EnumeratorCancellation] receives a token, but the token the framework merges from WithCancellation at the call site is silently discarded. The compiler does not warn. The endpoint looks correct. The cancellation never fires.

Architect's checklist

🗓️ Every public IAsyncEnumerable<T> method has [EnumeratorCancellation] on its cancellation-token parameter.

🗓️ Every await inside a streaming iterator uses ConfigureAwait(false) when the method is library-grade code.

🗓️ The SSE endpoint reads HttpContext.RequestAborted and passes that token to the chat-client streaming call.

🗓️ The SignalR streaming hub method accepts a CancellationToken parameter marked with [EnumeratorCancellation] so client-side unsubscribe fires.

🗓️ The reverse proxy in front of the SSE endpoint is verified to not buffer text/event-stream — X-Accel-Buffering: no on nginx, equivalent header on the edge.

🗓️ Channel-based producer paths use Channel.CreateBounded<T> with BoundedChannelFullMode.Wait, never CreateUnbounded.

🗓️ The integration test suite includes a "client cancels at the second token" scenario that verifies the provider stopped generating.

🗓️ Streaming endpoints enforce the same per-request token budget, per-tenant rate limit, and per-session daily cap as the non-streaming endpoints.

🗓️ Reconnect-aware streaming uses an idempotency key plus ChatResponseUpdate.ContinuationToken for resumption rather than restarting the model call.

🗓️ OpenTelemetry middleware on ChatClientBuilder is in the pipeline so the span scope covers the entire enumeration, not just the call.

🗓️ JSON serialisation on the SSE hot path is avoided where it can be — pre-encode to IAsyncEnumerable<SseItem<string>> when the event payload is small and uniform.

🗓️ The provider package's cancellation behaviour is verified end-to-end against the provider's billing console, not asserted from the docs.

A mental model

The model is not a function that returns a string. It is a sequence that yields words. .NET already had the right primitive — IAsyncEnumerable<T> — three years before streaming-chat became the dominant LLM interaction model. The transport above is a choice between two ASP.NET Core surfaces — SSE on a Minimal API for the one-way case, SignalR for the bidirectional and fan-out cases — and the failure modes are not in the model, they are in the pipe.

The well-architected version of this surface is invisible. The user sees only the word arriving. The architect's job is to keep the connection that brought it boring.

What's next

Article 6 closes Series 4 — A Reference Architecture for the .NET Agentic Service. It assembles Articles 1 through 5 into one canonical layered service — IChatClient from Article 2 as the substrate, structured outputs from Article 3 as the contract, the hosted-service lifecycle from Article 4 as the runtime, the streaming surface from this article as the interactive face, and the DI lifetimes, configuration sources, and observability pipeline that bind them together. It is the deck a senior architect should be able to defend before the first commit on a new .NET agentic service in late 2026.

Series 4 progress