Search for a command to run...
Your senior dev has spent three days hand rolling wrapper classes for logging, caching, and provider switching around the OpenAI SDK. Microsoft already shipped that, called it boring, and put it in the box with the rest of .NET. Surprise.
No comments yet. Be the first to comment.
The architectural patterns of Series 1-3, rebuilt on the .NET runtime. Semantic Kernel, Microsoft.Extensions.AI, hosted agents, streaming.
🗓️ Last updated: August 2026 The architect inherits a brownfield .NET 10 service that classifies incoming customer-support emails into six ticket categories, extracts the order number if there is on
🗓️ Last updated: August 2026 The architect inherits a brownfield .NET 10 service that classifies incoming customer-support emails into six ticket categories, extracts the order number if there is on
You bet a year of roadmap on Semantic Kernel. Then Microsoft renamed the bet. Here is what actually changed, what did not, and why the architects who shrugged are the ones who read the contract instead of the brand.
A recursive agent spent the weekend asking itself the same question, failing the same check, and billing you for both. No error fired, because no metric you watched had the word stop in it. This article is about putting it there.
The dashboards are healthy, the error rate is zero, nothing shipped this week, and the model just told a customer something expensive and wrong. There is no commit to revert. Welcome to the incident that classical on call was never designed for.
🗓️ Last updated: August 2026
The architect picks up a brownfield .NET 10 service in the second week of August. It is a customer support backend that has been calling OpenAI directly through the official OpenAI .NET SDK since the project started. The ticket on the board this sprint reads: add structured logging on every model call, emit OpenTelemetry spans matching the rest of the service, support Azure OpenAI as a secondary provider for a regulated tenant, and put a cache in front of the deterministic prompts the support classifier sends a few thousand times an hour. Four tickets. One sprint. The team's senior dev has been writing the wrapper classes by hand for three days, and the abstractions are starting to look like a small framework of their own. Nobody asked for a small framework.
This is the recurring shape of a brownfield .NET LLM codebase in 2026. The application code talks directly to a vendor SDK. The cross cutting concerns, logging, telemetry, caching, function calling, provider switching, get bolted on as wrapper classes that nobody wants to own. The team is six months from where they need to be and they cannot rewrite the world.
The answer is not a new framework. The answer is Microsoft.Extensions.AI, a small, deliberately boring set of abstractions that ships in the same .NET 10 stack as Microsoft.Extensions.Logging and Microsoft.Extensions.Caching. The value of M.E.AI is not what it adds. Its value is what it asks you to standardize on, and what it lets you compose on top of that standard. Boring, in this case, is the whole pitch.
This is not a Microsoft.Extensions.AI tutorial. The Microsoft Learn docs are excellent at that, and a blog should not try to replace them. It is not a comparison of Semantic Kernel and M.E.AI; Article 1 of this series covered that consolidation. It is not an argument that you should rewrite working OpenAI SDK code today.
It is an architect's read on why M.E.AI is the load bearing layer underneath the rest of the .NET LLM stack, and how to design the rest of your service so that the layer earns its keep.
Microsoft.Extensions.AI is the .NET LLM ecosystem's ASP.NET Core moment. It is a small set of interfaces, IChatClient, IEmbeddingGenerator, ChatMessage, ChatOptions, and a middleware pipeline pattern that composes cross cutting concerns the same way ASP.NET Core's request pipeline composes HTTP middleware. The interfaces are not the point. The pipeline is. An architecture that takes the pipeline seriously inherits logging, telemetry, caching, function invocation, and provider switching as configuration concerns rather than code concerns. An architecture that treats M.E.AI as yet another SDK wrapper pays the cost of the abstraction without collecting the benefit.
ASP.NET Core's middleware pipeline is the prior art every .NET architect already understands. A request enters the pipeline, passes through ordered middleware components, hits an endpoint, and returns through the same components on the way out. Authentication, authorization, exception handling, response compression, request logging: every cross cutting concern in the HTTP stack is a middleware component composed by configuration. The application code at the bottom of the stack handles only the domain logic.
The same pattern, applied to model calls, is what Microsoft.Extensions.AI shipped. A chat call enters a pipeline, passes through ordered middleware (logging, OpenTelemetry, caching, function invocation), hits the actual provider client, and the response returns through the same components. Application code at the bottom calls IChatClient.GetResponseAsync and gets back a ChatResponse. The cross cutting concerns are configured at composition time and disappear from the application's reading view.
The architect who has internalized the ASP.NET Core middleware model has already internalized the M.E.AI middleware model. The vocabulary is the same. The abstraction is the same. Only the pipeline contents change. If you have ever written app.UseAuthentication() and not thought twice about it, you already know how this ends.
Microsoft.Extensions.AI 10.6.0, the current stable as of mid 2026, GA in the .NET 10 stack, exposes a deliberately small surface. The four types worth knowing on day one are below. Everything else in the namespace either supports these four or is a convenience over them.
IChatClient. The abstraction over any chat completion style model. The interface exposes two methods that matter:
Task<ChatResponse> GetResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
CancellationToken cancellationToken = default);
IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
CancellationToken cancellationToken = default);
A non streaming call returns the full ChatResponse when the model finishes. A streaming call returns an IAsyncEnumerable<ChatResponseUpdate> that yields incremental updates as they arrive. The streaming method is the foundation for Article 5 of this series.
IEmbeddingGenerator. The corresponding abstraction for embedding models. Generic over input type and embedding type so the same interface covers text embeddings, image embeddings, and multimodal embeddings. The pattern matches IChatClient: pipeline composable, provider pluggable, deliberately small.
ChatMessage. The unit of conversation. A role (ChatRole.System, ChatRole.User, ChatRole.Assistant, ChatRole.Tool), a content payload, optional name and author identification. The same ChatMessage shape is produced by every provider adapter: OpenAI, Azure OpenAI, Anthropic via community adapter, Ollama via OllamaSharp. The application code never sees a vendor specific message type.
ChatOptions. Per call configuration: temperature, max tokens, top p, tool list, response format, stop sequences. The architectural point is that ChatOptions is per call, not per client. A single registered IChatClient can serve many call sites with different configurations. This is the difference between a model call being a configuration concern and being a wiring concern.
The composition surface is ChatClientBuilder. The pattern mirrors ASP.NET Core's IApplicationBuilder and WebApplicationBuilder: a fluent chain of Use* extension methods that wrap the inner client in middleware components, in declared order, returning a single composed IChatClient at the end. A representative composition looks like this:
services
.AddChatClient(provider => new OpenAIClient(apiKey)
.GetChatClient("gpt-4o").AsIChatClient())
.UseLogging()
.UseOpenTelemetry(sourceName: "MyApp.Chat")
.UseDistributedCache()
.UseFunctionInvocation();
Each Use* extension method comes from Microsoft.Extensions.AI. The semantics:
UseLogging wraps the inner client with a logging middleware that emits the request, the response, and any exceptions through ILogger. The category and verbosity are configurable. In production, the architect typically wants the request and response payloads at Debug level only, with timing and metadata at Information.
UseOpenTelemetry wraps the inner client with an OpenTelemetry middleware that emits spans and metrics aligned with the OpenTelemetry GenAI semantic conventions Series 3 Article 3 covered in detail. The middleware participates in the same System.Diagnostics.ActivitySource infrastructure the rest of the .NET service is already using. No separate wiring.
UseDistributedCache wraps the inner client with a caching middleware that hashes the inputs into a cache key, looks the response up in an IDistributedCache implementation (Redis, SQL Server, in memory), and short circuits the model call on a hit. The cache key composition and TTL are configurable. The architect's design responsibility is to be honest about which calls are actually cacheable, which is fewer than you would like.
UseFunctionInvocation wraps the inner client with a function calling middleware that recognizes tool call responses, dispatches them to the registered AIFunction instances, and feeds the tool results back into the model loop until the conversation reaches a terminal assistant response. This is the middleware that turns the model call from a single round trip into the agent loop everyone assumes is going to be hard to implement. It is not hard to implement. It is one method call.
Order matters. Logging and OpenTelemetry wrap function invocation so that each tool dispatch is traced and logged individually. Caching wraps function invocation if you want tool aware caching; caching wraps only the outer call if you want naive caching. That is the architect's choice, and the architect's choice to write down for the next person, who will otherwise reverse engineer it from a stack trace at midnight.
IChatClient is the abstraction. The providers are concrete adapters that surface their vendor SDK behind the interface. The architecture is the substrate; the provider is the bottom of the pipeline. As of mid 2026, the production ready adapter set on .NET is:
Microsoft.Extensions.AI.OpenAI, which wraps the official OpenAI .NET SDK. Adapter method: AsIChatClient on OpenAI.Chat.ChatClient.Microsoft.Extensions.AI.AzureAIInference, which wraps the Azure AI Inference SDK for Azure hosted models.Azure.AI.OpenAI SDK exposes its chat clients through the OpenAI adapter shape. In practice this is the most common configuration in Microsoft hosted production.Microsoft.Extensions.AI.Ollama preview package is deprecated; the OllamaSharp library implements IChatClient directly. Useful for local model development.Provider switching at the M.E.AI layer is a single registration change, the services.AddChatClient(...) line, with no impact on application code. The pipeline and the consumers below it are provider agnostic. The architect's discipline is to keep them that way: zero provider specific types reach the application layer. This is the one rule that, broken once, quietly undoes the entire abstraction.
Function calling, the mechanism by which the model invokes typed .NET methods, is the most consequential middleware in the pipeline. M.E.AI ships the convention.
The unit is AIFunction. The factory is AIFunctionFactory.Create. Any delegate, lambda, or method handle becomes an AIFunction: the factory inspects the parameter types, generates a JSON schema, picks up [Description] attributes from the method and its parameters, and returns an AIFunction you can attach to ChatOptions.Tools. A representative tool registration:
AIFunction getOrderStatus = AIFunctionFactory.Create(
(string orderId) => orderStatusService.LookupAsync(orderId),
new AIFunctionFactoryOptions
{
Name = "get_order_status",
Description = "Look up the current status of a customer order by ID."
});
var options = new ChatOptions { Tools = [getOrderStatus] };
var response = await chatClient.GetResponseAsync(messages, options);
With UseFunctionInvocation in the pipeline, the middleware handles the model's tool call response, executes getOrderStatus, feeds the result back into the conversation, and returns the final assistant message. The application code never parses a tool call by hand.
The architectural point is that a tool is now a typed C# method surfaced by metadata. No string parsing, no manual schema wiring, no provider specific tool format. The architect's job is to keep the tool methods at the right granularity, small enough to be model friendly, large enough to do one useful unit of work, and to recognize that every tool method is also part of the public contract of the agent.
A second architectural point gets lost in the convenience: a tool method is a C# method, which means it is unit testable as a C# method. The model layer is not in the test path. Tool methods should be tested in the same project, with the same conventions, as any other application service, and the agent level integration test that wires the tool into a real model call should be a separate, smaller suite. Confusing the two test surfaces produces unit tests that depend on a live model and integration tests that exercise no model behavior. Both are common failures in 2026 .NET LLM codebases. Both are avoidable by treating the tool method as ordinary code, because it is.
Article 1 noted that the Microsoft Agent Framework sits on top of M.E.AI's IChatClient. The natural question is why M.E.AI exists at all if the Agent Framework already wraps it.
The answer is that not every .NET LLM service is an agent. Many production services need a model call with caching, telemetry, and a tool or two, not a multi turn agent with handoffs, sessions, and orchestration. M.E.AI is the layer where those services live. The Agent Framework is the layer where agent services live. Both layers are first party. Both are part of the .NET stack. Picking the right altitude for the service is part of the architectural decision, and the altitude is reversible: a service that grows into an agent can adopt the Agent Framework without rewriting its M.E.AI substrate.
The corollary: a service whose only LLM interaction is a single classifier call per request should not be running the Agent Framework. M.E.AI plus a caching middleware is the right altitude and the simpler answer. Reaching for the agent framework here is the .NET equivalent of bringing a forklift to carry a sandwich.
Semantic Kernel, now in maintenance mode following the Microsoft Agent Framework's April 2026 GA, provides documented bridge extensions (AsChatClient, AsChatCompletionService) between its IChatCompletionService and M.E.AI's IChatClient. An existing Semantic Kernel codebase can introduce M.E.AI for new call sites without disturbing the existing Kernel composition; the two coexist, and the M.E.AI pipeline can wrap or be wrapped by Kernel mediated calls depending on which surface owns the call site.
The honest framing: M.E.AI is the substrate both Semantic Kernel and the Agent Framework now share. Code that goes through IChatClient directly survives any future framework decision. Code that goes through framework specific composition has a shorter half life. The architect's preference, for new code, should be to drop one layer below the framework whenever the framework is not doing useful work.
For an existing Semantic Kernel codebase, the practical migration pattern is not a rewrite. It is an additive one. New call sites land on IChatClient directly with the M.E.AI pipeline; existing Kernel mediated call sites stay where they are. Over time, as the Kernel composition becomes the smaller surface and the M.E.AI composition becomes the larger one, the center of gravity moves on its own. The architect's job is to make sure the new code lands in the right place, not to retroactively rewrite the old code. The two surfaces are designed to coexist precisely because Microsoft expects this transition to happen gradually across most production codebases.
The patterns that turn a healthy M.E.AI adoption into a fragile one, worst first.
ChatResponse on failure looks defensive and is catastrophic. The downstream caller treats the response as success, the model call silently failed, and the failure is invisible in telemetry. Mitigation: middleware lets exceptions propagate unless the architect has consciously decided on a fallback semantic that the caller can detect.UseDistributedCache caches against a hash of the inputs. A prompt that includes the current timestamp, the user's session ID, or any other per call identifier will produce a cache miss every time and a unique cache entry every time, wasting cache memory and producing zero hit rate. Mitigation: caching is opt in per call site, gated on the call site being demonstrably deterministic, and the cache key composition is reviewed.IChatClient because each one might be useful. Cost goes up, because every middleware allocates per call, and the failure surface multiplies. Mitigation: a minimum viable pipeline at each composition point, and a separate IChatClient registration for separate concerns rather than a maximalist one pipeline fits all composition.ChatOptions as a singleton. Building a single ChatOptions instance at startup and reusing it across calls produces behavior that is miserable to debug the day one call site mutates a field. Mitigation: ChatOptions is per call, constructed at the call site, immutable after construction.UseOpenTelemetry ships with every IChatClient composition that runs in production, full stop. The cost is negligible. The value is total.🗓️ Application code talks to IChatClient, never to a vendor SDK type.
🗓️ IChatClient is registered in DI through services.AddChatClient(...), with the provider adapter as the inner client.
🗓️ Provider specific types (OpenAIClient, ChatClient, vendor SDK message types) stop at the registration boundary.
🗓️ Every production IChatClient composition includes UseLogging and UseOpenTelemetry; OTel emits through the same ActivitySource as the rest of the service.
🗓️ UseDistributedCache is added only on call sites with reviewed cache key determinism.
🗓️ UseFunctionInvocation is added on call sites that use tools; tool functions are constructed via AIFunctionFactory.Create with explicit names and descriptions.
🗓️ ChatOptions is constructed per call, not reused across calls.
🗓️ Custom middleware lets exceptions propagate by default; any fallback semantic is written down and reviewed.
🗓️ The middleware order is documented at each composition point: what wraps function invocation, what wraps caching, what wraps the outer call.
🗓️ Streaming call sites use GetStreamingResponseAsync and treat the IAsyncEnumerable<ChatResponseUpdate> as the unit of cancellation.
🗓️ Provider adapter packages are pinned to a known stable version range; the adapter set is reviewed at each major release.
🗓️ IEmbeddingGenerator registrations follow the same pipeline discipline as IChatClient: embedding calls are equally observable, equally cacheable, equally provider agnostic.
🗓️ No application code depends on a Semantic Kernel or Agent Framework type for work that M.E.AI can do directly.
M.E.AI is to the .NET LLM stack what IHttpClientFactory was to the .NET HTTP stack. The interface is small. The composition surface is the value. The pipeline is the thing that earns the abstraction its place in the dependency graph. Every cross cutting concern that used to be a hand written wrapper class becomes a one line middleware registration. Every provider switch becomes a one line composition change. Every observability requirement becomes a one line UseOpenTelemetry call that lights up the same telemetry the rest of the service already emits.
The architect who treats M.E.AI as yet another SDK abstraction misses the design center. The architect who treats it as a pipeline misses nothing.
ASP.NET Core won by being the substrate. M.E.AI is winning the same way.
Article 3, Structured Outputs in C#: Schema, Records, and the End of String Parsing, goes one level up the stack. The ChatOptions.ResponseFormat surface, the JSON schema from C# records pattern, the source generator path for AOT safe serialization, the refusal case when the model declines to produce structured output, and the architectural pattern of mapping the model contract to a typed C# record at the boundary. Subscribe at pragmaticstack.hashnode.dev for Friday's drop.