Hosting Long-Running Agents — IHostedService in .NET 10

🗓️ Last updated: August 2026

It is 09:14 on a Tuesday. A senior dev is staring at the Grafana panel for a brownfield .NET 10 worker service that runs an order-investigation agent. The agent takes four to six minutes per ticket. It calls three tools — the order database, the shipping carrier's API, the fraud service — and writes a structured verdict to a queue. The service runs on Kubernetes. Twenty-three minutes ago, a node went unhealthy and the pod restarted. Twenty-three agent runs were in flight when the pod died. None of them came back. The entry point is a BackgroundService that calls agent.RunAsync in a loop. There is no checkpointing. There is no resume token. The cancellation token is honoured at the top of the loop but not inside the tool calls. The senior dev is writing a postmortem and the first line is a question for the architect — why does a six-minute agent run treat its host process as if it would live forever.

This is the recurring shape of a brownfield .NET service that learned to host agents before agents got long. The model used to return in under a second and the loop was small enough to re-run from the top. Then the agent grew tools, the tools grew latency, and one day the average run crossed the pod's graceful shutdown window. From that day on, every deploy was an outage for the in-flight work.

The answer is not a parallel scheduler around the agent. The .NET hosting model already has the primitives — IHostedService, BackgroundService, IHostApplicationLifetime, the cancellation-token contract, the shutdown timeout — and the Agent Framework already has the abstraction for the unit of work that survives a restart. The architect's job is to wire one to the other.

What this article is NOT

This is not a Kubernetes tutorial. It is not a defence of running long agent loops inside an ASP.NET Core request handler — if you are still doing that in 2026, the readiness probe is lying to you. It is not a comparison of Hangfire, Quartz.NET, and the generic host.

It is an architect's read on the .NET hosting model as the right altitude for a long-running agent, on the contract the cancellation token expresses, and on the persistence boundary the agent has to cross to be resumable at all.

Thesis

An agent that outlives a single HTTP request is a hosted service. A hosted service in .NET is IHostedService and its canonical subclass BackgroundService. The cancellation token passed into ExecuteAsync is the contract — the host promises to fire it at shutdown, the service promises to honour it. The shutdown timeout is the budget the service has between "the token fires" and "the host stops waiting." The agent session is the unit of state the service has to write down at every observable boundary so that the next process can pick the work up from a known point. The hosting model has been load-bearing since .NET Core 2.1. The Agent Framework ships AgentSession, SerializeSessionAsync, and AgentSessionStore as the named surface for the resumable unit of work. The architecture is to use them as designed and stop inventing a parallel lifecycle around the LLM.

What the classical .NET version looked like

The pre-Generic-Host pattern was a Windows Service hosted by ServiceBase, a long-running loop kicked off in OnStart, a cancellation flag polled by the loop, and a Thread.Sleep between iterations. On the web side it was a singleton Task.Run parked in Application_Start whose only signal that the app pool was recycling was a half-second window before IIS killed the worker process. Graceful shutdown was a phrase in a slide deck. State across restarts meant a database table with a JobStatus column that the next process scanned on boot.

The Generic Host shipped in .NET Core 2.1 in May 2018 and consolidated all of this into one composition root. IHostedService, BackgroundService, and IHostApplicationLifetime became the named surfaces, and the Worker Service template — dotnet new worker — shipped them as a first-class project type. The architectural problem in 2026 is not the host. It is what the host runs.

The hosting primitives

There are three primitives the architect has to know by name.

IHostedService is the interface. StartAsync(CancellationToken) is called once at startup and is expected to return promptly. StopAsync(CancellationToken) is called once at shutdown and is given the host's shutdown timeout to wind down cleanly.

BackgroundService is the abstract base class for long-running loops. You override one method — ExecuteAsync(CancellationToken stoppingToken). Internally StartAsync creates a CancellationTokenSource linked to the start token, kicks off ExecuteAsync on a Task.Run, and returns Task.CompletedTask immediately. StopAsync cancels the linked source — which fires the stoppingToken your loop is observing — then waits for the execute task to complete, bounded by HostOptions.ShutdownTimeout — thirty seconds by default, verified in the runtime's own test suite against TimeSpan.FromSeconds(30).

IHostApplicationLifetime is the named surface for host lifecycle events. ApplicationStarted fires when the host has fully started. ApplicationStopping fires when shutdown begins — the signal you want for per-shutdown bookkeeping. ApplicationStopped fires when the host has finished stopping. The interface also exposes StopApplication() for programmatic shutdown. The older IApplicationLifetime exists but is marked [Obsolete] and points to IHostApplicationLifetime.

The cancellation contract

The cancellation token passed into ExecuteAsync is the most load-bearing contract in the hosting model. The shape is one sentence — every awaitable inside the loop honours the token, and OperationCanceledException is allowed to propagate during shutdown.

The token fires when the host begins shutdown, and past HostOptions.ShutdownTimeout the process is going down whether the loop is finished or not. Every blocking operation — every database call, every HTTP request, every model call, every wait between retries — has to take the token. A blocking call with no token is a bet that the operation finishes faster than the shutdown timeout. The bet is paid by the platform team during the next deploy.

The second half of the contract is what to do with OperationCanceledException. Let it propagate when it was triggered by the stopping token, and only catch it when the cancellation came from a different source the loop actually owns. A swallowed cancellation is a host that thinks the service has stopped and a service that thinks it is still allowed to run — and the next thing that happens is the host disposes the scope and the loop writes to a disposed DbContext.

public sealed class OrderInvestigationWorker : BackgroundService
{
    private readonly IServiceScopeFactory _scopeFactory;
    private readonly ILogger<OrderInvestigationWorker> _logger;

    public OrderInvestigationWorker(
        IServiceScopeFactory scopeFactory,
        ILogger<OrderInvestigationWorker> logger)
    {
        _scopeFactory = scopeFactory;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            await using var scope = _scopeFactory.CreateAsyncScope();
            var queue = scope.ServiceProvider.GetRequiredService<ITicketQueue>();
            var runner = scope.ServiceProvider.GetRequiredService<IAgentRunner>();

            var ticket = await queue.DequeueAsync(stoppingToken);
            if (ticket is null)
            {
                await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken);
                continue;
            }

            try
            {
                await runner.InvestigateAsync(ticket, stoppingToken);
            }
            catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
            {
                _logger.LogInformation(
                    "Ticket {TicketId} cancelled by host shutdown; checkpoint preserved.",
                    ticket.Id);
                throw;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex,
                    "Ticket {TicketId} failed; will retry on next pickup.",
                    ticket.Id);
            }
        }
    }
}

Four things in that loop are deliberate. The scope is created per work unit, not for the lifetime of the service. The dequeue carries the stopping token. The cancellation catch uses an exception filter so it only matches when the host is the one cancelling. The rethrow lets the framework treat the loop as having shut down cleanly.

Scoped services in a singleton lifetime

BackgroundService is registered as a singleton, so its constructor cannot take a scoped dependency. The pattern is the one shown above — inject IServiceScopeFactory, create a scope per work unit, resolve scoped services through the scope's ServiceProvider, dispose with await using.

This is the single most common lifetime bug in .NET worker services and it does not surface until production. A DbContext resolved at the singleton root is shared across every iteration of the loop. The change-tracker accumulates entities forever. The connection is held for the lifetime of the process. The second concurrent operation throws because DbContext is not thread-safe. None of this shows up in a unit test where the loop runs once; all of it shows up in production where the loop runs ten thousand times. A repository linter rule that flags any scoped registration resolved in a BackgroundService constructor pays for itself the first time it catches the bug.

The agent session is the unit of resumability

A BackgroundService survives a restart by virtue of the host being restarted. The work it was doing does not survive unless it was written down. For an agent, the unit of state worth writing down is the agent session — the conversation history, the tool-call transcript, the in-progress reasoning, anything the agent needs to pick the run up from the point it stopped.

The Agent Framework names this abstraction AgentSession in Microsoft.Agents.AI.Abstractions. The XML documentation on the type makes the design intent explicit — to support conversations that may need to survive application restarts or separate service requests, an AgentSession can be serialized and deserialized, so that it can be saved in a persistent store. The AIAgent base class exposes three methods that together define the resumability contract — CreateSessionAsync creates a fresh session, SerializeSessionAsync writes it to a JsonElement, DeserializeSessionAsync reads it back. The session carries a StateBag of arbitrary serializable data so that components attached to the agent can each save and restore their own state alongside the conversation.

The framework also ships an AgentSessionStore abstraction for the durable persistence layer. The canonical resume pattern is one switch expression — if there is a stored session for the conversation, deserialize it; otherwise, create a new one.

public sealed class RedisAgentSessionStore : AgentSessionStore
{
    private readonly IConnectionMultiplexer _redis;

    public RedisAgentSessionStore(IConnectionMultiplexer redis) => _redis = redis;

    public override async ValueTask SaveSessionAsync(
        AIAgent agent,
        string conversationId,
        AgentSession session,
        CancellationToken cancellationToken = default)
    {
        var key = $"{agent.Id}:{conversationId}";
        var serialized = await agent.SerializeSessionAsync(
            session,
            cancellationToken: cancellationToken).ConfigureAwait(false);
        var db = _redis.GetDatabase();
        await db.StringSetAsync(key, serialized.GetRawText());
    }

    public override async ValueTask<AgentSession> GetSessionAsync(
        AIAgent agent,
        string conversationId,
        CancellationToken cancellationToken = default)
    {
        var key = $"{agent.Id}:{conversationId}";
        var db = _redis.GetDatabase();
        var stored = await db.StringGetAsync(key);

        if (stored.IsNullOrEmpty)
        {
            return await agent.CreateSessionAsync(cancellationToken).ConfigureAwait(false);
        }

        using var doc = JsonDocument.Parse(stored!);
        return await agent.DeserializeSessionAsync(
            doc.RootElement.Clone(),
            cancellationToken: cancellationToken).ConfigureAwait(false);
    }
}

Two things about that store are non-negotiable. The session is saved at observable boundaries — after every tool call, before every model call, on shutdown — not on a fixed timer. The save key combines the agent identity and the conversation identity so that a session restored for one agent cannot be replayed by another. The AgentSession XML doc spells out the security consequence in one sentence — treat restoring a session from an untrusted source as equivalent to accepting untrusted input; a compromised storage backend could alter message roles to escalate trust, or inject adversarial content that influences LLM behavior.

Some of the concrete hosting types around AgentSessionStore are still marked [Experimental] in the current package while the diagnostic surface settles. The architect treats the shape as stable and the diagnostic identifiers as movable.

Trace continuity across the restart boundary

OpenTelemetry treats every Activity as a node in a tree rooted at the operation that started the work. A pod restart breaks the tree. The new process creates a new root activity and the resumed run has no parent. Two traces appear in the backend. The investigator who picks up the incident a week later sees a four-minute run that ended in cancelled and a two-minute run that started from nowhere. They are the same investigation. The tooling does not know.

The fix is to write the parent activity context into the agent session alongside the conversation state. On resume, the worker reads the stored trace ID and span ID, constructs an ActivityContext with ActivityTraceFlags.Recorded, and starts the resumed work as a child of that context. The trace IDs match across both halves of the loop. The backend stitches the run into one tree.

var parent = new ActivityContext(
    ActivityTraceId.CreateFromString(checkpoint.TraceId),
    ActivitySpanId.CreateFromString(checkpoint.ParentSpanId),
    ActivityTraceFlags.Recorded);

using var resumed = _source.StartActivity(
    "agent.investigate.resume",
    ActivityKind.Internal,
    parent);

await runner.ResumeAsync(checkpoint, stoppingToken);

The pattern generalises. Any unit of work that crosses a process boundary — a queue handoff, a restart, a delegation to another service — carries the parent ActivityContext as part of its payload. Breaking the trace tree at a restart is the single most expensive observability bug an agent platform can ship with.

Health for an agent loop is not "is the process running"

The liveness probe is the question is this process alive. The readiness probe is the question can this process accept work right now. Neither answers the question is the agent loop making progress. An agent loop that has silently deadlocked on a tool call returns a healthy 200 to both probes because the HTTP server in the same process is still answering. The pod stays in the load balancer. The kafka backlog grows.

The fix is a third health check the architect writes. A watchdog counter, updated by the loop at the top of every iteration, observed by an IHealthCheck that returns Healthy if the counter has changed within a configured window and Unhealthy otherwise. The check is registered as tagged and mapped at a separate path so Kubernetes restarts the pod if it fails for long enough. The agent loop now participates in the same liveness contract as the rest of the platform.

services.AddHealthChecks()
    .AddCheck<AgentLoopHeartbeatCheck>(
        "agent-loop",
        tags: new[] { "live" });

app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = r => r.Tags.Contains("live")
});

The watchdog window is a budget, not a guess. Set it to the longest atomic operation the agent is allowed to perform plus one tool-call timeout, and document it next to HostOptions.ShutdownTimeout because both belong to the same conversation about how long things are allowed to take.

Failure modes, severity-ordered

1. Scoped service resolved at the singleton root. The single most expensive bug in the hosting model. DbContext resolved in the BackgroundService constructor — change-tracker grows forever, the connection is held for the lifetime of the process, concurrent operations throw because DbContext is not thread-safe, and none of it shows up in a unit test.

2. Blocking operation that does not honour the stopping token. A tool call with no cancellation parameter, a Task.Delay with no token, an HTTP client with an infinite timeout. The host signals shutdown, the loop continues, the host kills the process at the shutdown-timeout boundary, the in-flight work dies with no checkpoint.

3. OperationCanceledException swallowed without checking the token source. The catch block treats every cancellation as a transient error and continues. The host believes the service has stopped; the service writes to a disposed scope. The exception surfaces as ObjectDisposedException six layers down and is unrecognisable as a shutdown bug.

4. Agent state never written to durable storage. The BackgroundService runs the agent in memory and never persists the AgentSession. Every restart loses every in-flight conversation, and the kafka backlog is the one that notices.

5. Trace context broken at the restart boundary. The parent ActivityContext is not persisted. The resumed run starts a new trace. The trace tree splits into two halves the backend cannot rejoin, and incident retrospectives lose ten minutes per restart correlating IDs that should already be linked.

6. Health check that returns Healthy while the loop has stalled. The liveness probe asks the HTTP server, the HTTP server is in the same process as the deadlocked loop, and the HTTP server answers. The pod stays in the load balancer while the agent loop is not progressing.

7. Shutdown timeout shorter than the agent's longest atomic operation. HostOptions.ShutdownTimeout left at the thirty-second default for a worker whose tool calls can take ninety seconds. Every deploy guarantees at least one in-flight operation gets killed mid-call.

The architect's checklist

🗓️ Every long-running agent runs in BackgroundService, not in Task.Run, not in Timer, not on a static field set in Program.cs. 🗓️ ExecuteAsync honours the stopping token at every await point — every queue read, every HTTP call, every model call, every delay. 🗓️ OperationCanceledException from the stopping token is allowed to propagate; cancellations from other sources are caught explicitly with an exception filter. 🗓️ Scoped services are resolved through IServiceScopeFactory.CreateAsyncScope() per work unit, never in the worker's constructor. 🗓️ AgentSession is serialized at observable boundaries — after each tool call, before each model call, on shutdown — through an AgentSessionStore backed by a durable store. 🗓️ The save key combines agent.Id and conversationId; restoring a session under a different agent identity is treated as an authorisation event. 🗓️ Trace context is persisted with the session and restored as the parent ActivityContext on resume, so the agent run is one tree in the backend. 🗓️ HostOptions.ShutdownTimeout is configured to the agent's longest atomic operation plus one tool-call retry, not left at the thirty-second default. 🗓️ A liveness check tests the host process; a separate watchdog check fails when the agent loop has not advanced within a configured window. 🗓️ The Worker Service template is the starting point; the composition root resolves the agent and the session store the same way the chat layer was resolved in Article 2. 🗓️ Restoring a session from storage is treated as accepting untrusted input — message roles are validated, conversation IDs are scoped to the caller, the storage backend has access controls and encryption at rest. 🗓️ Shutdown logs include the count of unfinished agent runs and the conversation IDs whose checkpoints survive — that line is the first thing the on-call engineer reads after a deploy.

Mental model

The host is the lifecycle. The worker is the loop. The cancellation token is the contract. The session is the state. The session store is the boundary between the loop and the platform. Get those five right and a pod restart becomes a thirty-second pause in the middle of a six-minute run, not the loss of twenty-three investigations. Get any one of them wrong and the kafka backlog tells the story before the postmortem does.

A long-running agent does not need a parallel scheduler. It needs the boring half of .NET — the hosting model that has been load-bearing since 2018 — wired correctly to the new abstraction that owns its state.

What's next

Article 5 — Streaming Chat — IAsyncEnumerable, Minimal APIs, and SignalR — pivots from the worker back to the request handler, but at a faster cadence. The model now returns one token at a time. The transport has to carry it. The client has to render it. The cancellation semantics, the backpressure model, and the failure modes all change shape. The hosting work in this article still applies — streaming endpoints are still hosted services, still run inside the same lifecycle — but the unit of work is now a single response and the architectural question is how the .NET surface for IAsyncEnumerable<ChatResponseUpdate> reaches a browser without an unbounded buffer in the middle. Subscribe at pragmaticstack.hashnode.dev for Friday's drop.

Series 4 progress

#	Title	Status
1	From Semantic Kernel to Microsoft Agent Framework	Published
2	Microsoft.Extensions.AI — The Unified Client Layer	Published
3	Structured Outputs in C#	Published
4	Hosting Long-Running Agents	You are here
5	Streaming Chat — IAsyncEnumerable and SignalR	Coming next
6	A Reference Architecture for the .NET Agentic Service	Coming soon

Hosting Long-Running Agents — IHostedService, BackgroundService, and the Worker Lifecycle

Comments

AI for the .NET Architect

The Framework Died, the Architecture Lived: From Semantic Kernel to the Microsoft Agent Framework

More from this blog

Structured Outputs in C# — Schema, Records, and the End of String-Parsing

The Substrate That Wins by Being Boring: Microsoft.Extensions.AI as the Unified Client Layer

The Framework Died, the Architecture Lived: From Semantic Kernel to the Microsoft Agent Framework

The Invoice Was the Only Alert That Fired: Cost Guardrails for LLM Systems

What this article is NOT

Thesis

What the classical .NET version looked like

The hosting primitives

The cancellation contract

Scoped services in a singleton lifetime

The agent session is the unit of resumability

Trace continuity across the restart boundary

Health for an agent loop is not "is the process running"

Failure modes, severity-ordered

The architect's checklist

Mental model

What's next

Series 4 progress

Command Palette

Comments

AI for the .NET Architect

The Framework Died, the Architecture Lived: From Semantic Kernel to the Microsoft Agent Framework

More from this blog

What this article is NOT

Thesis

What the classical .NET version looked like

The hosting primitives

The cancellation contract

Scoped services in a singleton lifetime

The agent session is the unit of resumability

Trace continuity across the restart boundary

Health for an agent loop is not "is the process running"

Failure modes, severity-ordered

The architect's checklist

Mental model

What's next

Series 4 progress