TokenBudgetChatMiddleware

NexusLabs.Needlr.AgentFramework.Workflows ¶

NexusLabs.Needlr.AgentFramework.Workflows.Budget ¶

TokenBudgetChatMiddleware Class¶

Microsoft.Extensions.AI.DelegatingChatClient that enforces token budget limits by aborting when NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker thresholds are exceeded. Depends on TokenUsageRecordingMiddleware (wired as an inner middleware) to keep the tracker's token counts up to date.

public sealed class TokenBudgetChatMiddleware : Microsoft.Extensions.AI.DelegatingChatClient

Inheritance System.Object 🡒 Microsoft.Extensions.AI.DelegatingChatClient 🡒 TokenBudgetChatMiddleware

Remarks¶

This middleware does NOT record token usage — that is handled by TokenUsageRecordingMiddleware, which runs before this middleware in the pipeline. Use UsingTokenBudget() to wire both correctly.

Budget enforcement uses two mechanisms: 1. System.OperationCanceledException wrapping NexusLabs.Needlr.AgentFramework.Budget.TokenBudgetExceededException thrown from the middleware (works for direct agent runs). 2. NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker.BudgetCancellationToken cancelled when tokens are recorded past the limit (works for MAF workflow runs — pass this token to the workflow).

Constructors¶

TokenBudgetChatMiddleware(IChatClient, ITokenBudgetTracker, IProgressReporterAccessor) Constructor¶

public TokenBudgetChatMiddleware(Microsoft.Extensions.AI.IChatClient innerClient, NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker tracker, NexusLabs.Needlr.AgentFramework.Progress.IProgressReporterAccessor progressAccessor);

Parameters¶

innerClient Microsoft.Extensions.AI.IChatClient

The inner chat client to delegate to.

tracker NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker

The token budget tracker scoped to the current pipeline run.

progressAccessor NexusLabs.Needlr.AgentFramework.Progress.IProgressReporterAccessor

Progress reporter accessor for emitting budget events.

Methods¶

TokenBudgetChatMiddleware.GetResponseAsync(IEnumerable<ChatMessage>, ChatOptions, CancellationToken) Method¶

Sends chat messages and returns the response.

public override System.Threading.Tasks.Task<Microsoft.Extensions.AI.ChatResponse> GetResponseAsync(System.Collections.Generic.IEnumerable<Microsoft.Extensions.AI.ChatMessage> messages, Microsoft.Extensions.AI.ChatOptions? options, System.Threading.CancellationToken cancellationToken);