TokenUsageRecordingMiddleware

NexusLabs.Needlr.AgentFramework.Workflows ¶

NexusLabs.Needlr.AgentFramework.Workflows.Budget ¶

TokenUsageRecordingMiddleware Class¶

Lightweight Microsoft.Extensions.AI.DelegatingChatClient that records token usage from each LLM call into NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker. Does NOT enforce budgets — that is the responsibility of TokenBudgetChatMiddleware.

public sealed class TokenUsageRecordingMiddleware : Microsoft.Extensions.AI.DelegatingChatClient

Inheritance System.Object 🡒 Microsoft.Extensions.AI.DelegatingChatClient 🡒 TokenUsageRecordingMiddleware

Remarks¶

Wired automatically by UsingTokenTracking(), UsingTokenBudget(), and UsingDiagnostics(). Idempotent — only one instance is wired regardless of how many extensions request it.

Constructors¶

TokenUsageRecordingMiddleware(IChatClient, ITokenBudgetTracker) Constructor¶

public TokenUsageRecordingMiddleware(Microsoft.Extensions.AI.IChatClient innerClient, NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker tracker);

Parameters¶

innerClient Microsoft.Extensions.AI.IChatClient

The inner chat client to delegate to.

tracker NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker

The token budget tracker to record usage into.

Methods¶

TokenUsageRecordingMiddleware.GetResponseAsync(IEnumerable<ChatMessage>, ChatOptions, CancellationToken) Method¶

Sends chat messages and returns the response.

public override System.Threading.Tasks.Task<Microsoft.Extensions.AI.ChatResponse> GetResponseAsync(System.Collections.Generic.IEnumerable<Microsoft.Extensions.AI.ChatMessage> messages, Microsoft.Extensions.AI.ChatOptions? options=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));