ITokenBudgetTracker

NexusLabs.Needlr.AgentFramework

NexusLabs.Needlr.AgentFramework.Budget

ITokenBudgetTracker Interface

Tracks token usage within a scoped budget, enabling pipeline-level token limits for total, input, and/or output tokens independently.

public interface ITokenBudgetTracker

Derived
↳ TokenBudgetTracker

Remarks

Each call to BeginScope(long) or BeginScope(Nullable<long>, Nullable<long>, Nullable<long>) opens a budget window in the current async context. Concurrent pipeline runs each maintain their own independent token counts via System.Threading.AsyncLocal<>.

ITokenBudgetTracker is automatically registered in DI by UsingAgentFramework(). Wire the chat-level middleware by calling UsingTokenBudget() on AgentFrameworkSyringe.

Properties

ITokenBudgetTracker.BudgetCancellationToken Property

Gets the System.Threading.CancellationToken for the active scope that is cancelled when any budget limit is exceeded.

System.Threading.CancellationToken BudgetCancellationToken { get; }

Property Value

System.Threading.CancellationToken
System.Threading.CancellationToken.None if no scope is active.

ITokenBudgetTracker.CurrentInputTokens Property

Gets the input tokens accumulated so far in the active scope.

long CurrentInputTokens { get; }

Property Value

System.Int64
0 if no scope is active.

ITokenBudgetTracker.CurrentOutputTokens Property

Gets the output tokens accumulated so far in the active scope.

long CurrentOutputTokens { get; }

Property Value

System.Int64
0 if no scope is active.

ITokenBudgetTracker.CurrentTokens Property

Gets the total tokens accumulated so far in the active scope.

long CurrentTokens { get; }

Property Value

System.Int64
0 if no scope is active.

ITokenBudgetTracker.MaxInputTokens Property

Gets the input token budget limit of the active scope.

System.Nullable<long> MaxInputTokens { get; }

Property Value

System.Nullable<System.Int64>
null if no scope is active or no input limit set.

ITokenBudgetTracker.MaxOutputTokens Property

Gets the output token budget limit of the active scope.

System.Nullable<long> MaxOutputTokens { get; }

Property Value

System.Nullable<System.Int64>
null if no scope is active or no output limit set.

ITokenBudgetTracker.MaxTokens Property

Gets the total token budget limit of the active scope.

System.Nullable<long> MaxTokens { get; }

Property Value

System.Nullable<System.Int64>
null if no scope is active or no total limit set.

Methods

ITokenBudgetTracker.BeginChildScope(string, Nullable<long>) Method

Opens a child scope with its own budget that counts against the parent. Token usage in the child rolls up to the parent in real-time. Exceeding the child's limit cancels the child's token. If the parent scope is cancelled, all active children are also cancelled.

System.IDisposable BeginChildScope(string name, System.Nullable<long> maxTokens=null);

Parameters

name System.String

Human-readable name for diagnostics (e.g., stage name).

maxTokens System.Nullable<System.Int64>

Maximum total tokens for this child scope, or null for unlimited (still counts against parent).

Returns

System.IDisposable
A disposable handle that restores the parent scope when disposed.

Exceptions

System.InvalidOperationException
No parent scope is active.

ITokenBudgetTracker.BeginScope(long) Method

Opens a token-budget scope with a total token limit.

System.IDisposable BeginScope(long maxTokens);

Parameters

maxTokens System.Int64

Maximum total tokens allowed.

Returns

System.IDisposable
A disposable handle that ends the scope when disposed.

ITokenBudgetTracker.BeginScope(Nullable<long>, Nullable<long>, Nullable<long>) Method

Opens a token-budget scope with granular limits for input, output, and/or total tokens. At least one limit must be specified.

System.IDisposable BeginScope(System.Nullable<long> maxInputTokens=null, System.Nullable<long> maxOutputTokens=null, System.Nullable<long> maxTotalTokens=null);

Parameters

maxInputTokens System.Nullable<System.Int64>

Maximum input tokens, or null for no limit.

maxOutputTokens System.Nullable<System.Int64>

Maximum output tokens, or null for no limit.

maxTotalTokens System.Nullable<System.Int64>

Maximum total tokens, or null for no limit.

Returns

System.IDisposable
A disposable handle that ends the scope when disposed.

Exceptions

System.ArgumentException
All three parameters are null.

ITokenBudgetTracker.Record(long) Method

Records tokenCount as total tokens against the active scope's budget. Called automatically by TokenBudgetChatMiddleware after each LLM response.

void Record(long tokenCount);

Parameters

tokenCount System.Int64

ITokenBudgetTracker.Record(long, long) Method

Records input and output tokens separately against the active scope's budget. Called automatically by TokenBudgetChatMiddleware after each LLM response.

void Record(long inputTokens, long outputTokens);

Parameters

inputTokens System.Int64

outputTokens System.Int64