ITokenBudgetTracker
NexusLabs.Needlr.AgentFramework¶
NexusLabs.Needlr.AgentFramework.Budget¶
ITokenBudgetTracker Interface¶
Tracks token usage within a scoped budget, enabling pipeline-level token limits for total, input, and/or output tokens independently.
Derived
↳ TokenBudgetTracker
Remarks¶
Each call to BeginScope(long) or BeginScope(Nullable<long>, Nullable<long>, Nullable<long>) opens a budget window in the current async context. Concurrent pipeline runs each maintain their own independent token counts via System.Threading.AsyncLocal<>.
ITokenBudgetTracker is automatically registered in DI by
UsingAgentFramework(). Wire the chat-level middleware by calling
UsingTokenBudget() on AgentFrameworkSyringe.
Properties¶
ITokenBudgetTracker.BudgetCancellationToken Property¶
Gets the System.Threading.CancellationToken for the active scope that is cancelled when any budget limit is exceeded.
Property Value¶
System.Threading.CancellationToken
System.Threading.CancellationToken.None if no scope is active.
ITokenBudgetTracker.CurrentInputTokens Property¶
Gets the input tokens accumulated so far in the active scope.
Property Value¶
System.Int64
0 if no scope is active.
ITokenBudgetTracker.CurrentOutputTokens Property¶
Gets the output tokens accumulated so far in the active scope.
Property Value¶
System.Int64
0 if no scope is active.
ITokenBudgetTracker.CurrentTokens Property¶
Gets the total tokens accumulated so far in the active scope.
Property Value¶
System.Int64
0 if no scope is active.
ITokenBudgetTracker.MaxInputTokens Property¶
Gets the input token budget limit of the active scope.
Property Value¶
System.Nullable<System.Int64>
null if no scope is active or no input limit set.
ITokenBudgetTracker.MaxOutputTokens Property¶
Gets the output token budget limit of the active scope.
Property Value¶
System.Nullable<System.Int64>
null if no scope is active or no output limit set.
ITokenBudgetTracker.MaxTokens Property¶
Gets the total token budget limit of the active scope.
Property Value¶
System.Nullable<System.Int64>
null if no scope is active or no total limit set.
Methods¶
ITokenBudgetTracker.BeginScope(long) Method¶
Opens a token-budget scope with a total token limit.
Parameters¶
maxTokens System.Int64
Maximum total tokens allowed.
Returns¶
System.IDisposable
A disposable handle that ends the scope when disposed.
ITokenBudgetTracker.BeginScope(Nullable<long>, Nullable<long>, Nullable<long>) Method¶
Opens a token-budget scope with granular limits for input, output, and/or total tokens. At least one limit must be specified.
System.IDisposable BeginScope(System.Nullable<long> maxInputTokens=null, System.Nullable<long> maxOutputTokens=null, System.Nullable<long> maxTotalTokens=null);
Parameters¶
maxInputTokens System.Nullable<System.Int64>
Maximum input tokens, or null for no limit.
maxOutputTokens System.Nullable<System.Int64>
Maximum output tokens, or null for no limit.
maxTotalTokens System.Nullable<System.Int64>
Maximum total tokens, or null for no limit.
Returns¶
System.IDisposable
A disposable handle that ends the scope when disposed.
Exceptions¶
System.ArgumentException
All three parameters are null.
ITokenBudgetTracker.Record(long) Method¶
Records tokenCount as total tokens against the active scope's budget.
Called automatically by TokenBudgetChatMiddleware after each LLM response.
Parameters¶
tokenCount System.Int64
ITokenBudgetTracker.Record(long, long) Method¶
Records input and output tokens separately against the active scope's budget.
Called automatically by TokenBudgetChatMiddleware after each LLM response.
Parameters¶
inputTokens System.Int64
outputTokens System.Int64