Skip to content

TokenBudgetTracker

NexusLabs.Needlr.AgentFramework

NexusLabs.Needlr.AgentFramework.Budget

TokenBudgetTracker Class

System.Threading.AsyncLocal<>-scoped implementation of ITokenBudgetTracker with granular input/output/total budget tracking.

public sealed class TokenBudgetTracker : NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker

Inheritance System.Object 🡒 TokenBudgetTracker

Implements ITokenBudgetTracker

Properties

TokenBudgetTracker.BudgetCancellationToken Property

Gets the System.Threading.CancellationToken for the active scope that is cancelled when any budget limit is exceeded.

public System.Threading.CancellationToken BudgetCancellationToken { get; }

Implements BudgetCancellationToken

Property Value

System.Threading.CancellationToken
System.Threading.CancellationToken.None if no scope is active.

TokenBudgetTracker.CurrentInputTokens Property

Gets the input tokens accumulated so far in the active scope.

public long CurrentInputTokens { get; }

Implements CurrentInputTokens

Property Value

System.Int64
0 if no scope is active.

TokenBudgetTracker.CurrentOutputTokens Property

Gets the output tokens accumulated so far in the active scope.

public long CurrentOutputTokens { get; }

Implements CurrentOutputTokens

Property Value

System.Int64
0 if no scope is active.

TokenBudgetTracker.CurrentTokens Property

Gets the total tokens accumulated so far in the active scope.

public long CurrentTokens { get; }

Implements CurrentTokens

Property Value

System.Int64
0 if no scope is active.

TokenBudgetTracker.MaxInputTokens Property

Gets the input token budget limit of the active scope.

public System.Nullable<long> MaxInputTokens { get; }

Implements MaxInputTokens

Property Value

System.Nullable<System.Int64>
null if no scope is active or no input limit set.

TokenBudgetTracker.MaxOutputTokens Property

Gets the output token budget limit of the active scope.

public System.Nullable<long> MaxOutputTokens { get; }

Implements MaxOutputTokens

Property Value

System.Nullable<System.Int64>
null if no scope is active or no output limit set.

TokenBudgetTracker.MaxTokens Property

Gets the total token budget limit of the active scope.

public System.Nullable<long> MaxTokens { get; }

Implements MaxTokens

Property Value

System.Nullable<System.Int64>
null if no scope is active or no total limit set.

Methods

TokenBudgetTracker.BeginScope(long) Method

Opens a token-budget scope with a total token limit.

public System.IDisposable BeginScope(long maxTokens);

Parameters

maxTokens System.Int64

Maximum total tokens allowed.

Implements BeginScope(long)

Returns

System.IDisposable
A disposable handle that ends the scope when disposed.

TokenBudgetTracker.BeginScope(Nullable<long>, Nullable<long>, Nullable<long>) Method

Opens a token-budget scope with granular limits for input, output, and/or total tokens. At least one limit must be specified.

public System.IDisposable BeginScope(System.Nullable<long> maxInputTokens=null, System.Nullable<long> maxOutputTokens=null, System.Nullable<long> maxTotalTokens=null);

Parameters

maxInputTokens System.Nullable<System.Int64>

Maximum input tokens, or null for no limit.

maxOutputTokens System.Nullable<System.Int64>

Maximum output tokens, or null for no limit.

maxTotalTokens System.Nullable<System.Int64>

Maximum total tokens, or null for no limit.

Implements BeginScope(Nullable<long>, Nullable<long>, Nullable<long>)

Returns

System.IDisposable
A disposable handle that ends the scope when disposed.

Exceptions

System.ArgumentException
All three parameters are null.

TokenBudgetTracker.Record(long) Method

Records tokenCount as total tokens against the active scope's budget. Called automatically by TokenBudgetChatMiddleware after each LLM response.

public void Record(long tokenCount);

Parameters

tokenCount System.Int64

Implements Record(long)

TokenBudgetTracker.Record(long, long) Method

Records input and output tokens separately against the active scope's budget. Called automatically by TokenBudgetChatMiddleware after each LLM response.

public void Record(long inputTokens, long outputTokens);

Parameters

inputTokens System.Int64

outputTokens System.Int64

Implements Record(long, long)