TokenBudgetTracker

NexusLabs.Needlr.AgentFramework

NexusLabs.Needlr.AgentFramework.Budget

TokenBudgetTracker Class

System.Threading.AsyncLocal<>-scoped implementation of ITokenBudgetTracker with granular input/output/total budget tracking.

public sealed class TokenBudgetTracker : NexusLabs.Needlr.AgentFramework.Budget.ITokenBudgetTracker

Inheritance System.Object 🡒 TokenBudgetTracker

Implements ITokenBudgetTracker

Properties

TokenBudgetTracker.BudgetCancellationToken Property

Gets the System.Threading.CancellationToken for the active scope that is cancelled when any budget limit is exceeded.

public System.Threading.CancellationToken BudgetCancellationToken { get; }

Implements BudgetCancellationToken

Property Value

System.Threading.CancellationToken
System.Threading.CancellationToken.None if no scope is active.

TokenBudgetTracker.CurrentInputTokens Property

Gets the input tokens accumulated so far in the active scope.

public long CurrentInputTokens { get; }

Implements CurrentInputTokens

Property Value

System.Int64
0 if no scope is active.

TokenBudgetTracker.CurrentOutputTokens Property

Gets the output tokens accumulated so far in the active scope.

public long CurrentOutputTokens { get; }

Implements CurrentOutputTokens

Property Value

System.Int64
0 if no scope is active.

TokenBudgetTracker.CurrentTokens Property

Gets the total tokens accumulated so far in the active scope.

public long CurrentTokens { get; }

Implements CurrentTokens

Property Value

System.Int64
0 if no scope is active.

TokenBudgetTracker.MaxInputTokens Property

Gets the input token budget limit of the active scope.

public System.Nullable<long> MaxInputTokens { get; }

Implements MaxInputTokens

Property Value

System.Nullable<System.Int64>
null if no scope is active or no input limit set.

TokenBudgetTracker.MaxOutputTokens Property

Gets the output token budget limit of the active scope.

public System.Nullable<long> MaxOutputTokens { get; }

Implements MaxOutputTokens

Property Value

System.Nullable<System.Int64>
null if no scope is active or no output limit set.

TokenBudgetTracker.MaxTokens Property

Gets the total token budget limit of the active scope.

public System.Nullable<long> MaxTokens { get; }

Implements MaxTokens

Property Value

System.Nullable<System.Int64>
null if no scope is active or no total limit set.

Methods

TokenBudgetTracker.BeginChildScope(string, Nullable<long>) Method

Opens a child scope with its own budget that counts against the parent. Token usage in the child rolls up to the parent in real-time. Exceeding the child's limit cancels the child's token. If the parent scope is cancelled, all active children are also cancelled.

public System.IDisposable BeginChildScope(string name, System.Nullable<long> maxTokens=null);

Parameters

name System.String

Human-readable name for diagnostics (e.g., stage name).

maxTokens System.Nullable<System.Int64>

Maximum total tokens for this child scope, or null for unlimited (still counts against parent).

Implements BeginChildScope(string, Nullable<long>)

Returns

System.IDisposable
A disposable handle that restores the parent scope when disposed.

Exceptions

System.InvalidOperationException
No parent scope is active.

TokenBudgetTracker.BeginScope(long) Method

Opens a token-budget scope with a total token limit.

public System.IDisposable BeginScope(long maxTokens);

Parameters

maxTokens System.Int64

Maximum total tokens allowed.

Implements BeginScope(long)

Returns

System.IDisposable
A disposable handle that ends the scope when disposed.

TokenBudgetTracker.BeginScope(Nullable<long>, Nullable<long>, Nullable<long>) Method

Opens a token-budget scope with granular limits for input, output, and/or total tokens. At least one limit must be specified.

public System.IDisposable BeginScope(System.Nullable<long> maxInputTokens=null, System.Nullable<long> maxOutputTokens=null, System.Nullable<long> maxTotalTokens=null);

Parameters

maxInputTokens System.Nullable<System.Int64>

Maximum input tokens, or null for no limit.

maxOutputTokens System.Nullable<System.Int64>

Maximum output tokens, or null for no limit.

maxTotalTokens System.Nullable<System.Int64>

Maximum total tokens, or null for no limit.

Implements BeginScope(Nullable<long>, Nullable<long>, Nullable<long>)

Returns

System.IDisposable
A disposable handle that ends the scope when disposed.

Exceptions

System.ArgumentException
All three parameters are null.

TokenBudgetTracker.BeginTrackingScope() Method

Opens a pure tracking scope with no budget limits. Token usage is accumulated via Record(long) and Record(long, long) but no cancellation or enforcement occurs. Use this when you need visibility into token usage without restricting it.

public System.IDisposable BeginTrackingScope();

Implements BeginTrackingScope()

Returns

System.IDisposable
A disposable handle that ends the tracking scope when disposed.

TokenBudgetTracker.Record(long) Method

Records tokenCount as total tokens against the active scope's budget. Called automatically by TokenBudgetChatMiddleware after each LLM response.

public void Record(long tokenCount);

Parameters

tokenCount System.Int64

Implements Record(long)

TokenBudgetTracker.Record(long, long) Method

Records input and output tokens separately against the active scope's budget. Called automatically by TokenUsageRecordingMiddleware after each LLM response.

public void Record(long inputTokens, long outputTokens);

Parameters

inputTokens System.Int64

outputTokens System.Int64

Implements Record(long, long)