EvaluationCaptureChatClient

NexusLabs.Needlr.AgentFramework.Evaluation ¶

EvaluationCaptureChatClient Class¶

Microsoft.Extensions.AI.DelegatingChatClient that persists every LLM request/response pair to an IEvaluationCaptureStore and replays cached responses on subsequent calls with an identical request. Intended to make evaluator runs deterministic and cheap to re-execute.

public sealed class EvaluationCaptureChatClient : Microsoft.Extensions.AI.DelegatingChatClient

Inheritance System.Object 🡒 Microsoft.Extensions.AI.DelegatingChatClient 🡒 EvaluationCaptureChatClient

Remarks¶

Cache keys are derived from a stable SHA-256 hash of the request messages (role + text) and a small subset of Microsoft.Extensions.AI.ChatOptions that affect output — currently ModelId, Temperature, TopP, and MaxOutputTokens. Requests that differ only in non-captured options will collide; callers that rely on other options producing distinct responses must not use this middleware.

Streaming calls materialize cached responses as a single Microsoft.Extensions.AI.ChatResponseUpdate per message. On cache miss the stream is fully buffered before being persisted and replayed to the caller.

Constructors¶

EvaluationCaptureChatClient(IChatClient, IEvaluationCaptureStore) Constructor¶

public EvaluationCaptureChatClient(Microsoft.Extensions.AI.IChatClient innerClient, NexusLabs.Needlr.AgentFramework.Evaluation.IEvaluationCaptureStore store);

Parameters¶

innerClient Microsoft.Extensions.AI.IChatClient

The inner chat client to delegate to.

store IEvaluationCaptureStore

Backing store used for capture and replay.

Methods¶

EvaluationCaptureChatClient.GetResponseAsync(IEnumerable<ChatMessage>, ChatOptions, CancellationToken) Method¶

Sends chat messages and returns the response.

public override System.Threading.Tasks.Task<Microsoft.Extensions.AI.ChatResponse> GetResponseAsync(System.Collections.Generic.IEnumerable<Microsoft.Extensions.AI.ChatMessage> messages, Microsoft.Extensions.AI.ChatOptions? options=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));

Parameters¶

messages System.Collections.Generic.IEnumerable<Microsoft.Extensions.AI.ChatMessage >

The sequence of chat messages to send.

options Microsoft.Extensions.AI.ChatOptions

The chat options with which to configure the request.

cancellationToken System.Threading.CancellationToken

The System.Threading.CancellationToken to monitor for cancellation requests. The default is System.Threading.CancellationToken.None.

Implements GetResponseAsync(IEnumerable<ChatMessage>, ChatOptions, CancellationToken)

Returns¶

System.Threading.Tasks.Task<Microsoft.Extensions.AI.ChatResponse >
The response messages generated by the client.

Exceptions¶

System.ArgumentNullException
messages is null.

EvaluationCaptureChatClient.GetStreamingResponseAsync(IEnumerable<ChatMessage>, ChatOptions, CancellationToken) Method¶

Sends chat messages and streams the response.

public override System.Collections.Generic.IAsyncEnumerable<Microsoft.Extensions.AI.ChatResponseUpdate> GetStreamingResponseAsync(System.Collections.Generic.IEnumerable<Microsoft.Extensions.AI.ChatMessage> messages, Microsoft.Extensions.AI.ChatOptions? options=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));