EvaluationQualityGate

NexusLabs.Needlr.AgentFramework.Evaluation

EvaluationQualityGate Class

Configurable quality gate that asserts evaluation metrics meet defined thresholds. Designed for CI pipelines — call Assert(EvaluationResult[]) after running evaluators to fail the build when metrics regress.

public sealed class EvaluationQualityGate

Inheritance System.Object 🡒 EvaluationQualityGate

Example

var gate = new EvaluationQualityGate()
    .RequireBoolean(ToolCallTrajectoryEvaluator.AllSucceededMetricName, expected: true)
    .RequireBoolean(IterationCoherenceEvaluator.TerminatedCoherentlyMetricName, expected: true)
    .RequireNumericMax(EfficiencyEvaluator.TotalTokensMetricName, max: 50_000)
    .RequireBoolean(EfficiencyEvaluator.UnderBudgetMetricName, expected: true);

// Throws QualityGateFailedException if any threshold is violated.
gate.Assert(trajectoryResult, coherenceResult, efficiencyResult);

Remarks

Thresholds are defined fluently via RequireNumericMax(string, double), RequireNumericMin(string, double), and RequireBoolean(string, bool). Each threshold names a metric (using the *MetricName constants from evaluator classes) and a bound. Assert(EvaluationResult[]) checks all thresholds against the evaluation result and throws QualityGateFailedException listing every violation.

Metrics not present in the Microsoft.Extensions.AI.Evaluation.EvaluationResult are silently skipped — this allows a gate to be used with evaluators that conditionally emit metrics (e.g., IterationCoherenceEvaluator only emits when execution mode is IterativeLoop).

Methods

EvaluationQualityGate.Assert(EvaluationResult[]) Method

Checks all thresholds against the provided evaluation results. Metrics are looked up across all results — the first match wins.

public void Assert(params Microsoft.Extensions.AI.Evaluation.EvaluationResult[] results);

Parameters

results Microsoft.Extensions.AI.Evaluation.EvaluationResult[]

One or more Microsoft.Extensions.AI.Evaluation.EvaluationResult instances to check.

Exceptions

QualityGateFailedException
Thrown when one or more thresholds are violated. The exception message lists every violation.

EvaluationQualityGate.RequireBoolean(string, bool) Method

Requires a Microsoft.Extensions.AI.Evaluation.BooleanMetric to equal expected.

public NexusLabs.Needlr.AgentFramework.Evaluation.EvaluationQualityGate RequireBoolean(string metricName, bool expected);

Parameters

metricName System.String

The metric name (use evaluator *MetricName constants).

expected System.Boolean

The required boolean value.

Returns

EvaluationQualityGate
This gate instance for fluent chaining.

EvaluationQualityGate.RequireNumericMax(string, double) Method

Requires a Microsoft.Extensions.AI.Evaluation.NumericMetric to be at most max.

public NexusLabs.Needlr.AgentFramework.Evaluation.EvaluationQualityGate RequireNumericMax(string metricName, double max);

Parameters

metricName System.String

The metric name (use evaluator *MetricName constants).

max System.Double

The maximum allowed value (inclusive).

Returns

EvaluationQualityGate
This gate instance for fluent chaining.

EvaluationQualityGate.RequireNumericMin(string, double) Method

Requires a Microsoft.Extensions.AI.Evaluation.NumericMetric to be at least min.

public NexusLabs.Needlr.AgentFramework.Evaluation.EvaluationQualityGate RequireNumericMin(string metricName, double min);

Parameters

metricName System.String

The metric name (use evaluator *MetricName constants).

min System.Double

The minimum allowed value (inclusive).

Returns

EvaluationQualityGate
This gate instance for fluent chaining.