ILangfuseScenario
NexusLabs.Needlr.AgentFramework.Langfuse¶
ILangfuseScenario Interface¶
Represents a single Langfuse trace scoped to one eval scenario or agent run. Created via BeginScenario(string, string, string, IEnumerable<string>, IReadOnlyDictionary<string,string>).
Implements System.IDisposable
Remarks¶
Beginning a scenario starts a root OpenTelemetry span carrying Langfuse trace-level attributes (name, session id, user id, tags, metadata). Agent telemetry produced while the scenario is active nests under this trace. Disposing the scenario ends the root span.
Evaluation scores are attached to the scenario's trace through the Langfuse Scores API, keyed by TraceId. Because Langfuse links scores to traces asynchronously, scores may be recorded while the scenario is still open.
Properties¶
ILangfuseScenario.Activity Property¶
Gets the root Activity for this scenario, or null when the owning session is disabled or no listener is sampling the span.
Property Value¶
ILangfuseScenario.TraceId Property¶
Gets the Langfuse trace id (the lowercase-hex OpenTelemetry trace id) for this scenario, or null when the owning session is disabled.
Property Value¶
Methods¶
ILangfuseScenario.RecordEvaluationAsync(EvaluationResult, CancellationToken) Method¶
Projects every metric in a Microsoft.Extensions.AI.Evaluation.EvaluationResult to a Langfuse score on this scenario's trace. Numeric metrics become numeric scores, boolean metrics become boolean scores, and string metrics become categorical scores. Each metric's reason is sent as the score comment. Metrics whose value is unset are skipped.
System.Threading.Tasks.Task RecordEvaluationAsync(Microsoft.Extensions.AI.Evaluation.EvaluationResult result, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));
Parameters¶
result Microsoft.Extensions.AI.Evaluation.EvaluationResult
The evaluation result to project.
cancellationToken System.Threading.CancellationToken
A cancellation token.
Returns¶
System.Threading.Tasks.Task
A task that completes when Langfuse has accepted all projected scores.
ILangfuseScenario.RecordScoreAsync(string, bool, string, CancellationToken) Method¶
Records a boolean score (stored as 1/0) on this scenario's trace.
System.Threading.Tasks.Task RecordScoreAsync(string name, bool value, string? comment=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));
Parameters¶
name System.String
The score name.
value System.Boolean
The boolean value.
comment System.String
An optional explanation surfaced in Langfuse.
cancellationToken System.Threading.CancellationToken
A cancellation token.
Returns¶
System.Threading.Tasks.Task
A task that completes when Langfuse has accepted the score.
ILangfuseScenario.RecordScoreAsync(string, double, string, CancellationToken) Method¶
Records a numeric score on this scenario's trace.
System.Threading.Tasks.Task RecordScoreAsync(string name, double value, string? comment=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));
Parameters¶
name System.String
The score name.
value System.Double
The numeric value.
comment System.String
An optional explanation surfaced in Langfuse.
cancellationToken System.Threading.CancellationToken
A cancellation token.
Returns¶
System.Threading.Tasks.Task
A task that completes when Langfuse has accepted the score.
ILangfuseScenario.RecordScoreAsync(string, string, string, CancellationToken) Method¶
Records a categorical score on this scenario's trace.
System.Threading.Tasks.Task RecordScoreAsync(string name, string value, string? comment=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));
Parameters¶
name System.String
The score name.
value System.String
The category label.
comment System.String
An optional explanation surfaced in Langfuse.
cancellationToken System.Threading.CancellationToken
A cancellation token.
Returns¶
System.Threading.Tasks.Task
A task that completes when Langfuse has accepted the score.
ILangfuseScenario.RecordSessionScoreAsync(string, bool, string, CancellationToken) Method¶
Records a boolean session score (stored as 1/0) for this scenario's session.
System.Threading.Tasks.Task RecordSessionScoreAsync(string name, bool value, string? comment=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));
Parameters¶
name System.String
The score name.
value System.Boolean
The boolean value.
comment System.String
An optional explanation surfaced in Langfuse.
cancellationToken System.Threading.CancellationToken
A cancellation token.
Returns¶
System.Threading.Tasks.Task
A task that completes when Langfuse has accepted the score.
ILangfuseScenario.RecordSessionScoreAsync(string, double, string, CancellationToken) Method¶
Records a numeric score against the session this scenario belongs to (the session id passed when the scenario began), scoring the whole multi-turn conversation rather than this single trace. Surfaced as a non-fatal skip when the scenario has no session id.
System.Threading.Tasks.Task RecordSessionScoreAsync(string name, double value, string? comment=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));
Parameters¶
name System.String
The score name.
value System.Double
The numeric value.
comment System.String
An optional explanation surfaced in Langfuse.
cancellationToken System.Threading.CancellationToken
A cancellation token.
Returns¶
System.Threading.Tasks.Task
A task that completes when Langfuse has accepted the score.
ILangfuseScenario.RecordSessionScoreAsync(string, string, string, CancellationToken) Method¶
Records a categorical session score for this scenario's session.
System.Threading.Tasks.Task RecordSessionScoreAsync(string name, string value, string? comment=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));
Parameters¶
name System.String
The score name.
value System.String
The category label.
comment System.String
An optional explanation surfaced in Langfuse.
cancellationToken System.Threading.CancellationToken
A cancellation token.
Returns¶
System.Threading.Tasks.Task
A task that completes when Langfuse has accepted the score.
ILangfuseScenario.SetInput(object) Method¶
Sets the trace-level input shown at the top of the trace in Langfuse. Strings are stored verbatim; other values are serialized to JSON. No-op when the owning session is disabled.
Parameters¶
input System.Object
The input value (typically the eval's prompt or request).
ILangfuseScenario.SetOutput(object) Method¶
Sets the trace-level output shown at the top of the trace in Langfuse. Strings are stored verbatim; other values are serialized to JSON. Typically called with the agent's final answer before the scenario is disposed. No-op when the owning session is disabled.
Parameters¶
output System.Object
The output value (typically the agent's final answer).
ILangfuseScenario.SetTracePublic(bool) Method¶
Marks this scenario's trace as public so Langfuse exposes it via a shareable URL that needs no login — useful for linking a failing eval trace in a pull request. No-op when the owning session is disabled.
Parameters¶
isPublic System.Boolean
Whether the trace is public. Defaults to true.
ILangfuseScenario.SetVersion(string) Method¶
Sets the version of this scenario's trace (for example a git SHA or prompt/logic version),
emitted as langfuse.version. No-op when the owning session is disabled.
Parameters¶
version System.String
The version identifier.