ILangfuseExperimentRun

NexusLabs.Needlr.AgentFramework.Langfuse

ILangfuseExperimentRun Interface

Represents an in-progress Langfuse experiment (dataset run). Each call to BeginItemAsync(string, string, IEnumerable<string>, IReadOnlyDictionary<string,string>, CancellationToken) starts a trace for one dataset item and links it to this run, so the scores recorded on that trace roll up into the run's experiment-comparison view.

public interface ILangfuseExperimentRun

Remarks

The referenced dataset and its items must already exist (see ILangfuseDatasetClient). The run itself is created implicitly by Langfuse on the first linked item. Run names are caller-supplied (for example a git SHA or CI run id) so runs are comparable and reproducible.

Properties

ILangfuseExperimentRun.DatasetName Property

Gets the dataset this run is scored against.

string DatasetName { get; }

Property Value

System.String

ILangfuseExperimentRun.RunName Property

Gets the run name (for example a git SHA or CI run id).

string RunName { get; }

Property Value

System.String

Methods

ILangfuseExperimentRun.BeginItemAsync(string, string, IEnumerable<string>, IReadOnlyDictionary<string,string>, CancellationToken) Method

Begins a scenario for one dataset item and links its trace to this run as a dataset-run-item. Run your agent under the returned scenario and record scores on it as usual; they aggregate into the experiment view for this run.

System.Threading.Tasks.Task<NexusLabs.Needlr.AgentFramework.Langfuse.ILangfuseScenario> BeginItemAsync(string datasetItemId, string? scenarioName=null, System.Collections.Generic.IEnumerable<string>? tags=null, System.Collections.Generic.IReadOnlyDictionary<string,string>? metadata=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));

Parameters

datasetItemId System.String

The id of the dataset item being evaluated.

scenarioName System.String

An optional trace name. Defaults to a name derived from the dataset and item id.

tags System.Collections.Generic.IEnumerable<System.String>

Optional trace tags.

metadata System.Collections.Generic.IReadOnlyDictionary<System.String,System.String>

Optional filterable trace metadata.

cancellationToken System.Threading.CancellationToken

A cancellation token.

Returns

System.Threading.Tasks.Task<ILangfuseScenario>
The scenario for this item. Dispose it when the item run completes.