ToolInvocationRunner

NexusLabs.Needlr.AgentFramework.Testing

ToolInvocationRunner Class

Test harness for invoking [AgentFunction]-decorated tool methods through their source-generated Microsoft.Extensions.AI.AIFunction wrapper, with the same plumbing Microsoft.Extensions.AI.FunctionInvokingChatClient uses in production.

public sealed class ToolInvocationRunner

Inheritance System.Object 🡒 ToolInvocationRunner

Example

Minimal test, full bring-your-own service provider:

var sp = new ServiceCollection()
    .AddAgentFrameworkAccessors()
    .AddSingleton<GrepTool>()
    .BuildServiceProvider();

var runner = new ToolInvocationRunner(sp)
    .WithWorkspace(ws => ws.TryWriteFile("a.txt", "hi"));

var result = await runner.InvokeAsync<GrepTool>("grep_files", a =>
{
    a["pattern"] = "hi";
    a["path"]    = "/";
});

result.AssertSuccess();
result.AssertResultContains("a.txt");

Remarks

The runner exists to remove the boilerplate consumers face when testing tools they wrote for Needlr's Agent Framework integration: build a service provider, register the tool, look up the source-generated NexusLabs.Needlr.AgentFramework.IAIFunctionProvider, find the right Microsoft.Extensions.AI.AIFunction by name, build Microsoft.Extensions.AI.AIFunctionArguments, establish an ambient NexusLabs.Needlr.AgentFramework.Context.IAgentExecutionContext so the tool can read NexusLabs.Needlr.AgentFramework.Context.IAgentExecutionContextAccessor.Current, and invoke.

By default the runner resolves only via the source-generated NexusLabs.Needlr.AgentFramework.IAIFunctionProvider registered by the [ModuleInitializer] the Needlr Agent Framework generator emits in the consuming assembly. This is the same path production agents take. To exercise the reflection-based Microsoft.Extensions.AI.AIFunctionFactory path instead, call GetFunctionAllowingReflection<TTool>(string) explicitly — that method carries System.Diagnostics.CodeAnalysis.RequiresUnreferencedCodeAttribute annotations because it is incompatible with NativeAOT.

Instances are immutable: each With* method returns a new runner with the configuration applied. Each InvokeAsync<TTool>(string, Action<AIFunctionArguments>, CancellationToken) call creates a fresh Microsoft.Extensions.DependencyInjection.IServiceScope (when an Microsoft.Extensions.DependencyInjection.IServiceScopeFactory is available) so tools with scoped dependencies behave correctly across invocations.

Constructors

ToolInvocationRunner(IServiceProvider) Constructor

Creates a runner over an already-built System.IServiceProvider. Use this when the test fixture builds its own DI container (typical for Syringe-based test fixtures, or for tests that need to share a provider across multiple invocations).

public ToolInvocationRunner(System.IServiceProvider serviceProvider);

Parameters

serviceProvider System.IServiceProvider

Provider that has the tool type and its dependencies registered. The Needlr accessors (NexusLabs.Needlr.AgentFramework.Context.IAgentExecutionContextAccessor, NexusLabs.Needlr.AgentFramework.Diagnostics.IAgentDiagnosticsAccessor) must also be registered — call NexusLabs.Needlr.AgentFramework.AgentFrameworkAccessorServiceCollectionExtensions.AddAgentFrameworkAccessors(Microsoft.Extensions.DependencyInjection.IServiceCollection) or use the broader UsingAgentFramework() Syringe extension.

Properties

ToolInvocationRunner.IsGeneratedProviderAvailable Property

Whether a source-generated NexusLabs.Needlr.AgentFramework.IAIFunctionProvider is currently registered with NexusLabs.Needlr.AgentFramework.AgentFrameworkGeneratedBootstrap. Useful as a precondition assertion in tests that require the source generator to have run for at least one assembly in the process.

public bool IsGeneratedProviderAvailable { get; }

Property Value

System.Boolean

Remarks

This property tells you only whether \<em>any\</em> generated provider exists globally; it does not tell you whether \<em>your specific tool\</em> is resolvable. Call GetFunction<TTool>(string) if you need to verify a specific function — the error message it throws when a type or method is missing is the canonical signal.

Methods

ToolInvocationRunner.AssertGeneratedProviderAvailable() Method

Throws when no source-generated NexusLabs.Needlr.AgentFramework.IAIFunctionProvider is registered. Use this as a fail-fast guard in tests that depend on the generator output for at least one assembly in the process.

public void AssertGeneratedProviderAvailable();

Exceptions

System.InvalidOperationException
Thrown when no generated provider is available, with guidance on how to enable it.

Remarks

In practice this rarely throws because the Needlr Agent Framework assembly itself emits a (possibly empty) provider via [ModuleInitializer], which registers as soon as NexusLabs.Needlr.AgentFramework.dll loads. The check is still useful as a sanity guard in environments where modules may not have initialized yet (e.g. some custom AssemblyLoadContext scenarios).

ToolInvocationRunner.Create(Action<IServiceCollection>) Method

Creates a runner over a fresh service provider with only the Needlr accessors registered. Use this overload when registering multiple tools or when you want full control over the service collection setup.

public static NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationRunner Create(System.Action<Microsoft.Extensions.DependencyInjection.IServiceCollection>? configureServices=null);

Parameters

configureServices System.Action<Microsoft.Extensions.DependencyInjection.IServiceCollection>

Hook to register the tools and any dependencies they need.

Returns

ToolInvocationRunner

ToolInvocationRunner.CreateFor<TTool>(Action<IServiceCollection>) Method

Creates a runner over a fresh service provider with the Needlr accessors and TTool registered as a singleton. Additional services can be added via the optional configureServices callback.

public static NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationRunner CreateFor<TTool>(System.Action<Microsoft.Extensions.DependencyInjection.IServiceCollection>? configureServices=null)
    where TTool : class;

Type parameters

TTool

The tool class to register.

Parameters

configureServices System.Action<Microsoft.Extensions.DependencyInjection.IServiceCollection>

Optional hook to register additional dependencies (e.g. IHttpClientFactory, options, fakes for scoped services).

Returns

ToolInvocationRunner

ToolInvocationRunner.GetFunction(Type, string) Method

Resolves an Microsoft.Extensions.AI.AIFunction for toolType by method name via the source-generated NexusLabs.Needlr.AgentFramework.IAIFunctionProvider.

public Microsoft.Extensions.AI.AIFunction GetFunction(System.Type toolType, string methodName);

Parameters

toolType System.Type

methodName System.String

Returns

Microsoft.Extensions.AI.AIFunction

ToolInvocationRunner.GetFunction<TTool>(string) Method

Resolves an Microsoft.Extensions.AI.AIFunction for TTool by method name via the source-generated NexusLabs.Needlr.AgentFramework.IAIFunctionProvider.

public Microsoft.Extensions.AI.AIFunction GetFunction<TTool>(string methodName)
    where TTool : class;

Type parameters

TTool

The tool class declaring the [AgentFunction] method.

Parameters

methodName System.String

The function name as exposed to the LLM (defaults to the C# method name when the source generator emits the wrapper).

Returns

Microsoft.Extensions.AI.AIFunction

Exceptions

System.InvalidOperationException
Thrown when no generated provider is registered, or when no function with that name is found for the type.

ToolInvocationRunner.GetFunctionAllowingReflection(Type, string) Method

Resolves an Microsoft.Extensions.AI.AIFunction for toolType by method name, preferring the source-generated provider but falling back to reflection-based discovery via Microsoft.Extensions.AI.AIFunctionFactory.Create(System.Reflection.MethodInfo,System.Object,Microsoft.Extensions.AI.AIFunctionFactoryOptions) when the generator output is not available.

public Microsoft.Extensions.AI.AIFunction GetFunctionAllowingReflection(System.Type toolType, string methodName);

Parameters

toolType System.Type

methodName System.String

Returns

Microsoft.Extensions.AI.AIFunction

ToolInvocationRunner.GetFunctionAllowingReflection<TTool>(string) Method

Resolves an Microsoft.Extensions.AI.AIFunction for TTool by method name, preferring the source-generated provider but falling back to reflection-based discovery via Microsoft.Extensions.AI.AIFunctionFactory.Create(System.Reflection.MethodInfo,System.Object,Microsoft.Extensions.AI.AIFunctionFactoryOptions) when the generator output is not available.

public Microsoft.Extensions.AI.AIFunction GetFunctionAllowingReflection<TTool>(string methodName)
    where TTool : class;

Type parameters

TTool

Parameters

methodName System.String

Returns

Microsoft.Extensions.AI.AIFunction

Remarks

This method is incompatible with NativeAOT because the reflection branch dynamically generates marshalling code. Tests targeting AOT must use GetFunction<TTool>(string) instead.

ToolInvocationRunner.InvokeAsync(Type, string, Action<AIFunctionArguments>, CancellationToken) Method

Resolves and invokes a function for toolType. Identical to the generic overload but accepts the type as a parameter for callers who only have a runtime System.Type.

public System.Threading.Tasks.Task<NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationResult> InvokeAsync(System.Type toolType, string methodName, System.Action<Microsoft.Extensions.AI.AIFunctionArguments>? configureArgs=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken));

Parameters

toolType System.Type

methodName System.String

configureArgs System.Action<Microsoft.Extensions.AI.AIFunctionArguments>

cancellationToken System.Threading.CancellationToken

Returns

System.Threading.Tasks.Task<ToolInvocationResult>

ToolInvocationRunner.InvokeAsync<TTool>(string, Action<AIFunctionArguments>, CancellationToken) Method

Resolves the source-generated Microsoft.Extensions.AI.AIFunction for the given tool method, builds an Microsoft.Extensions.AI.AIFunctionArguments via configureArgs, establishes the configured execution context, and invokes the function. Captures any thrown exception into the returned ToolInvocationResult rather than propagating.

public System.Threading.Tasks.Task<NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationResult> InvokeAsync<TTool>(string methodName, System.Action<Microsoft.Extensions.AI.AIFunctionArguments>? configureArgs=null, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken))
    where TTool : class;

Type parameters

TTool

The tool class declaring the [AgentFunction] method.

Parameters

methodName System.String

The function name as exposed to the LLM.

configureArgs System.Action<Microsoft.Extensions.AI.AIFunctionArguments>

Optional hook to populate Microsoft.Extensions.AI.AIFunctionArguments. Pass null to invoke with no arguments.

cancellationToken System.Threading.CancellationToken

Cancellation token forwarded to Microsoft.Extensions.AI.AIFunction.InvokeAsync(Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken). Tools that accept a System.Threading.CancellationToken parameter receive this token via the source-generated wrapper.

Returns

System.Threading.Tasks.Task<ToolInvocationResult>

ToolInvocationRunner.InvokeAsync<TTool>(string, IReadOnlyDictionary<string,object>, CancellationToken) Method

Same as InvokeAsync<TTool>(string, Action<AIFunctionArguments>, CancellationToken) but accepts a pre-built dictionary of arguments. Convenient when the test already has a System.Collections.Generic.IReadOnlyDictionary<> of args ready (e.g. captured from a prior run or shared across multiple invocations).

public System.Threading.Tasks.Task<NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationResult> InvokeAsync<TTool>(string methodName, System.Collections.Generic.IReadOnlyDictionary<string,object?> arguments, System.Threading.CancellationToken cancellationToken=default(System.Threading.CancellationToken))
    where TTool : class;

Type parameters

TTool

Parameters

methodName System.String

arguments System.Collections.Generic.IReadOnlyDictionary<System.String,System.Object>

cancellationToken System.Threading.CancellationToken

Returns

System.Threading.Tasks.Task<ToolInvocationResult>

ToolInvocationRunner.LimitToTools(Type[]) Method

Returns a new runner that scopes NexusLabs.Needlr.AgentFramework.AgentFrameworkGeneratedBootstrap to expose only toolTypes during function resolution. Useful in consumer test projects that contain many [AgentFunction] types and want to assert behavior against a specific subset without disturbing other tests.

public NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationRunner LimitToTools(params System.Type[] toolTypes);

Parameters

toolTypes System.Type[]

The set of tool types visible to the source-generated provider during this runner's invocations. Must contain at least one type.

Returns

ToolInvocationRunner

Exceptions

System.ArgumentException
Thrown when toolTypes is empty.

ToolInvocationRunner.WithExecutionContext(Action<AgentExecutionContextBuilder>) Method

Returns a new runner that establishes an NexusLabs.Needlr.AgentFramework.Context.IAgentExecutionContext built by configure for the duration of each InvokeAsync call.

public NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationRunner WithExecutionContext(System.Action<NexusLabs.Needlr.AgentFramework.Testing.AgentExecutionContextBuilder> configure);

Parameters

configure System.Action<AgentExecutionContextBuilder>

Returns

ToolInvocationRunner

Remarks

Successive calls replace the previously-configured context (immutable copy semantics).

ToolInvocationRunner.WithWorkspace(IWorkspace) Method

Convenience: returns a new runner that attaches the supplied workspace to the execution context.

public NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationRunner WithWorkspace(NexusLabs.Needlr.AgentFramework.Workspace.IWorkspace workspace);

Parameters

workspace NexusLabs.Needlr.AgentFramework.Workspace.IWorkspace

Returns

ToolInvocationRunner

ToolInvocationRunner.WithWorkspace(Action<IWorkspace>) Method

Convenience: returns a new runner that creates a fresh NexusLabs.Needlr.AgentFramework.Workspace.InMemoryWorkspace, runs seed against it, and attaches it to the execution context.

public NexusLabs.Needlr.AgentFramework.Testing.ToolInvocationRunner WithWorkspace(System.Action<NexusLabs.Needlr.AgentFramework.Workspace.IWorkspace> seed);

Parameters

seed System.Action<NexusLabs.Needlr.AgentFramework.Workspace.IWorkspace>

Returns

ToolInvocationRunner