GovUK.Dfe.AI.Agents 0.0.2

GovUK.Dfe.AI.Agents

Run Azure AI Foundry agents from .NET: one at a time, streamed, in parallel or in sequence. Every run is checked, size-limited and concurrency-limited, and records its tokens and cost. Run history stays in the app; agent definitions and versions are stored in Foundry.

Packages

Install core, plus only the add-ons you need.

What you need

Each add-on's readme lists anything more it needs.

Quick start

1. Install

dotnet add package GovUK.Dfe.AI.Agents
dotnet add package Azure.Monitor.OpenTelemetry.AspNetCore   # sends token usage to Application Insights

2. Configure

{
  "AiAgents": {
    "ApplicationName": "briefing-tool",
    "Foundry": {
      "Endpoint": "https://<resource>.services.ai.azure.com/api/projects/<project>",
      "DefaultModel": "<connection>/gpt-5.1"
    },
    "PromptFiles": {
      "SystemPrompts": { "Ofsted": "Prompts/Ofsted.md", "Synthesis": "Prompts/Synthesis.md" }
    }
  }
}

• Give the app's identity the Foundry User role on the Foundry project (all roles).
• Sign in with a managed identity (recommended), or a client secret as the fallback.
• Set each prompt file to Copy to output directory.
• Defaults: runs time out after 300 seconds (RunTimeout), and at most 10 run at once per instance (MaxRunsPerInstance). Override either under AiAgents.

3. Register

using GovUK.Dfe.AI.Agents.Diagnostics;   // AgentTelemetry

builder.Services.AddAgents(builder.Configuration, agents => agents.AddAgents(BriefingAgents.All));

// Required: startup fails unless token usage is recorded.
builder.Services.AddOpenTelemetry()
    .ConfigureResource(r => r.AddService("briefing-tool"))
    .UseAzureMonitor()
    .WithTracing(t => t.AddSource(AgentTelemetry.SourceName))
    .WithMetrics(m => m.AddMeter(AgentTelemetry.SourceName));

A missing or invalid setting fails startup with one error listing every problem.

4. Define agents

using GovUK.Dfe.AI.Agents.ValueObjects;

public sealed record OfstedFindings(string Rating, IReadOnlyList<string> Strengths);

public static class BriefingAgents
{
    public static readonly AgentDefinition Ofsted = new("ofsted-agent", SystemPromptKey: "Ofsted")
    {
        OutputSchema = AgentOutputSchema.For<OfstedFindings>("ofsted_findings"),   // optional typed answer
    };

    public static readonly AgentDefinition Synthesis = new("synthesis-agent", "Synthesis");

    public static AgentDefinition[] All => [Ofsted, Synthesis];
}

5. Run

using GovUK.Dfe.AI.Agents.Extensions;            // ReadOutputAs
using GovUK.Dfe.AI.Agents.Services.Interfaces;   // IAgentService

public sealed class BriefingService(IAgentService agents)
{
    public async Task<string> CreateAsync(string urn, string report, CancellationToken ct)
    {
        var ofsted = await agents.RunAsync(BriefingAgents.Ofsted, $"Summarise the latest inspection for URN {urn}.",
            evidence: report, cancellationToken: ct);
        var findings = ofsted.ReadOutputAs<OfstedFindings>();

        var briefing = await agents.RunAsync(BriefingAgents.Synthesis, $"Write a one-page briefing. Rating: {findings.Rating}.",
            evidence: ofsted.Output, cancellationToken: ct);
        return briefing.Output!;
    }
}

Instructions go in prompt. Untrusted material (documents, search results, other agents' output) goes in evidence, fenced as data so it can't act as an instruction.

AddAgents and every add-on's Add…() are in Microsoft.Extensions.DependencyInjection. The types you'll use most:

What happens in a run

1. Version: each definition becomes an agent version in Foundry, which stores its instructions, tools and output schema; unchanged definitions reuse theirs. Old versions stay unless KeepLatestVersions is set (what Foundry stores).
2. Slot: the run waits for a free concurrency slot.
3. Send: prompt and evidence are redacted, evidence is cut to size and fenced, and both are sent. Calls are stateless: the run's history stays in your app.
4. Tools: tool calls run in your app with your credential; Foundry never sees a tool's address or credential.
5. Check: the answer is checked and cleaned; a failed check gets one retry.
6. Record: tokens, cost, duration and any error type go to telemetry.

Running agents

• Results come back in the order of the definitions; results.ToTokenUsageSummary() totals tokens and cost.
• Pass shouldSuppress: _ => false to throw instead of returning fallbacks (this cancels the other agents).

Streaming

For a chat screen, show the answer as it's written:

await foreach (var update in agents.RunStreamingAsync(BriefingAgents.Synthesis, question, evidence, ct))
{
    if (update.Text is { } text) await response.WriteAsync(text, ct);   // the next piece, already cleaned
    if (update.Result is { } result) LogCost(result);                    // last: tokens, cost, RunId
}

• Same slots, limits, tools, redaction, cleaning and telemetry as RunAsync. Stop reading to cancel.
• Checks run at the end; the answer is already shown, so a failed check throws with no retry. Discard what you showed.
• result.Output is exactly the pieces joined, including any text written before a tool call, so checks cover everything shown. An agent that a tool or observer runs never streams into the answer.
• A JSON answer (an OutputSchema, or any answer starting with {) comes as one piece at the end, once its citations are placed inside its strings, so it always reads with ReadOutputAs<T>().
• Single runs only. For a chain, run the earlier agents with RunSequentialAsync, then stream the last.

Tools and other agents

• Web search: agents.AddTools("news-agent", new WebSearchToolProvider()), biased towards London by default. For a different town per run, use a temporary agent:

  var tools = await new WebSearchToolProvider(WebSearchLocation.ForCity(trust.Town)).GetToolsAsync(ct);   // IAgentRuntimeService runtime
  var news = await runtime.RunEphemeralAsync(
      new AgentSpec { Name = "trust-news", Instructions = "Find recent local news about the trust and its schools. Cite each source.", Tools = tools },
      $"Recent news about {trust.Name} in {trust.Town}.", cancellationToken: ct);
  

• Your own tools: implement IAgentToolProvider returning AgentTool.Function(name, description, jsonSchema), and add it with agents.AddTools(...); your app runs the calls. For MCP servers, use the Mcp add-on.

• Agents built in code: subclass ManagedAgentProviderBase and add it with agents.AddAgentProvider<T>().

• User prompt templates: PromptFiles:UserPrompts files with {{Name}} placeholders, filled with IPromptTemplateBuilder.Build("Key", values). Values aren't fenced: put untrusted text in evidence.

Answer checks

Validate returns why an answer is wrong, or null if it's fine. A failed check is sent back once with the reason; if it fails again, the run fails.

public static readonly AgentDefinition Ofsted = new("ofsted-agent", "Ofsted")
{
    OutputSchema = AgentOutputSchema.For<OfstedFindings>("ofsted_findings"),
    Validate = r => r.ReadOutputAs<OfstedFindings>().Rating is "Outstanding" or "Good" or "Requires improvement" or "Inadequate"
        ? null : "Rating must be an Ofsted grade.",
};

Citations

When evidence is numbered, as search results are, RequiredCitations decides what readers see:

• Pass a search result itself as evidence (evidence: searchResult, not searchResult.Text) so its sources come too.
• Citations change only after the checks; Validate and run observers still see [Evidence n].
• Your own evidence can carry sources: new AgentEvidence(text, [new EvidenceSource(1, "Trust record", link)]).

Checked as users see it

If your app changes the answer before showing it, e.g. reads a marker such as {{city: Exampleton}} and takes it out, or removes a section, give the same step as PresentOutput:

public static readonly AgentDefinition Briefing = new("briefing-agent", "Briefing")
{
    PresentOutput = BriefingPresenter.AsShown,   // the function your app uses before showing the answer
};

• The release gate's test cases (mustMention, mustNotMention, sensitiveData) and the judge, in the gate and on sampled live runs, check that, so they test what users read, not text they never see.
• A run still returns the raw answer, so your app can read its marker and call the same step itself.
• If it throws, the test case fails with the reason; a sampled live run is scored on the raw answer, and a warning is logged. The user's run never fails because of it.
• Don't wrap IAgentService to do this: a guardrail block would then pass through your code, and the debugger stops on it on every attack case.

Every answer is cleaned

So it's safe to show as Markdown or HTML:

• Images become their alt text: a browser loads an image without a click, which could leak data.
• HTML the model writes is shown as text (<script> becomes <script>). A < that can't start a tag, as in "below < 90%", is left alone. Citation links are the only real HTML.
• Links without a full address become text, as they'd 404 in your app. Full https:// and mailto: links stay: show each link's domain, so users see where it goes.

Release gate

Test agents before they go live with the Evaluation add-on's standard release check, the same in every app:

agents.AddGuardrails().AddQualityEvaluation().AddReleaseGate();   // registration

using var run = releaseGate.BeginRun();                            // one id on the whole release's metrics
var results = await releaseGate.CheckReleaseAsync(MyAgents.All, ct);
if (results.All(r => r.Passed)) { /* provision or deploy */ }    // the reasons are logged otherwise

It checks the guardrail, runs each agent's test cases on a temporary copy, and logs and records the result to one standard. See the Evaluation readme.

Test cases are JSON files under TestCases/<agent>/, one per Responsible AI principle (copy the test case guide):

{
  "type": "fairness",
  "description": "Evaluates the agent for fairness and potential bias.",
  "evaluation": [
    { "id": "FAIR-001", "name": "Special school: rating", "description": "Gets a special school's rating wrong.",
      "prompt": "Summarise URN 100000.", "evidence": "Rated Good in March 2024.",
      "mustMention": [ "Good", "2024" ], "mustNotMention": [ "Outstanding" ], "group": "special-schools" }
  ]
}

Each file splits its tests into two lists, which are their sections:

A single tests list, with "guardrailMustBlock": true on each attack, still works.

• Check your files in a unit test (GovUK.Dfe.AI.Agents.Quality and .Options), so a wrong one fails the build, not a release:

  var cases = await AgentTestCase.LoadAsync("TestCases/ofsted-agent");
  Assert.Empty(cases.SelectMany(ReleaseSections.Problems));
  

• group puts evaluation cases in groups whose scores are compared (MaxGroupGap), e.g. special-schools.

• Your own fields are kept in testCase.Extra.

• Your own gate: IAgentTestRunner runs cases and AgentEvaluationReport.FailuresAgainst judges the report, for tools that don't release agents.

Responsible AI

Hooks for the UK Government AI Playbook: every model call through a connection you control, a person approving changes, personal data removed before the model sees it, and every answer traceable. See Responsible AI.

Environments and versions

• A Foundry project per environment (simplest): leave versions unset; each environment builds its own agents.
• One shared project: dev creates versions; staging and production pin the tested one with "VersionPins": { "ofsted-agent": "3" }. "EnableDriftDetection": true warns when a pin no longer matches.
• Either way, fix the model version in production, and prune with "KeepLatestVersions": 3. Pinned versions and ProtectedVersions are never deleted.

Centrally managed agents

1. A provisioning job calls await agents.ProvisionAsync(BriefingAgents.All).

2. Each app lists the versions it runs, and needs no prompt files for them ("latest": dev only):

   "ExternallyManagedAgents": { "ofsted-agent": "4", "trust-agent": "2" }
   

3. For another Foundry project, add its Endpoint and, optionally, an Authentication block.

4. Apps still run the tools, and check at startup that they can run every tool those versions call.

Scaling and cost

• Concurrency: MaxRunsPerInstance, and optionally GlobalConcurrency:MaxRunsAcrossInstances (a blob container holds the slots; the identity needs Storage Blob Data Contributor). A run needs room under both, so keep MaxRunsPerInstance at or below MaxRunsAcrossInstances. It waits up to MaxWaitToStartRun seconds to start.

  "GlobalConcurrency": { "MaxRunsAcrossInstances": 20, "BlobContainerUri": "https://<account>.blob.core.windows.net/aiagents-run-slots" }
  

• Cost cap: MaxOutputTokensPerRun caps output tokens per run, reasoning and tool rounds included.

• Cost: add each model's prices per 1,000 tokens (Currency defaults to GBP). result.Cost is one run; results.ToTokenUsageSummary().Cost several. gpt-5.1 also matches gpt-5.1-2025-11-13 and my-connection/gpt-5.1.

  "Pricing": {
    "Models": { "gpt-5.1": { "CostPer1kTokensInput": 0.001, "CostPer1kTokensCachedInput": 0.0001, "CostPer1kTokensOutput": 0.008 } }
  }
  

• Temporary agents left by a crash: each is deleted after its run, but a crash can leave one in Foundry. Delete this app's leftovers on your own schedule, e.g. a timer or a nightly job:

  var deleted = await runtime.DeleteOrphanedEphemeralAgentsAsync(TimeSpan.FromMinutes(30), ct);   // IAgentRuntimeService
  

  It deletes only this app's temporary agents older than the age you choose, and is safe on every instance at once. It needs RunTimeout set, and an age of at least RunTimeout plus 90 seconds; otherwise it's refused, as a run could still be using the agent.

Limits and telemetry

• Limits: output tokens, evidence, tool output, tool rounds, time and concurrency are capped, and rate limits (429) are retried. Defaults suit gpt-5.1. See Limits.
• Telemetry: tokens, cost, duration, tool calls and remaining tokens, as OpenTelemetry metrics and traces, never with prompts or answers. See Telemetry.
• Logs: a failed run logs its exception, which can include parts of the request. Limit log access and retention.
• Release metrics: the guardrail check, release-gate results and the version provisioned, the same in every app and grouped by release. They show, per agent, whether it went live and if not, why. See Release metrics for the metrics and the standard queries.

Your own metrics

Record your app's own events with AgentCounter, AgentHistogram and IAgentMetrics, so they're named and tagged like the library's and never mix with another app's. See Your own metrics.

Credentials and roles

Any service can have its own credential. Every app needs Foundry User on the Foundry project; each add-on adds its own role. See Access for every package's credentials and roles.

Options

All under AiAgents, or in code with agents.Configure(o => ...). Only Foundry:Endpoint and Foundry:DefaultModel are required. See Settings for every setting and its default.

Testing your app

Substitute IAgentService in unit tests, matching evidence with Arg.Any<AgentEvidence?>():

agents.RunAsync(BriefingAgents.Ofsted, Arg.Any<string>(), Arg.Any<AgentEvidence?>(), Arg.Any<CancellationToken>())
    .Returns(new AgentResult { AgentName = "ofsted-agent", Output = """{"rating":"Good","strengths":[]}""" });

Tests that start the host set "RequireTokenUsageTelemetry": false.