Where the Authority's models are built and proven
Model choice per use case, evaluation against VRA's own questions, and a registry so nothing reaches a plant or a board pack without evidence.
Model profiles
Right model for the task — the expensive one is not the default
| Profile | Use case | Evaluation | Cost |
|---|---|---|---|
Reasoning model (large) Reserved for executive drafting | Board narrative, cross-domain synthesis | 92% agreement | High |
General chat model Default for Copilot Studio agents | Document Q&A, summarisation | 89% agreement | Medium |
Small fast model Approval routing and document triage | Classification, routing, extraction | 91% on tagged set | Low |
Anomaly model (custom) Trained on labelled failure history | Sensor series on rotating equipment | 0.87 F1 (illustrative) | Compute |
Evaluation pipeline
What stands between an idea and a pilot
- 01Ground
Register the sources an agent may use: contracts, manuals, Fabric gold tables, published policy.
- 02Evaluate
A signed-off question set per agent, scored on every change before anything is released.
- 03Guardrail
Content safety, data-loss prevention, prohibited actions and a named human for every escalation.
- 04Register
Model, prompt, grounding and owner recorded so production is never a mystery.
- 05Pilot
One directorate, a measured outcome, a review date — then decide on expansion.
Agent registry
Every agent, its grounding, its guardrails and how it scored
| Agent | Owner | Evaluation | Status |
|---|---|---|---|
Board Pack Composer Drafts only; no external send; every figure linked to lineage | Office of the Chief Executive | 88% of 24 | Pilot candidate |
Procurement Threshold Advisor Cites clause; escalates beyond threshold; no award decisions | Procurement | 94% of 31 | Pilot candidate |
Fleet Condition Explainer Advisory only; no write path to OT; flags low confidence | Technical Services | 81% of 40 | Evaluation |
Receivables Analyst Read-only; customer data minimised; no dunning without review | Finance | 86% of 22 | Evaluation |
Hydrology Briefing Agent Cites data cut-off; no spill decisions | Water Resources & RE | 72% of 18 | Design |
What a pilot needs before it starts
- An Azure landing zone with private networking and data residency agreed with the Authority.
- A named business owner per agent, with authority to withdraw it.
- Evaluation question sets written by the directorate, not by the technology team.
- A responsible-AI review covering bias, safety and human escalation.
- A written position on what the agent may never do — award, approve, instruct, or operate.
The honest position on agentic tooling
What to tell a board
Multi-step agents that assemble a board pack or reconcile a subsidiary return are the most compelling part of this story — and the least proven in a regulated utility context. The recommendation is to run them in evaluation alongside a human doing the same task, measure the difference on quality and time, and publish the result. If the evidence is not there, the agent does not ship.