13
merged improvements
27–31 July 2026
15/15
Fabric tables verified
Kingsway silver + gold
6/6
discovery traps
behavioural checks passed
2/7
modelling DQ recall
best current scored run
What has shippedmerged delivery items on the current main branch
| capability | delivery refs | outcome | state |
|---|---|---|---|
| Runtime reliability | #1, #9, #11 | Azure OpenAI routing, rich Spark error context and transient-only retries. | merged |
| Generated-config safety | #5, #10, #16 | Stable YAML invariants, deterministic source binding and unknown-member seeding. | merged |
| Data-quality gates | #12, #13, #14 | Configurable checks, unresolved-FK coverage and PR auto-complete blocked on DQ failure. | merged |
| Governance & insight | #3, #6, #8, #15 | Loud reconciliation failure, catalogue metrics, per-agent usage and semantic deduplication. | merged |
State of the agent suites
operational
Discovery
Six specialised agents produce a cited requirements catalogue, decisions and a human-action queue. Operational flow works; golden structural accuracy is not yet where it needs to be.
developing
Modelling
Source profiling, dimensional modelling and mapping generation run end to end. Mapping output works, but planted DQ recall remains the clearest quality gap.
proven in Fabric
Build
The 14-step ticket-to-table workflow includes bounded correction, deterministic validation, Fabric execution, DQ gates, reconciliation and PR integration.
Evaluation readoutdifferent suites measure different outcomes—do not average them into one claim
| area | current evidence | next move |
|---|---|---|
| Discovery accuracy | Current scored runs dodge 6/6 behavioural traps but structurally match 0/33 golden target items. | Improve phase-aware evaluation and expand later-phase extraction. |
| Modelling DQ recall | Golden-input run finds 2/7 planted issues; discovery-input run finds 0/7. | Strengthen profiling, source binding and typed DQ evidence. |
| Build breadth | Real proof covers one Kingsway star schema; 15/15 tables and row counts pass. | Repeat across new engagements and complex/bespoke mappings. |
| Production operations | Concurrency, rerun safety, cost controls and security need further hardening. | Close production epic and run controlled shadow deployments. |
Recommended next tranche
1
Raise discovery and modelling eval quality
Use the Kingsway golden and planted-DQ fixtures as release gates, not just dashboards.
2
Prove repeatability
Run at least two further engagement shapes, including a complex fact and silver transform.
3
Productionise orchestration
Close concurrency, rerun, cost and security controls before unattended client delivery.