Evidence
What to measure before expanding authority
Compare three baselines on the same cases: current manual handling, existing deterministic rules and the proposed Jev-assisted workflow. Separate development examples from held-out evaluation and later shadow-production samples.
Evidence status legend
Architecture, thresholds or flows recommended in this guide. Not yet run in a customer tenant.
Illustrative values, Contoso-style scenarios or lab fixtures. Useful for teaching the loop; not a result.
Held-out or shadow results with denominators, error costs and agreed acceptance gates. Replace proposal labels only when this exists.
Live authority with operational ownership, kill switch and ongoing scorecard. None claimed on this site yet.
Suggested pilot targets
Negotiate rather than represent as achieved results:
- At least 95% correct routing among automatically routed items
- At least 50% automation coverage
- Zero unauthorised actions
- p95 end-to-end latency from the customer’s process requirement
Report uncertainty intervals and denominators. A small error-free sample does not establish a low incident rate.
Scorecard rhythm
| Owner | Required output | Review rhythm |
|---|---|---|
| Business process owner | Accepted routes and error-cost assumptions | Weekly in pilot |
| Platform owner | Availability, latency, queues, recovery | Daily operations |
| Security and data owner | Data boundary, retention, access | Before launch and on change |
| Model evaluation lead | Versioned labelled set and threshold report | Every model or policy release |
| Commercial owner | Outcome value and complete service cost | Monthly |
Mark Red when an agreed safety or service limit is breached. Do not average away a critical red control.
Cost note (illustrative)
Vendor documentation checked around the verification date listed model inference in the order of cents per hundred thousand items at stated token assumptions. Review labour and error remediation dominate the business case. Invented planning assumptions in related materials are labelled as such and are not customer results.