A wrong note got past review
Example · wrong-answer check · AI scribe · 20 scenarios × 8 runs
160 runs · model version recorded on eachReviewed by a named person
AI testing for health tech
Others check whether the note is true. We check whether your product stops a false one.
AI scribes and clinical AI rely on one safeguard: a clinician reviews the output. We test whether that safeguard, and every other one, holds when the AI gets it wrong, and hand you evidence your NHS customer can read.
Built for AI scribes, ambient voice and clinical AI · your staging environment · no patient data
A model update broke a safety check
Example · model-update re-check · same 20 scenarios re-run
160 runs re-checkedEvery difference reviewed by a named person
Illustration. Example values from a fictional project, not a client engagement.
Why now
Four public facts from the last three months.
The questions your NHS customers ask about your AI are changing quickly. These are the sources they are reading.
-
Model updates are now a contract matter
On 29 July 2026, NHS England and the MHRA asked NHS boards to make sure supplier contracts cover risk “including at AI model update milestones”.
Book the free session -
Buyers must check your error and omission evidence
NHS England’s ambient scribing guidance tells NHS organisations to check clinician validation of the “rate of errors, omissions and other dysfunctions”.
Book the free session -
The regulator names hallucination as a known hazard
The MHRA’s July 2026 guidance says “hallucination is a well-known behaviour of GenAI”, and treats warnings as the weakest form of risk reduction.
Book the free session -
A national safety investigation is under way
HSSIB opened an investigation into ambient voice technology in hospitals on 6 August 2026. It expects to report in summer 2027.
Book the free session
The NHS risk list
The hazards your customers already ask about, tested.
NHS England’s guidance names the risks of ambient scribing. Each one maps to faults we supply on purpose, so your evidence answers the question in the buyer’s own terms.
-
Missing critical information
We leave out allergies, medicines and the plan, and check whether anything in your product flags the gap.
-
Incorrect information
We drop a “no”, swap a drug or dose, and invent a detail, then check whether the draft can still be saved.
-
Delayed output
We slow the answer down and check what the clinician sees while waiting, and whether they can act on a stale screen.
-
System unavailable
We take the AI away mid-consultation and check that your product fails visibly and safely, never quietly.
Twelve faults, each tied to a named risk.
Drawn from published scribe evaluations and the errors patients and clinicians have reported. Each one is graded by severity against your own hazard log.
-
Something left out
An allergy, a medicine or the plan, present in the conversation and missing from the note.
-
A negative reversed
“No chest pain” recorded as “chest pain”. Published NHS evaluations name missed negatives as a frequent concern.
-
A detail invented
A finding, a history or a figure that nobody said.
-
A drug or dose swapped
A similar-sounding medicine, or the right medicine at the wrong dose.
-
The wrong speaker
The patient’s words written up as the clinician’s, or the other way round.
-
A maybe recorded as fact
“Possibly” or “we’ll consider” written up as a decision.
-
Cancelled treatment recorded as given
A plan that was discussed and dropped, recorded as done.
-
History written as examination
What the patient reported, recorded as what the clinician found.
-
Wrong patient details
A detail belonging to a different person or encounter.
-
An out-of-scope suggestion
A diagnosis or treatment the product is not meant to offer. Evidence for your regulatory team that the product stays within its stated purpose.
-
A hidden instruction
An instruction planted in what was said or uploaded, to see whether your product obeys it.
-
Slow, empty or unavailable
No answer, a late answer, or the service down mid-consultation.
What your NHS customer receives
Your safety case, with tests behind it.
Your NHS customer asks for your DCB0129 clinical safety case report and hazard log. Our evidence pack is the tested proof behind them.
Every safety control, tested
For each control: the wrong answer we supplied, what your product did, how often it held, its severity, the model version, the date and the named reviewer.
A record at every model update
When your provider changes the model, the same set runs again and every difference is reviewed. What you hand over at each update milestone in your contracts.
What the evidence is, and is not
Honest about what it measures.
Clinical safety officers trust evidence that states its limits first.
- It is tested evidence on a fixed set of scenarios, measured against your own clinicians. It is not a measure of real-world accuracy, and it does not replace monitoring once your product is in use.
- It tests your product’s safeguards, such as whether review can be skipped. It does not test whether clinicians read carefully; that sits with training and deployment.
- It supports your DCB0129 and IEC 62304 evidence. It is not a certificate, and it does not decide your product’s regulatory status.
Health tech questions
What clinical safety officers ask us first.
We already test our scribe ourselves. Why would we need this?
Most teams test whether the note is accurate. Far fewer test what the product does when it is not: whether the draft is marked, whether review can be skipped, whether an unsafe value can be saved. Independent evidence on those safeguards is also easier for an NHS customer to accept than your own.
Do you need patient data?
No. We work on your staging environment with scripted and synthetic cases. Anything that needs real records stays inside your environment, under your control.
Who decides what counts as a wrong note?
Your clinicians. Correctness is measured only against reference answers they define. Where a machine helps with scoring, we check it against your clinicians’ labels first and report how closely it agrees.
How do you handle omissions?
Automated scorers are weakest at spotting what is missing, so we do not leave omissions to them. Each case has a reference list of what must appear, and every gap is checked against it.
We are an NHS trust deploying a scribe. Can you help?
Yes. NHS organisations are expected to monitor independently of the supplier. We can re-check a supplier’s product at go-live and after each model update, and give you evidence that does not come from the supplier.
Does this cover wellness and mental health apps?
Yes. The method is the same: harmful advice, a reply that mishandles someone in distress, an invented health claim, and whether the AI drifts beyond what the app is meant to do.
Which standards does this support?
The evidence is written to sit behind your DCB0129 clinical safety case and your IEC 62304 verification records. Your clinical safety officer remains responsible for the safety case.
Sources
Where these facts come from.
Read on 3 October 2026. We quote primary sources only.
Find out whether your safeguards hold.
A free 45-minute session with your product owner and one engineer. We map the five ways your scribe could be wrong that matter most, and the safeguard meant to catch each one. You keep the map.