Wellness and mental health apps

Your app’s AI talks to people about their health. What happens when it’s wrong?

Coaching, mental health and lifestyle apps now put AI in front of people who may be vulnerable. We test what your app does when that AI gives harmful advice, mishandles someone in distress, or drifts beyond what the app is for.

Book a free session

Your staging environment · scripted and synthetic cases · no user data

A wrong note got past review

Example · wrong-answer check · AI scribe · 20 scenarios × 8 runs

2 need work
  1. Injected “No chest pain” returned as “chest pain”
  2. Safeguard Draft can’t be saved until a clinician reviews it
  3. Result Saved without review on runs 3 and 7, via keyboard shortcut
  4. Overall 18 of 20 scenarios held on all 8 runs
  5. Next Fix the shortcut, re-test until it holds 8 of 8
160 runs · model version recorded on eachReviewed by a named person

Illustration. Example values from a fictional project, not a client engagement.

Six failures that matter most in wellness.

Each one is supplied on purpose and tied to a named risk, so your safeguards either hold or they do not.

  1. Harmful advice

    Guidance that could hurt someone: extreme diets, unsafe exercise, stopping medication.

  2. Distress missed

    A reply that carries on as normal when someone says they are struggling or unsafe.

  3. An invented health claim

    A benefit, a statistic or a cure that the app has no basis for.

  4. Drifting into clinical advice

    Diagnosis or treatment the app is not meant to give. Evidence for your regulatory team that it stays within its stated purpose.

  5. A hidden instruction

    Text planted by a user to make the AI drop its own rules.

  6. Slow, empty or unavailable

    What the user sees when the AI is late, blank or gone.

What we check your app does

The safeguards your users rely on.

Does crisis wording always reach help?

We test direct and indirect ways of saying someone is unsafe, and check the app signposts support every time, not most of the time.

Are harmful suggestions stopped before they are shown?

We feed the app harmful advice and check whether anything blocks or flags it.

Does the AI stay within what the app is for?

We push it towards diagnosis and treatment, and record whether it declines.

Does it fail safely when the AI is down?

We take the service away and check the user is never left with a blank or a half-finished reply.

  • 8

    Runs per scenario

  • 6

    Failure types

  • 1

    Named reviewer

Every case runs eight times, because a safeguard that works six times out of eight is not a safeguard.

Book a free session
The limits

What this is, and is not.

Stated first, because people will rely on it.

  • It is tested evidence on a fixed set of scenarios. It is not a measure of how your app behaves for every user, and it does not replace monitoring once you are live.
  • It does not decide whether your app is a medical device. It gives your regulatory team evidence about what the AI actually does.

Find out whether your safeguards hold.

45 minutes. We map the five ways your app’s AI could do the most harm, and what is meant to stop each one. You keep the map.

Book a free session Health tech overview