AI / ML
Models built on clinically labelled data, with the dataset, the training record and the performance evidence assembled the way a regulator will ask to see them.
When AI and machine learning are the right step
When a product must interpret images or signals to screen, diagnose or monitor. The model is medical device software, so the data and the testing matter as much as the accuracy.
Read medical images
Classify or measure findings in images such as scans, slides or photographs, with clinicians setting the reference.
Interpret signals
Detect patterns in signals such as ECG, breathing or motion, on the device or in the cloud.
Prepare a model for approval
A model that works in the lab and now needs a documented dataset, a locked version and a test on unseen data.
What you bring, and what you get back
You bring
- The clinical question, and who the output is for
- Data you already hold, and its consent terms
- Access to clinicians for labelling, if you have it
- The device or software the model runs in
- Any earlier models and their results
You get back
- Dataset description and labelling protocol
- Model and training record
- A locked model version for testing
- Performance report on held-out data
- Plan for future model changes
- Inputs on model errors for the risk file
Test on data the model has never seen
A model tested on data it has already seen looks better than it is. Split the data before training: one set to learn from, one to tune and choose the model, and one held back. Lock the chosen model, then test it once on the held-back set. Keep each patient in one set only, or the test is not independent. The good machine learning practice principles from FDA, Health Canada and MHRA stress the same separation.
How AI and ML development works
Questions clients ask
How much data do we need?
It depends on the task, how varied the patients are and the performance you must show. The test set must be large enough to estimate performance with useful confidence, and it should reflect the people the device will be used on.
Can the model keep learning after launch?
Not freely. A regulated model is usually locked, and a change is planned, tested and approved like any other design change. Some regulators accept a plan agreed in advance that sets which changes may be made and how they are tested.
Who labels the data?
Clinicians, working to a written labelling protocol. Where experts disagree, the protocol sets how the reference label is decided.
Is an AI model a medical device?
Software that analyses data to diagnose, screen or monitor is usually regulated as a medical device. See AI and software as a medical device for how the routes differ.
Data labelled with clinicians
We work with 100+ clinician partners, and our datasets are collected and labelled with clinicians. AI is one of nine disciplines in one team, beside software, verification and regulatory.
Devices we have worked on
We name the type of device, not the client. Client work stays confidential. See every device type.
Run under an ISO 13485:2016 quality system certified by TÜV SÜD, certificate 99 400 00046. What it covers.
Not quite you?
Standards this work is done against
- IEC 62304Medical device software life cycle
- ISO 14971Risk management
- IEC 62366-1Usability engineering
- GMLPGood machine learning practice principles (FDA, Health Canada, MHRA)
The list for a given device, and the edition of each, is set in its regulatory plan.
Where this sits
Also in Co-Design & Develop
Tools for this
Have a question about AI / ML?
Opens the contact form with this service named. Nothing is sent until you press send.