Regulating algorithms in healthcare: the GDPR and IVDR in practice

This report summarises the findings of the workshop: Regulating algorithms in healthcare: the GDPR and IVDR in practice.

This workshop brought together those with expertise in data protection and medical device law. These two fields face similar problems and may learn from each other.

Regulating algorithms in healthcare: the GDPR and IVDR in practice is free to download

The workshop generated the following key findings:

• Different regulatory bodies speak in different and often contradictory terms, using (and sometimes not defining) such terms as ‘algorithms’, ‘software’, ‘computer programs’ and ‘automated processing’
• As algorithms increase in complexity and ubiquity, there is growing interest in the ethical challenges associated with their use, and particularly in the novel ethical challenges that may arise
• Determining what constitutes ‘meaningful transparency’ in a given context is a key priority for the field
• There is disagreement over whether there is in fact a right to explanation for individual decisions under the GDPR and what kind of explanation might satisfy this right if it does exist
• Counterfactual explanations may be one way of satisfying the right to explanation. However, they have limitations and are unlikely to be helpful when making judgments about the fairness of a system
• Many algorithms used in healthcare will be regulated as medical devices and will be subject to the new MDR or IVDR. It is therefore important to understand how these Regulations differ from the previous three Directives
• Arguably, EU and US methods to determine whether a device is regulated as a medical device are converging, both being sensitive to the function and risk a device poses
• Some machine learning devices may pose novel problems for medical device regulation, because they may be human uninterpretable or may retrain and so represent a moving target for regulators

The synergies and findings of this and a subsequent workshop informed another PHG Foundation report and a Wellcome Trust funded PHG Foundation project – Black Box Medicine and Transparency – which take forward some of the topics and challenges noted in this report.

By Alison Hall and Johan Ordish