Technology: Concerns about potentially ‘monopolistic’ Google access to NHS patient data

In its annual report for 2018, the independent panel set up by the AI firm acquired by Google four years ago, has principally recommended that the company be transparent about its business model.

It notes fears that the company’s low-cost offerings could be luring in the NHS at the moment, only for the company to suddenly extract enormous profits from the health service in the future once it is dependent on the technology.

Given the current environment, and with no clarity about DeepMind Health’s business model, people are likely to suspect that there must be an undisclosed profit motive or a hidden agenda.

“Given the current environment, and with no clarity about DeepMind Health’s business model, people are likely to suspect that there must be an undisclosed profit motive or a hidden agenda,” the independent reviewers state.

“We do not believe this to be the case, but would urge DeepMind Health to be transparent about their business model, and their ability to stick to that without being overridden by Alphabet. For once an idea of hidden agendas is fixed in people’s mind, it is hard to shift, no matter how much a company is motivated by the public good.”

The panel also addressed the company’s controversial access to NHS patients’ data, which onlookers have complained has occurred without patient’s informed consent.

Last year, a Sky News investigation into the sharing of patient data with DeepMind revealed that the health service’s data guardian had warned that she believed the private company received 1.6 million patients’ data on an “inappropriate legal basis”.

Two months later, the Information Commissioner’s Office (ICO) which was investigating the data transfer, declared that the Royal Free NHS Trust had illegally given the patient data to Google for the purpose of testing a smartphone app.

In a statement released on its site at the time, DeepMind said it welcomed “the ICO’s thoughtful resolution of this case, which we hope will guarantee the ongoing safe and legal handling of patient data for Streams”.

“Although today’s findings are about the Royal Free, we need to reflect on our own actions too,” the company added.

Although the company does not currently share the patient data it receives with the larger corporation at Google and Alphabet – something which its clinical lead Dominic King stressed to Sky News last year – the independent panel said it was concerned that this could change in the future.

Responding to the annual report, DeepMind stated: “We will explore further ways to ensure there is clarity about the binding legal frameworks that govern all our NHS partnerships. Trusts remain in full control of the data at all times.

“We are legally and contractually bound to only using patient data under the instructions of our partners. We will continue to make our legal agreements with Trusts publicly available to allow scrutiny of this important point.”

The panel’s chair, Dr Julian Huppert, an academic at the University of Cambridge and former Liberal Democrat MP, said the company needed to commit to a specific business model to reassure the public whose data it is collecting.

0:48

“It is important for the public to have reassurance about DeepMind Health’s business model,” Dr Huppert wrote.

“They want to know that DeepMind Health’s revenue is – or will be – coming from a source they consider appropriate.”

Currently, DeepMind Health’s work with the NHS includes the Streams app to detect acute kidney injuries, which is not based on AI technologies, and machine learning algorithms which analyse medical scans for signs of illness.

An audit by Linklaters on behalf of the ICO into the use of the Streams app at Royal Free considered that it was lawful, but identified areas for improvement.

DeepMind Health says that it will use these services with the NHS on an outcome basis, only charging when the NHS says that their services were key to providing a positive medical outcome.

Dr Huppert wrote that the public wants to know “that any promises made now by DeepMind Health will be kept in the future and that if this generation enjoys a service for little or no cost, that the next generation will not pay the price in excessive costs or lack of control further down the line”.

0:38

The panel suggested the company adopt as a principle the notion of making a “reasonable profit” and “not use its assets or position to seek to extract excessive profits in its dealings with the public sector”.

It also reiterated its previous recommendation that the company “demonstrate its commitment to transparency” by publishing all of its contracts with the public sector, “with minimal or no redactions”.

DeepMind Health responded to the report by thanking the reviewers and stating it was working on addressing its longer-term business model and road map.

The company said it was already publishing all of its NHS contacts and added: “We will continue to make our legal agreements with Trusts publicly available to allow scrutiny of this important point.”

Regarding its business model, DeepMind respondeD: “Rather than charging for the early stages of our work, our first priority has been to prove that our technologies can help improve patient care and reduce costs.

“We believe that our business model should flow from the positive impact we create, and will continue to explore outcomes-based elements so that costs are at least in part related to the benefits we deliver.”

It added: “By holding us to account, recognising where we’re getting it right and challenging us where we can improve, [the independent reviewers] help us to do a better job for patients, nurses, doctors, carers, families, and all those who rely on healthcare systems around the world.”