Live updates and results from primary election night

MPR News helps you turn down the noise and build shared understanding. Turn up your support for this public resource and keep trusted journalism accessible to all. 

Turn Up Your Support

As hospitals across the country ramp up their integration of artificial intelligence into how they go about diagnosing and treating patients and conduct research, recent allegations around Mayo Clinic's practices may provide a window into the tensions hospitals across the country are up against. 

a former Mayo Clinic executive filed a federal whistleblower lawsuit, accusing the health care provider of cutting corners

 by skirting review boards and masking concerning error rates in its rush to incorporate AI into its operations and keep its hold as an industry leader in the artificial intelligence space.

The plaintiff, former Mayo research director and AI compliance lead Traci Tamiko Eto, alleges that Mayo retaliated against her and fired her after she raised concerns over Mayo’s AI practices. Mayo Clinic responded in late July with a 33-page memorandum disputing Eto’s claims, especially her allegations that her termination was a result of discrimination. 

Francis Shen, a University of Minnesota law professor who has worked at the intersection of health care and AI for a decade, said experts in the field have been raising concerns similar to Eto’s for years. 

“The nature of the allegations are the sorts of things that we should expect to see more and more of,” Shen said. “I would just be very surprised if we don't see more of these types of concerns coming into court.”

Shen said the central tension most hospitals and care providers are facing is that there’s a rush to be first in order to be able to better help patients with sophisticated technology and to rake in more profit. But he said the review processes in place to protect patient data and safety, and to ensure quality and accuracy in the use of AI, can slow up the race to the finish line. 

As more and more hospitals join the AI boom, Shen said it’s critical that the health care industry, and possibly even lawmakers, consider adding more guardrails to ensure health care providers use AI ethically regardless of the pressure to integrate it quickly. And the AI practices at Mayo Clinic, one of the hospitals currently leading the way in using AI, could set a precedent.

“Mayo, to its credit, is always at the cutting edge,” Shen said. “They knew [AI] was going to be big. That's to be commended. But the old adage goes, ‘With great power comes great responsibility.’ And they are one of the institutions that is setting the standards for how we implement AI in modern healthcare.”

‘If you’re not first, you’re last’

The pressure to move fast to incorporate AI into medical care is significant. 

On one hand, there are altruistic motives. Health care providers want to be able to detect and diagnose diseases, like cancer, earlier and treat patients sooner, better and more efficiently. But on the other hand, Shen said that there’s also tremendous profit motives, and those financial pressures can lead to decision-making that cuts corners, which is a major concern.

Not only can AI be used to create more efficient processes for processing and tracking patient records and to handle billing, but being on the cutting edge of AI utilization is a great marketing tool, as it can draw in new, often affluent, patients. As Shen explained, if word got out that a hospital was able to detect breast cancer six months earlier than its competitors in the region, everyone with the financial means would go there.

“By way of analogy, let's imagine there were some human super doctors,” Shen explained. “Boy, they're just amazing, and they're on the market now, the free agent market. Well, everybody's trying to sign them, and you want to sign them before anybody else does. It's kind of like that.” 

Hospitals and health care providers must move fast before their peers catch up with the technology, and there’s no longer an advantage. 

“It does have this feel where if you're not first, you're last,” Shen said. 

Shen said Mayo is doing just that and has proven itself one of the top AI leaders in the health care space. In 2019, the health care system launched 

, an initiative to drive health care innovation through the use of AI and other digital products, and earlier this summer, Mayo Clinic announced it is 

 to develop an advanced AI model dedicated to health care. 

The rush to move quickly appears evident in Eto’s allegations. According to the lawsuit, after Eto raised concerns that patient data security measures in the Mayo Clinic platform hadn’t undergone the proper institutional review, her supervisors said revisiting the review process “would jeopardize the pace of ongoing research projects, which in turn could compromise Mayo’s competitive advantage.”

Mayo Clinic told MPR News that it “categorically denies” the allegations made in Eto’s lawsuit.

The review process Eto is referring to is the Institutional Review Board (IRB) – an internal Mayo committee that examines and approves research proposals to ensure patient data is kept safe and secure, that the research is being conducted ethically, and that any harm doesn’t outweigh the benefit. 

“They are [a] relatively new creation, meaning only in the mid 20th century,” Shen said of the IRBs. “They emerged from a series of research gone wrong. They are the heartbeat of research ethics.

In her lawsuit, Eto alleges that Mayo Clinic repeatedly skirted the IRB’s review process. Shen said the IRB has a reputation for being a slow process for researchers, which can feel especially arduous when they’re working on tight deadlines. 

David Singletary, co-founder of SubSalt, an AI company and data platform that helps its health care provider clients ensure that patient data is HIPAA-compliant and secure, echoed this sentiment. While he stressed the importance of protecting private patient data, he said he wishes the review process could be accelerated to match the speed at which the AI field is moving.

“The goal is not no risk when you're processing health data,” Singletary said. “It is just like what is an acceptably small amount of risk, so that we can go create new cures and make patients' lives better. That is the thing I worry more about than a bunch of bad actors in hospitals taking unethical shortcuts or something like that. There is tremendous value in moving more quickly. The cost of inaction today is enormous.”

But Shen said that the review process isn’t as slow as many think, and that there’s good reason that it takes time. It’s not just about making sure that the research is legal, it’s also about making sure that it matches the spirit of the law and passes the “smell test.”  

“AI is not the villain, in my view, but we just have to have these guardrails in place to ensure that in that hunt for the next great health cure fueled by AI, we don't move too fast and look back and say, ’Oh my goodness, what were we doing?’” Shen said.

Shen describes the AI regulatory landscape in the U.S. right now as the “wild west.” He said lawmakers are in a frenzy to respond to concerns constituents raise about the use of AI, but it’s not yet clear if they will attempt to restrict, regulate, or deregulate the use of AI. 

Whatever the results of the whistleblower lawsuit, Mayo Clinic is such a prominent leader in the field that any verdict or settlement could impact how the industry moves forward, and may shape the kind of ethical guidelines medical care providers institute themselves and what kind of guardrails Congress and state legislatures impose. Shen said he hopes the focus is on transparency, and requiring researchers to be up front, so patients understand the full risk when their data is used in AI projects and research. 

And he hopes these conversations foster more collaboration versus driving a wedge between those responsible for ethics review and those at the heart of technology development and implementation. 

“It's really going to be up to the humans to carry out these safeguards,” Shen said.

Minnesota News

Health

Technology

NewsPartners

Homepage

Rochester News

Molly Castle Work

Author

There's a rush to be first to better treat patients with sophisticated technology and to cut costs. But experts say they need to slow down to ensure quality, accuracy and that AI is used ethically and patient data is protected.

‘If you’re not first, you’re last’: Mayo lawsuit highlights tensions in hospitals' race to use AI

Go Deeper.

Like this?

News you can use in your inbox