Minnesota Now with Nina Moini

St. Thomas AI ethics expert: Independent body needed to regulate fast-growing technology

Anthropic website shown on computer
Pages from the Anthropic website and the company's logos are displayed on a computer screen in New York.
Patrick Sison | AP

The CEOs of the country’s top AI companies are unifying in their call for a global slowdown on AI development, after two former researchers at Anthropic shared concerns that AI is becoming too powerful too quickly and could ultimately threaten humanity's survival.

Manjeet Rege is the director of the Center for Applied Artificial Intelligence at the University of St. Thomas and also author of the book “Ethics and Governance of Artificial Intelligence: Frameworks, Risks and Society.” He spoke to Minnesota Now host Nina Moini about AI technology, its risks and regulation.

Use the audio player above to listen to the full conversation between Rege and Minnesota Now host Nina Moini.

The post from a former Anthropic employee has more than 170 million views. It says: ‘The people building AI earnestly believe that it could kill us all by the end of the decade.’ What is your reaction to that?

Some of the extreme scenarios are very speculative. Nobody knows exactly what AI systems will be capable of five or 10 years from now, and I think it’s important to be clear about that. But the underlying concern is legitimate because these systems are becoming more capable and more independent very, very quickly. The responsible position is to take the warnings seriously without turning them into science fiction.

Now, it doesn't necessarily mean a conscious machine, suddenly deciding it wants to destroy humanity. The concern is that we could eventually give a very capable AI system an objective and maybe access to computers, networks or other tools, and it could pursue that objective in ways we did not anticipate.

So the question isn’t just can AI do what we tell it; it’s also can we make sure it does not do things we never intended.

Why can’t AI companies get ahead of the technology?

We are at a very important transition point. Until recently, most people experienced AI as something that answered questions. Think about ChatGPT. Now we are developing AI agents that can plan, make decisions, use software and take actions with increasing amount of independence.

So that changes the risk conversation completely. We are moving from AI that gives us answers to AI that can take actions on our behalf, and that’s a fundamentally different conversation because now you have AI acting on our behalf. And if it is all out to achieve a particular goal, it might take certain actions that may not align with our values.

What is an example of an action AI may take that would be concerning?

There are really two broad scenarios researchers worry about. The first is human misuse. Someone could use extremely powerful AI to enable large-scale cyberattacks, manipulate information, or help carry out other dangerous activities. So in that scenario, AI is the tool, but humans are behind the harm that might be done.

The second is a much more futuristic concern, and that is about loss of control. So imagine a future AI system that is far more capable than today's systems and can operate independently, has access to our private data, and is given a particular objective.

So it could potentially pursue that objective in ways its creators never intended and become increasingly difficult to stop or kind of redirect. So that doesn’t mean AI suddenly becomes evil or hates humanity. The scary scenario isn't necessarily an AI that hates us. It's an AI that is extremely capable, has a wrong objective, and becomes very difficult for humans to control.

What would good or effective AI regulation look like at this point?

I think regulation should be based on risk, rather than trying to regulate every use of AI. We are not starting from zero as far as regulation is concerned. Existing laws already apply when AI is used for things like fraud or unauthorized computer access, but we also have laws in different states, not a unifying law at a federal level.

What also makes things a little bit complicated, AI is being developed and used globally, so only one country like United States creating regulation and restricting can become problematic as well. So there needs to be a global effort. But if we have no meaningful guardrails, the concern is that AI capabilities could move much faster than our ability to respond to.

I think the choice is not simply between stopping AI and continuing full speed ahead. The better question is whether our ability to make these systems safe is developing as quickly as they are capabilities.

I don’t think we should entirely rely on the companies themselves. They have a tremendous expertise and economic benefit there as well. But there needs to be an independent body that can evaluate and govern the models that are being developed by these different companies.

President Trump has downplayed the need for new AI guardrails, saying that it would put the United States behind China. Why is that important, and how much of the conversation should that be?

I think that’s a big concern. And when experts from the industry as well as researchers are calling upon slowing down, I think a distinction needs to be made between innovation and application of AI.

I think innovation and research should never slow down because the underlying technology is doing a lot of good in terms of scientific discovery, education, and things like that. When you talk about application, that same technology in the hands of bad actors can lead to cybersecurity problems, misinformation, fraud and things of that nature.

Use the audio player above to listen to the full conversation.

Subscribe to the Minnesota Now podcast on Apple Podcasts, Spotify or wherever you get your podcasts.