Category: Regulation

It’s vital to “keep humans in the loop” to avoid humanizing machine-learning models in research

Today I was part of a panel organized by the Carnegie Council and the UNESCO Information for All Programme Working Group on AI for Information Accessibility: From the Grassroots to Policy Action. We discussed three issues starting with the issue of environmental sustainability and artificial intelligence, then moving to the issue of principles for AI, and finally policies and regulation. I am in awe of the other speakers who were excellent and introduced new ways of thinking about the issues.

Dariia Opryshko, for example, talked about the dangers of how Too Much Trust in AI Poses Unexpected Threats to the Scientific Process. We run the risk of limiting what we think is knowable to what can be researchers by AI. We also run the risk that we trust only research conducted by AI. Alternatively the misuse of AI could lead to science ceasing to be trusted. The Scientific American article linked to above is based on the research published in Nature on Artificial intelligence and illusions of understanding in scientific research.

I talked about the implications of the sorts of regulations we seen in AIDA (AI and Data Act) in C-27. AIDA takes a risk-management approach to regulating AI where they define a class of dangerous AIs called “high-risk” that will be treated differently. This allows the regulation to be “agile” in the sense that it can be adapted to emerging types of AIs. Right now we might be worried about LLMs and misinformation at scale, but five years from now it may be AIs that manage nuclear reactors. The issue with agility is that it will depend on there being government officers who stay on top of the technology or the government will end up relying on the very companies they are supposed to regulate to advise them. We thus need continuous training and experimentation in government for it to be able to regulate in an agile way.