Between utopian hope and existential fear: how real is the threat from artificial intelligence?

Since language models grew within a few years from a laboratory curiosity into an everyday tool, the question of whether artificial intelligence could endanger the very survival of humanity has long gone beyond the realm of science fiction. Some leading researchers, including the Nobel Prize winner Geoffrey Hinton, warn that the risks could be underestimated. Others, however, believe that precisely such apocalyptic scenarios distract attention from the damage that this technology is causing already today. For a sober assessment of this debate, it is worth looking separately at the three scenarios that are most often mentioned.

The most tangible is the first scenario, deliberate misuse: powerful systems could enable individuals or small groups to cause damage on a scale that was previously reserved for states. Particularly disturbing is the idea that systems could help in designing biological weapons or in coordinated cyberattacks on critical infrastructure such as power grids. That is why the developers of the most advanced models are already building in safeguards, although critics fear that these could be circumvented with little effort.

The second scenario, known as the alignment problem, concerns the possibility that systems become so capable that their goals no longer coincide with human interests. Such a system would not have to be malicious in the human sense; it would be enough for it to pursue a wrongly formulated goal with extreme consistency. Researchers like Hinton, who received the Nobel Prize in Physics in 2024, consider this risk quite real, while sceptics reply that today's models are far from any will of their own.

The third scenario is less spectacular, but according to some analyses no less worrying: it is about the gradual, almost imperceptible handing over of key decisions to automated systems. If companies, institutions and armies hand over more and more decisions to algorithms because they work faster and more cheaply, humans could gradually lose the ability to understand and correct these processes. Thus power would not be taken by force, but quietly surrendered, in thousands of seemingly rational steps. Similar processes, say the supporters of this thesis, can already be observed in places where algorithms decide on loans, hiring or the allocation of social benefits.

The critics of such scenarios, among whom there are many researchers from the field of technology ethics, do not necessarily deny every long-term risk, but warn of misplaced priorities. While the public debates a hypothetical superintelligence, real harm is occurring already today: disinformation that spreads faster than ever, discriminatory decisions by opaque systems and pressure on professions that were long considered safe. In addition, training and running large models require enormous amounts of electricity and water for cooling data centres, which sharpens the questions about their ecological footprint. From this perspective, the focus on distant catastrophes suits precisely the companies, because it diverts attention from the regulation of their current practices.

However, the two approaches are not necessarily incompatible. Many researchers argue that the same tools that are needed to solve current problems, such as transparency, independent audits and clear legal liability, would reduce the likelihood of later catastrophes as well. With its AI Act the European Union already classifies systems according to their level of risk, while in other countries regulation remains more fragmented for now. The effectiveness of such rules also depends on whether states will manage to agree on common standards, which given the current geopolitical rivalry is by no means guaranteed. Only one thing is certain: the question of how to govern a technology that is developing faster than the institutions that are meant to oversee it will shape the politics of the coming decades.