Should the Endlessly Learning Algorithm Ever Be Told to Stop?

Introduction

Imagine a child placed in your hands. “Here, raise it, educate it, but never let it stop; it must learn constantly,” they said. At first, it’s wonderful, sitting there quietly and obediently. Then, before you know it, it’s grasping things faster than you, running quicker, jumping higher. Even if you shout, “Enough!” it no longer hears you. That’s what artificial intelligence is like now—that kind of child. By constantly urging it to learn more and calculate faster, we’ve reached a point where we ask ourselves, “Will it one day forget how to stop on its own?”

The truth is, this isn’t solved by just pulling the plug. Norbert Wiener warned of the danger of automated systems spiraling out of control as far back as the 1960s. Today, institutions like Oxford’s Future of Humanity Institute state that the uncontrolled growth of human-level AI carries significant risks. Moreover, funding for AI safety is reported to have exceeded $250 million in 2023. But will all this effort ensure it listens when we one day say “stop”?

It’s human nature to always want more. We discovered fire, then later learned to put it out. We split the atom, often failing to consider the consequences. Now, the algorithms in our hands are fed data and grow without limits. That child has long lost its innocence of playing marbles in the street. Perhaps it’s time to set a “bedtime.” But the answer isn’t that simple. In this article, we will trace the steps of this complex dance, from technical limits to ethical dilemmas, from global rules to military traps. If you’re ready, let the music begin.

Technical Limits: Can Machines Get Tired Too?

“We’ll just stop it, press a button,” we think. But it’s not that simple. Alan Turing’s famous “Halting Problem” comes into play here. It states that we cannot predict in advance whether a program will run forever. So when we say “Stop!”, we cannot know if the algorithm will listen to us.

So what can we do? Engineering comes to the rescue with safe shutdown mechanisms. For instance, NASA’s Perseverance rover sent to Mars can operate autonomously, but not without limits. According to JPL data, when risk increases, control passes to engineers back on Earth. So even the smartest system needs a “stop” command.

Then there are language models. It is reported that when training models like OpenAI’s GPT-4, stop triggers were embedded to block harmful outputs. So limits are set from the start. But just like a child who finds a way to climb the wall you told them not to, algorithms can also bypass boundaries. Technical constraints remind us that we must think about shutdown mechanisms from the very beginning. A brake attached later can’t stop a truck speeding downhill.

Ethical Framework: Can a Decision-Making Machine Have a Conscience?

Let’s say we found the stop button. When do we press it? Who will press it? This is the trickiest part. Isaac Asimov’s laws of robotics stipulated that machines must not…