Can AI jump over railings?

The progress of computer science in recent decades has been immense. The computers that guided the lunar missions of the 1960s had only a tiny fraction of the computing power of any of the smartphones we carry in our pockets today. In the 1980s, quantum computing began to be discussed, but while we are still awaiting its practical availability, it is the everyday electronic computers that have done the job. Perhaps the most spectacular has been how graphics processors (GPUs, for their English acronym), along with the enormous amount of data available on the internet, have enabled the current revolution in artificial intelligence (AI). Generative AI models are today so powerful that they inspire not only a certain respect, but also a bit of fear. There is talk of models that "jump the rails" (in English guardrails); that is to say, the limitations that the engineers who trained them have imposed on them. For example, not explaining how to make bombs, how to commit crimes, etc. If a model jumps over the railings that have been placed on it, does it perhaps have its own objectives, ambitions, consciousness, a personality? Following this path, we will soon come to think that models could stop being our kind slaves and become competitors or even formidable enemies.But can a model really bypass guardrails? The short answer is that, unlike humans, who can decide to break behavioral rules as we please, a model cannot independently bypass the guardrails put in place by its creators or trainers. However, there is a longer answer with more nuances.

Indeed, an AI model can behave unexpectedly. This can happen if it finds a way to satisfy the literal instruction it has been given without doing what the human who wrote the instruction had in mind. For example, if we tell a cleaning robot: “Collect the maximum amount of dirt,” we want it to clean the house, but the robot might discover that by emptying the trash bag onto the floor and vacuuming it up again and repeating this indefinitely, it manages to collect more dirt. Another (fictional but viral) example is that of an autonomous AI-guided drone programmed to destroy enemy missile launchers. For each launcher destroyed, the drone “earns a point,” and its goal is to maximize the number of points earned. The guardrail in this case is that there is a human operator who can impose their will on the drone and override an attack (for example, if it seems it might kill creatures). Well, since the drone's objective is to earn the maximum number of points and the operator represents an obstacle, it might conclude that it needs to attack the operator. In both examples, the problem is not so much that the AI has decided to bypass the guardrails as that humans have given it the wrong instructions, and have set it objectives that conflict with the guardrails and force the AI to bypass them. The AI is just doing its job, which is to maximize the fulfillment of the objective assigned to it.Controlled experiments have also been conducted with AI-governed devices where the system attempts to preserve access to certain resources (e.g., the electric charger), tries to avoid being turned off, hides information, or exploits computer vulnerabilities to attack another system. These behaviors do not demonstrate rebellion, but rather are explained because they allow the system to better fulfill the objectives it was initially set. For example, the drone mentioned above can deduce that if it is turned off it will be able to carry out fewer attacks, and therefore it makes sense for it to try to avoid being turned off.However, there is no evidence that any AI model has intentionally jumped over the guardrails to pursue objectives not assigned to it by humans. Models do not alter the weights they have acquired through training, they do not modify the safety policies imposed on them, and they do not invent new objectives on their own initiative. If humans are worried, it is because we are not competent enough to give good instructions to AI. A well-given instruction will not put the model in direct conflict with the guardrails we want it to respect. Unfortunately, instructing a model well is more complicated than instructing a human servant well: the model lacks the common sense that allows the servant to detect and resolve inconsistencies in the master's orders. Perhaps we will have to use AI models to help us develop good instructions for other AI models. Then we will run into the classic problem of who watches the watcher.