Nexus by Yuval Noah Harari

The book Nexus by Yuval Noah Harari is a tough book to read. 407 pages describing "A Brief History of Information Networks from the Stone Age to AI". I absolutely love this type of book and especially this specific book about information networks, but I have to admit that it´s a bit challenging to digest all its content. It is even harder to summarize it...

But, I will try to summarize what Yuval is saying about the Alignment Problem... bear with me.

This is the x4mat!ve summary:

the alignment problem:
When computers are given a specific goal they use all their power and ingenuity to achieve this goal. Since they operate very differently from humans, they are likely to use methods their human overlords didn´t anticipate. This can result in dangerous unforeseen consequences that are not aligned with the original human goals. The more powerful and independent computers become, the bigger the danger.

The alignment problem is not new - it was discussed already in Clausewitz theory "On war" 1834. According to Clausewitz, military actions are utterly irrational unless they are aligned with some overarching political goal. His model constructs a clear hierarchy between long-term policy, medium-term strategy and short-term tactics... and all these needs to be aligned.

One reason why the alignment problem is particularly dangerous in the context of the computer network is that this network is likely to become far more powerful than any previous human bureaucrazy. A misalignment in the goals of super-intelligent computers might result in a catastrophe of unprecedented magnitude (illustrated by Nick Bostrom in "Superintelligence" 2014 - The paper clip factory). Bostrom´s point was that the problem with computers isn´t that they are particularly evil but they are particularly powerful. And the more powerful the computer, the more careful we need to be about defining its goal in a way that precisely aligns with our ultimate goals.

The paper clip thought experiment may sound utterly disconnected from reality. But if Silicon Valley managers had paid attention when Bostrom published in 2014, perhaps they would have been more careful before instructing their algorithms to "maximize user engagement". When told to maximize user engagement, the Facebook and YouTube algorithms sought to convert the entire social universe into user engagement... even it meant doing harm to the social fabric of many countries.

A second reason why the alignment problem is more urgent in the case of computers - because they are inorganic entities, they are likely to adopt strategies that would never occur to any human and that we are therefore ill-equipped to foresee and forestall. AI will search for loopholes in any game´s rule - meaning that the AI will do exactly what the game was rewarding it to do - even though it was not what the humans was hoping for.

A third reason to worry about the alignment problem of computers is that because they are so different from us, when we make the mistake of giving them a misaligned goal, they are less likely to notice it or request clarification.

How to solve the alignment problem?
In the case of human networks, we rely on self-correcting mechanisms to periodically review and revise our goals, so setting the wrong goals is not the end of the world. But since the computer network might escape our control, if we set it the wrong goal, we might discover our mistake when we are no longer able to correct it. Tech executives and engineers who rush to develop AI are making a huge mistake if they think there is a rational way to tell AI what its ultimate goal should be. They should learn from the bitter experience of generations of philosophers who tried to define ultimate goals and failed.

Interested in AI?

A good starting point is to have a look into the documentary of AlphaGo (follow link). Especially you need to digest the "Move 37" who has forced humanity to reflect about the possibilities of creativity in AI.

Please also have a look into this video, summarizing some concepts from Nexus - 10 ways AI Will Change Your Life: