Superintelligence: paths, dangers, strategies

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation? To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological cognitive enhancement, and collective intelligence. --

With new afterword (pages 321-324).

1. Past developments and present capabilities -- Growth modes and big history -- Great expectations -- Seasons of hope and despair -- State of the art -- Opinions about the future of machine intelligence -- 2. Paths to superintelligence -- Artificial intelligence -- Whole brain emulation -- Biological cognition -- Brain : computer interfaces -- Networks and organizations -- 3. Forms of superintelligence -- Speed superintelligence -- Collective superintelligence -- Quality superintelligence -- Direct and indirect reach -- Sources of advantage for digital intelligence -- 4. The kinetics of an intelligence explosion -- Timing and speed of the takeoff -- Recalcitrance -- Non-machine intelligence paths -- Emulation and AI paths -- Optimization power and explosivity -- 5. Decisive strategic advantage -- Will the frontrunner get a decisive strategic advantage? -- How large will the successful project be? -- Monitoring -- International collaboration -- From decisive strategic advantage to singleton -- 6. Cognitive superpowers -- Functionalities and superpowers -- An AI takeover scenario -- Power over nature and agents -- 7. The superintelligent will -- The relation between intelligence an…