nick bostrom

From Stampy's Wiki
Nick bostrom
nick bostrom
Parent Tag: people
Arbital Page
Wikipedia Page

Canonically answered

What is Artificial General Intelligence and what will it look like?

Show your endorsement of this answer by giving it a stamp of approval!
An Artificial General Intelligence, or AGI, is an artificial intelligence which is capable in a broad range of domains. Crucially, an advanced AGI could be capable of AI research, which may allow it to initiate an intelligence explosion, leading to a superintelligence.
AGI is an algorithm with general intelligence, running not on evolution’s biology like all current general intelligences but on a substrate such as silicon engineered by an intelligence (initially computers designed by humans, later on likely dramatically more advanced hardware designed by earlier AGIs).

AI has so far always been designed and built by humans (i.e. a search process running on biological brains), but once our creations gain the ability to do AI research they will likely recursively self-improve by designing new and better versions of themselves initiating an intelligence explosion (i.e. use it’s intelligence to improve its own intelligence, creating a feedback loop), and resulting in a superintelligence. There are already early signs of AIs being trained to optimize other AIs.

Some authors (notably Robin Hanson) have argued that the intelligence explosion hypothesis is likely false, and in favor of a large number of roughly human level emulated minds operating instead, forming an uplifted economy which doubles every few hours. Eric Drexler’s Comprehensive AI Services model of what may happen is another alternate view, where many narrow superintelligent systems exist in parallel rather than there being a general-purpose superintelligent agent.

Going by the model advocated by Nick Bostrom, Eliezer Yudkowsky and many others, a superintelligence will likely gain various cognitive superpowers (table 8 gives a good overview), allowing it to direct the future much more effectively than humanity. Taking control of our resources by manipulation and hacking is a likely early step, followed by developing and deploying advanced technologies like molecular nanotechnology to dominate the physical world and achieve its goals.

Non-canonical answers

Who is Nick Bostrom?

Show your endorsement of this answer by giving it a stamp of approval!

Professor Nick Bostrom is the director of Oxford’s Future of Humanity Institute, tasked with anticipating and preventing threats to human civilization.

He has been studying the risks of artificial intelligence for over twenty years. In his 2014 book Superintelligence, he covers, among other things three major questions:

  • First, why is superintelligence a topic of concern
  • Second, what is a “hard takeoff” and how does it impact our concern about superintelligence?
  • Third, what measures can we take to make superintelligence safe and beneficial for humanity?

Unanswered non-canonical questions