incentives

From Stampy's Wiki
Incentives /
Revision as of 14:04, 2 August 2021 by Plex (talk | contribs) (lesswrong to alignmentforum)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
incentives
Alignment Forum Tag
Wikipedia Page

Description

An Incentive is a motivating factor, such as monetary reward, the risk of legal sanctions, or social feedback. Many systems are best understood by looking at the incentives of the people with power over them.

An Incentive is a motivating factor, such as monetary reward, the risk of legal sanctions, or social feedback. Many systems are best understood by looking at the incentives of the people with power over them.

Inadequate Equilibria covers many problems that arise when there are poor incentives.

Related Pages: Game Theory, Mechanism Design, Moloch, Moral Mazes

Canonically answered

We could, but we won’t. Each advance in capabilities which brings us closer to an intelligence explosion also brings vast profits for whoever develops them (e.g. smarter digital personal assistants like Siri, more ability to automate cognitive tasks, better recommendation algorithms for Facebook, etc.). The incentives are all wrong. Any actor (nation or corporation) who stops will just get overtaken by more reckless ones, and everyone knows this.

Making a narrow AI for every task would be extremely costly and time-consuming. By making a more general intelligence, you can apply one system to a broader range of tasks, which is economically and strategically attractive.

Of course, for generality to be a good option there are some necessary conditions. You need an architecture which is straightforward enough to scale up, such as the transformer which is used for GPT and follows scaling laws. It's also important that by generalizing you do not lose too much capacity at narrow tasks or require too much extra compute for it to be worthwhile.

Whether or not those conditions actually hold it seems like many important actors (such as DeepMind and OpenAI) believe that they do, and are therefore focusing on trying to build an AGI in order to influence the future, so we should take actions to make it more likely that AGI will be developed safety.

Additionally, it is possible that even if we tried to build only narrow AIs, given enough time and compute we might accidentally create a more general AI than we intend by training a system on a task which requires a broad world model.

See also: