tool ai

From Stampy's Wiki
Tool ai
tool ai
Alignment Forum Tag

Description

A tool AI is a type of Artificial Intelligence that is built to be used as a tool by the creators, rather than being an agent with its own action and goal-seeking behavior.

A tool AI is a type of Artificial Intelligence that is built to be used as a tool by the creators, rather than being an agent with its own action and goal-seeking behavior.

Generally meant to refer to AGI, tool AI is a proposed method for gaining some of the benefits of the intelligence while avoiding the dangers of having it act autonomously. It was coined by Holden Karnofsky, co-founder of GiveWell, in a critique of the Singularity Institute. Karnofsky proposed that, while he agreed that agent-based AGI was dangerous, it was an unnecessary path of development. His example of tool AI behavior was Google Maps, which uses complex algorithms and data to plot a route, but presents these results to the user instead of driving the user itself.

Eliezer Yudkowsky responded to this by enumerating several ways in which tool AI had similar difficulties in technical specification and safety. He also pointed out that it was not a common proposal among leading AGI thinkers.

See Also

External Links

Canonically answered

Isn’t AI just a tool like any other? Won’t it just do what we tell it to?

Show your endorsement of this answer by giving it a stamp of approval!

It likely will – however, intelligence is, by many definitions, the ability to figure out how to accomplish goals. Even in today’s advanced AI systems, the builders assign the goal but don’t tell the AI exactly how to accomplish it, nor necessarily predict in detail how it will be done; indeed those systems often solve problems in creative, unpredictable ways. Thus the thing that makes such systems intelligent is precisely what can make them difficult to predict and control. They may therefore attain the goal we set them via means inconsistent with our preferences.

Non-canonical answers

Why does AI need goals in the first place? Can’t it be intelligent without any agenda?

Show your endorsement of this answer by giving it a stamp of approval!

Goal-directed behavior arises naturally when systems are trained to on an objective. AI not trained or programmed to do well by some objective function would not be good at anything, and would be useless.

See Eliezer's and Gwern's posts about tool AI.