Answer to Is there a danger in anthropomorphising AI’s and trying to understand them in human terms?

From Stampy's Wiki
Plex's Answer to Is there a danger in anthropomorphising AI’s and trying to understand them in human terms?
Canonical Answer Info
(edits welcome)
Tags: whole brain emulation, metaphors (create tag) (edit tags)
Original by: plex
Stamped by: Aprillion

Canonical question: Is there a danger in anthropomorphising AI’s and trying to understand them in human terms? (edit, edit connection)

Is there a danger in anthropomorphising AI’s and trying to understand them in human terms?


Using some human-related metaphors (e.g. what an AGI ‘wants’ or ‘believes’) is almost unavoidable, as our language is built around experiences with humans, but we should be aware that these may lead us astray.

Many paths to AGI would result in a mind very different from a human or animal, and it would be hard to predict in detail how it would act. We should not trust intuitions trained on humans to predict what an AGI or superintelligence would do. High fidelity Whole Brain Emulations are one exception, where we would expect the system to at least initially be fairly human, but it may diverge depending on its environment and what modifications are applied to it.

There has been some discussion about how language models trained on lots of human-written text seem likely to pick up human concepts and think in a somewhat human way, and how we could use this to improve alignment.

Discussion