intelligence
Main Question: How is "intelligence" defined? (edit question) (edit answer) |
Alignment Forum Tag |
Arbital Page |
Wikipedia Page |
Description
Canonically answered
Why would great intelligence produce great power?
Intelligence is powerful. One might say that “Intelligence is no match for a gun, or for someone with lots of money,” but both guns and money were produced by intelligence. If not for our intelligence, humans would still be foraging the savannah for food.
Intelligence is what caused humans to dominate the planet in the blink of an eye (on evolutionary timescales). Intelligence is what allows us to eradicate diseases, and what gives us the potential to eradicate ourselves with nuclear war. Intelligence gives us superior strategic skills, superior social skills, superior economic productivity, and the power of invention.
A machine with superintelligence would be able to hack into vulnerable networks via the internet, commandeer those resources for additional computing power, take over mobile machines connected to networks connected to the internet, use them to build additional machines, perform scientific experiments to understand the world better than humans can, invent quantum computing and nanotechnology, manipulate the social world better than we can, and do whatever it can to give itself more power to achieve its goals — all at a speed much faster than humans can respond to.
See also
- Legg (2008). Machine Super Intelligence. PhD Thesis. IDSIA.
- Yudkowsky (2007). The Power of Intelligence.
How is "intelligence" defined?
Intelligence measures an agent’s ability to achieve goals in a wide range of environments.
This is a bit vague, but serves as the working definition of ‘intelligence’. For a more in-depth exploration, see Efficient Cross-Domain Optimization.
See also:
- Wikipedia, Intelligence
- Neisser et al., Intelligence: Knowns and Unknowns
- Wasserman & Zentall (eds.), Comparative Cognition: Experimental Explorations of Animal Intelligence
- Legg, Definitions of Intelligence
Non-canonical answers
Nick Bostrom defines superintelligence as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” A chess program can outperform humans in chess, but is useless at any other task. Superintelligence will have been achieved when we create a machine that outperforms the human brain across practically any domain.
We don’t yet have a dangerous superintelligence on our hands. However, that does not mean it’s too early to start preparing. Given the stakes, it is worth investing significant resources even if superintelligence is not an immediate risk.
And despite the fact that in some ways even our best AIs can’t match up to humans, we’ve been seeing domain after domain of human superiority being challenged or overturned over the past few years. GPT-3 showed that it was possible for a very simple architecture applied at scale to become a language model capable of performing a surprisingly general range of text-based tasks at a high level (e.g. writing short articles which are almost indistinguishable from human-written ones). ‘Generally capable agents emerge from open-ended play’ showed that by training artificial agents in diverse procedurally generated games, they develop the ability to learn and adapt. MuZero, and more recently EfficientZero, demonstrated that AIs can effectively and rapidly learn both the rules of the games they’re playing and how to win even faster than humans.
Even though AIs are probably not as smart as rats yet, it might only be a few decades until we create superintelligence. World renowned AI expert Stuart Russell expects that superintelligence will arrive within our children’s lifetimes. And, as Stephen Hawking put it: "If a superior alien civilisation sent us a message saying, "We'll arrive in a few decades," would we just reply, "OK, call us when you get here – we'll leave the lights on"? Probably not – but this is more or less what is happening with AI."
This dynamic is explored poetically in the The Unfinished Fable of the Sparrows.