How likely is an intelligence explosion?
Canonical Answer
Conditional on technological progress continuing, it seems extremely likely that there will be an intelligence explosion, as at some point generally capable intelligent systems will tend to become the main drivers of their own development both at a software and hardware level. This would predictably create a feedback cycle of more and more intelligent systems improving themselves more effectively. It seems like if the compute was used effectively, computers have many large advantages over biological cognition, so this scaling up might be very rapid.
Some ways technological progress could stop would be global coordination to stop AI research, disasters which permanently stop us from developing, or hardware reaching physical limits before an intelligence explosion is possible (though this last one seems unlikely, as atomically precise manufacturing promises many orders of magnitude of cost reduction and processing power increase).
(edits welcome) | |
|---|---|
| Asked by: | Luke Muehlhauser |
OriginWhere was this question originally asked |
MIRI's Intelligence Explosion FAQ |
| Date: | 2021-9-17 |
Discussion
