How likely is an intelligence explosion?

From Stampy's Wiki

Canonical Answer

Conditional on technological progress continuing, it seems extremely likely that there will be an intelligence explosion, as at some point generally capable intelligent systems will tend to become the main drivers of their own development both at a software and hardware level. This would predictably create a feedback cycle of more and more intelligent systems improving themselves more effectively. It seems like if the compute was used effectively, computers have many large advantages over biological cognition, so this scaling up might be very rapid.

Some ways technological progress could stop would be global coordination to stop AI research, disasters which permanently stop us from developing, or hardware reaching physical limits before an intelligence explosion is possible (though this last one seems unlikely, as atomically precise manufacturing promises many orders of magnitude of cost reduction and processing power increase).

Canonical Question Info
(edits welcome)
Asked by: Luke Muehlhauser
OriginWhere was this question originally asked
MIRI's Intelligence Explosion FAQ
Date: 2021-9-17


Discussion