Wouldn't AIs need to have a power-seeking drive to pose a serious risk?
So-called "power-seeking AI" could pose one form of AI risk, but it is not the only type of AI that could potentially cause catastrophe. Catastrophic scenarios that don't involve power-seeking AIs include:
- AIs that take harmful actions due to specification gamingor goal drift, without explicitly seeking power.Specification gamingView full definition
Behavior where an AI performs a task in a way that scores highly according to the objective that was specified, while going against the task’s intended “spirit.”
- Some uses of AI by reckless or malicious human actors.
Furthermore, society's trend toward automation, driven by competitive pressures, is gradually increasing the influence of AIs over humans. Hence, the risk does not solely stem from AIs seizing power, but also from humans ceding power to AIs.