5astelija's question on Mesa-Optimizers 2

From Stampy's Wiki
5astelija's question on Mesa-Optimizers 2 id:UgzesthihDmMHvbkuw14AaABAg

I know current AI's require some goals and training, but is that really necessary? Is is impossible to build an AI with no goal?

Tags: None (add tags)

Non-Canonical Answers

"Having goals" is really a conceptualization of AI's behavior, rather than its training. In the case of the mesa-optimizer, its goals are not provided at all; they emerge from the training process. We do have machine learning processes, such as GPT, that aren't designed to exhibit goal-seeking behavior. The problem is that there applications where we want goal-seeking behavior (GPT is never going to cure cancer, for instance) but only if we can control it.

Stamps: None

Tags: None (add tags)

Question Info
Asked by: 5astelija
OriginWhere was this question originally asked
YouTube (comment link)
On video: Deceptive Misaligned Mesa-Optimisers? It's More Likely Than You Think...
Date: 2021/06/27
Asked on Discord? Yes