5astelija's question on Mesa-Optimizers 2

5astelija's question on Mesa-Optimizers 2

Edit question

I know current AI's require some goals and training, but is that really necessary? Is is impossible to build an AI with no goal?

Mark as:

Canonical

Out of scope

Duplicate

For Rob

Not question

Review as:

Meh

Unreviewed

Approved

Good

Excellent

Tags: None (add tags)

Non-Canonical Answers

SlimeBunnyBat's Answer to 5astelija's question on Mesa-Optimizers 2

Mark as canonical

Edit answer

"Having goals" is really a conceptualization of AI's behavior, rather than its training. In the case of the mesa-optimizer, its goals are not provided at all; they emerge from the training process. We do have machine learning processes, such as GPT, that aren't designed to exhibit goal-seeking behavior. The problem is that there applications where we want goal-seeking behavior (GPT is never going to cure cancer, for instance) but only if we can control it.

Stamps: None 

Tags: None (add tags)

Question Info
Asked by:	5astelija ( create )
OriginWhere was this question originally asked	YouTube (comment link)
On video:	Deceptive Misaligned Mesa-Optimisers? It's More Likely Than You Think...
Date:	2021/06/27
Asked on Discord?	Yes

Write additional answer

Discussion