What is GPT-3?
GPT-3
A family of pretrained large language models by OpenAI, they power ChatGPT.
An AI model that takes in some text and predicts how the text is most likely to continue.
GPT-3 was the basis on which ChatGPT A chatbot interface for the GPT series of large language models by OpenAI.
Gwern produced several resources exploring GPT-3's abilities, limitations, and implications, including:
-
The Scaling Hypothesis: How simply increasing the amount of compute
with current algorithms might create very powerful systems.ComputeView full definitionShorthand for “computing power”. It may refer to, for instance, physical infrastructure such as CPUs or GPUs that perform processing, or the amount of processing power needed to train a model.
The appearance of GPT-3 marked a noticeable shift where powerful AI systems began to be discussed in wider cultural circles beyond computer science.1
Vox published this article in August 2020 explaining its significance to a popular audience. ↩︎