Extrapolating to Unnatural Language Processing with GPT-3’s In-context Learning: The Good, the Bad, and the Mysterious.
In mid-2020, OpenAI published the paper and commercial API for GPT-31, their latest generation of large-scale language models. Much of the discourse on GPT-3 has centered on the language model’s ability to perform complex natural language tasks, which often require extensive knowledge and natural language understanding. Yet, as headlined in the title of the original paper by OpenAI, “Language Models are Few-Shot Learners”, arguably the most intriguing finding is the emergent phenomenon of in-context learning.2 Unless otherwise specified, we use “GPT-3” to refer to the largest available (base) model served through the API as of writing, called Davinci.“Few-shot learning” is a more general term used in the literature across robotics, computer vision, NLP, and other subfields of AI that includes other ways of learning from few examples that might involve additional training. To avoid confusion and be more specific, we’ll use the phrase “in-context learning” introduced in the GPT-3 paper by OpenAI and used in follow-up work.
Read in full here:
This thread was posted by one of our members via one of our news source trackers.