What is the Abstraction and Reasoning Corpus (ARC)?

The Abstraction and Reasoning Corpus (ARC)¹

is a collection of tests designed to measure machine intelligence. It was introduced by François Chollet in order to test "intelligence" as defined in his paper “On the Measure of Intelligence”. ARC is designed to require the ability to generalize and to resist being solvable through rote memorization. While most human children can perform well on ARC tasks, unmodified LLMs2

Some models specifically trained for this task have near-human performance.

tend to perform poorly on it (as of 2024).

Each question in the test is composed of a few examples of sets of colored pixels on a grid, where the AI must guess the pattern and apply it to the test grid.

An example test grid where the solution involves filling the insides of green shapes with yellow blocks.

As of December 2024, the best-performing model for this task was OpenAI

's o3 model, which scored a substantial improvement over previous generations.

Further reading:

• Lab42's description of ARC

• François Chollet and Mike Knoop on the Dwarkesh podcast

---

1. Not to be confused with the Alignment Research Center, also abbreviated as ARC. ↩︎

2. Some models specifically trained for this task have near-human performance. ↩︎