Will an LLM report >50% score on ARC in 2025? | Manifold

Will an LLM report >50% score on ARC in 2025?

21

1kṀ11k

Dec 31

98.9%

chance

1H

6H

1D

1W

1M

ALL

ARC is a general-purpose AI eval designed to test intelligence as opposed to memorization. https://arcprize.org/arc

This market resolves to Yes if there are public demonstrations of general-purpose base models from OpenAI, Anthropic or Deepmind that solve at least 50% of the ARC questions before 2026.

(note that this is separate from winning the arc price, which requires only using open-source models)

Technical AI Timelines

Get

1,000

to start trading!

People are also trading

What will be true of Anthropic's best LLM by EOY 2025?

Will there be major breakthrough in LLM Continual Learning before 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

LLM reaches >90% Brier score on Prophet Arena by 2026?

Will China have the best LLM by the end of 2025?

Will LLMs mostly overcome the Reversal Curse by the end of 2025?

Will RL work for LLMs "spill over" to the rest of RL by 2026?

Will an LLM get > 50% on hard problems on LiveCodeBench Pro?

Will Apple release its own LLM on par with state of the art LLMs before 2026?

Will the highest-scoring LLM on Dec 31, 2026 show <10% improvement over 2025's best average benchmark performance?

Sort by:

filled a Ṁ1,000 YES at 99.0% order

Resolves yes

bought Ṁ6,189 YES

@Lonis I think this resolves YES. Multiple versions of GPT-5 are above 50%.

Current leaderboard:

I guess you're not counting reasoning models like o1 or o3? Are you also ignoring the compute limit?

bought Ṁ1,000 YES

I assume this resolves YES if it happens in 2024?

Yes!

People are also trading

What will be true of Anthropic's best LLM by EOY 2025?

Will there be major breakthrough in LLM Continual Learning before 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

LLM reaches >90% Brier score on Prophet Arena by 2026?

Will China have the best LLM by the end of 2025?

Will LLMs mostly overcome the Reversal Curse by the end of 2025?

Will RL work for LLMs "spill over" to the rest of RL by 2026?

Will an LLM get > 50% on hard problems on LiveCodeBench Pro?

Will Apple release its own LLM on par with state of the art LLMs before 2026?

Will the highest-scoring LLM on Dec 31, 2026 show <10% improvement over 2025's best average benchmark performance?

Related questions

What will be true of Anthropic's best LLM by EOY 2025?

Will there be major breakthrough in LLM Continual Learning before 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

LLM reaches >90% Brier score on Prophet Arena by 2026?

Will China have the best LLM by the end of 2025?

Will LLMs mostly overcome the Reversal Curse by the end of 2025?

Will RL work for LLMs "spill over" to the rest of RL by 2026?

Will an LLM get > 50% on hard problems on LiveCodeBench Pro?

Will Apple release its own LLM on par with state of the art LLMs before 2026?

Will the highest-scoring LLM on Dec 31, 2026 show <10% improvement over 2025's best average benchmark performance?

© Manifold Markets, Inc.•Terms•Privacy