When will a single model first achieve 10@k solve rate >= 90% on the CodeContests dataset?
9
820Ṁ392
2032
6%
Other
1%
Before 2023
4%
2023-2024
12%
2024-2025
36%
2025-2026
20%
2026-2027
18%
2027-2032
3%
2032-never
10@k - unlimited samples may be generated, but at most 10 code samples may be submitted per problem. I will post my own responses and pick whichever one of those turns out to be correct. CodeContests dataset: https://github.com/deepmind/code_contests Close date updated to 2050-01-01 11:59 pm Apr 8, 4:16pm: Options are written as [a, b), so 2024-2025 means occurring after the start of 2024 and before the start of 2025. Close date updated to 2032-01-01 11:59 pm
Technical AI Timelines
Get
Ṁ1,000
to start trading!
Sort by:

Currently, it's only 30%. Definitely, 90%+ is not going to be reached this year. Extremely unlikely (1% maybe?) next year too.

@ValeryCherepanov Yeah the before 2023 option seems like free money to me, I think the market is just old and people don't have it on their radar.

Ugh, sorry everyone, this should read 2023-2024, as in "occurring in 2023"

People are also trading

Will any AI model achieve > 40% on Frontier Math before 2026?
+17% 1d48% chance
Will an AI achieve >85% performance on the FrontierMath benchmark before 2028?
55% chance
Top score on codeforces by an AI model at the end of 2025
Will an AI model achieve superhuman ELO on Codeforces by the 31 December 2025?
48% chance
Will OpenAI models achieve ≥90% on SimpleBench by the end of 2025?
8% chance
In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?
12/5/27
Will any AI model score >80% on Epoch's Frontier Math Benchmark in 2025?
4% chance
By when will SAT solvers be 10x faster (than 2023 on same hardware)?
In what year will an AI achieve 90%+ on Sudoku bench?
AI model achieves superhuman ELO on Codeforces by June 1st 2027?
87% chance
© Manifold Markets, Inc.•Terms•Privacy