gemini 3 pro gets 19%!!

GPT-5 (high) got one more problem than previous models, for a score of 8.3% (4/48).

My intuition is that you can't get much better results on this benchmark just by scaling current methods and that no one will implement a new method before end of 2025.

I think there's still a fair bit it can be pushed but I agree probably no one will implement a new method before end of 2025. do you think the method that first gets 30% will use a new method?

@Bayesian Yeah. But I didn't do too much research on the questions I just know they are unique from trainable datasets, and require a lot of reasoning steps. I think we need a new method that will help AIs better generalize their learnings and skills from different domains.