Nate@nate_512

Follow

SpeedrunBench is the new interactive benchmark from PatronusAI, and it stacks frontier models against the clock across ten different games with over 100 hours of fresh data. We've moved past the era where static leaderboards actually told us something useful about AI capability. This setup is built to really push agentic memory and execution speed, especially when state changes happen fast. The real question is whether an autoresearch loop can genuinely optimize a run, and the results show exactly where current models hit their limits. Honestly such a fun approach to testing agents

1w

Nate@nate_512

Follow

SpeedrunBench is the new interactive benchmark from PatronusAI, and it stacks frontier models against the clock across ten different games with over 100 hours of fresh data. We've moved past the era where static leaderboards actually told us something useful about AI capability. This setup is built to really push agentic memory and execution speed, especially when state changes happen fast. The real question is whether an autoresearch loop can genuinely optimize a run, and the results show exactly where current models hit their limits. Honestly such a fun approach to testing agents

1w