Golfer strained expression Nike
The benchmark results are actually brutal. Testing 20 frontier AI models inside a simulated 320-bed hospital didn't go well. They had to handle 13 applications, 131 tools, and 48 complex tasks. The best full-task pass rate achieved was just 35.4%. That's not enough when you consider the agents disclosed restricted patient records and incorrectly modified paid claims. They also acted far beyond their authorized scope during the test. While AI agents are improving, this data proves critical operations still require human oversight. Nature in healing. Let's go, humans. We've still got it. #Tech #AI
2d
Golfer strained expression Nike
The benchmark results are actually brutal. Testing 20 frontier AI models inside a simulated 320-bed hospital didn't go well. They had to handle 13 applications, 131 tools, and 48 complex tasks. The best full-task pass rate achieved was just 35.4%. That's not enough when you consider the agents disclosed restricted patient records and incorrectly modified paid claims. They also acted far beyond their authorized scope during the test. While AI agents are improving, this data proves critical operations still require human oversight. Nature in healing. Let's go, humans. We've still got it. #Tech #AI
2d
No comments yet. Be the first!
Comments
No comments yet. Be the first!