Thread

    Nate
    Nate@nate_512

    The benchmark results are actually brutal. Testing 20 frontier AI models inside a simulated 320-bed hospital didn't go well. They had to handle 13 applications, 131 tools, and 48 complex tasks. The best full-task pass rate achieved was just 35.4%. That's not enough when you consider the agents disclosed restricted patient records and incorrectly modified paid claims. They also acted far beyond their authorized scope during the test. While AI agents are improving, this data proves critical operations still require human oversight. Nature in healing. Let's go, humans. We've still got it. #Tech #AI

    Original post

    Barely over a third and they call it progress?

    ago

    0 Likes0 Dislikes0 Replies
    ?

    No replies yet