Last updated
    Arena
    AI Evaluation Platform

    Arena

    by LMSYS

    "Crowd-sourced head-to-head comparisons of AI chatbots"

    92%

    Rating

    4.6/5

    FreeAvailable
    Community

    Overview

    Arena, previously known as LMArena and Chatbot Arena, is a public web platform where people pit anonymous AI chatbots against each other. You type in a prompt, two unnamed models respond, and you vote on which answer you prefer. After you cast your vote, the identities of the models are revealed. There is also a "Direct" mode where you can pick a specific model to test on your own.

    The platform has become a popular testing ground for AI companies. OpenAI, Google DeepMind, and Anthropic have all submitted their models for evaluation. It is also a place where companies try out new models before their official release. For instance, Chinese startup DeepSeek quietly tested prototype models on Arena months before its R1 model made headlines in Western media. OpenAI's GPT-5 appeared under the codename "summit," and Google DeepMind's Gemini 2.5 Flash Image, an image generation and editing model, was tested as "Nano Banana."

    Arena's crowd-sourced rankings have made it a go-to reference for AI enthusiasts and researchers, but the methodology is not without flaws. Studies have pointed out limitations in how the votes are collected and interpreted, which can skew results. Still, it remains a lively, hands-on way to see how AI models stack up in real-world conversations.

    Key Features

    • Anonymous head-to-head model comparisons
    • Direct mode for testing specific models
    • Crowd-sourced Elo-style rankings
    • Supports text and image generation models
    • Used by major AI companies for pre-release testing

    Specifications

    Platform TypeWeb-based
    Supported ModelsText and image generation models from multiple vendors
    Ranking SystemCrowd-sourced Elo-style
    AccessFree, no account required

    Pros & Cons

    Pros

    • • Free and accessible to anyone
    • • Real-world human preference data
    • • Wide range of models, including unreleased prototypes
    • • Community-driven and transparent

    Cons

    • • Methodology can be skewed by user preferences
    • • Anonymous voting can be gamed
    • • Rankings may not reflect objective quality

    Frequently Asked Questions

    Is LMArena free to use?
    Yes, LMArena is completely free. Users can vote on model comparisons or test specific models in Direct mode without any cost or account requirements.
    How does LMArena's ranking system work?
    LMArena uses anonymous head-to-head matchups where users vote on preferred responses. Votes are aggregated into an Elo-style ranking system, though studies have noted methodological limitations that can skew results.
    What is the difference between LMArena and Chatbot Arena?
    They are the same platform; Chatbot Arena was rebranded to LMArena. The core functionality of anonymous model comparisons and user voting remains unchanged.
    Which models are currently ranked highest on LMArena?
    Rankings fluctuate as new models are added and tested. Major companies including OpenAI, Google DeepMind, and Anthropic regularly submit models, with unreleased prototypes sometimes appearing under codenames.
    Can companies test their AI models on LMArena before release?
    Yes, this has become a common practice. DeepSeek tested prototype models before its R1 release, OpenAI tested GPT-5 under "summit," and Google DeepMind tested Gemini 2.5 Flash Image under "Nano Banana."

    Keywords

    AIchatbotLLMbenchmarkcomparisoncrowdsourcingLMSYSLMArenaChatbot Arena

    Sources

    Product Info

    Brand
    LMSYS
    Manufacturer
    Large Model Systems Organization (LMSYS)
    Category
    AI Evaluation Platform
    Released
    2023-04-01
    Price
    Free
    Availability
    Available
    Developer
    Large Model Systems Organization (LMSYS)
    Initial Release
    April 2023
    Former Names
    Chatbot Arena, LMArena
    Access
    Free

    Join the community

    fans discussing