--- title: "LLM Benchmarks · Promptmetheus" description: "A list of benchmarks and leaderboards for Large Language Models" canonical_url: "https://promptmetheus.com/resources/llm-benchmarks" meta: keywords: "Prompt IDE, Prompt Engineering, Prompt Testing, Prompt Optimization, AI Development" "og:description": "A list of benchmarks and leaderboards for Large Language Models" "og:image": "https://promptmetheus.com/og-image.jpg" "og:title": "LLM Benchmarks · Promptmetheus" --- > Index of every page on this site: [llms.txt](https://promptmetheus.com/llms.txt) # LLM Benchmarks A curated list of popular leaderboards and test datasets for Large Language Model (LLM) evaluation The quality of AI products and services depends intricately on the capabilities of the LLMs that power them, and as these models continue to improve, we have to regularly revise our choices. To keep you up to date on how new models are performing and which ones lead each category, we've put together a neat list of benchmarks and datasets. ## Leaderboards LM Arena Subjects: Chat models, Reasoning models Topics: reasoning language understanding user preference LM Arena is an open platform created by researchers from UC Berkeley that lets users pose questions and compare responses from two different Large Language Models without knowing which model produced which answer. The aggregate results of these head-to-head "battles" generate a ranking and leaderboard, similar to the ELO rating system used in chess. [Leaderboard Open LM Arena Leaderboard](https://arena.ai/leaderboard "Open LM Arena Leaderboard") [Paper Open LM Arena Publication](https://arxiv.org/abs/2403.04132 "Open LM Arena Publication") Scale Showdown Subjects: Chat models, Reasoning models Topics: user preference language demographics real-world use Scale Showdown ranks models using blind, optional comparisons gathered during real conversations. Its results can be explored by dimensions such as language, age, continent, education, profession, and prompt type, providing a broader view of user preference across populations. [Leaderboard Open Scale Showdown Leaderboard](https://labs.scale.com/showdown "Open Scale Showdown Leaderboard") Artificial Analysis Subjects: Chat models, Reasoning models, Code models Topics: intelligence coding speed latency price Artificial Analysis independently evaluates models and combines capability results with practical measurements such as output speed, latency, price, and token usage. Its standardized runs make it useful for comparing both model quality and the cost of deploying it. [Leaderboard Open Artificial Analysis Leaderboard](https://artificialanalysis.ai/leaderboards/models "Open Artificial Analysis Leaderboard") LiveBench Subjects: Chat models, Reasoning models, Code models Topics: math coding reasoning language instruction following data analysis LiveBench is an evolving benchmark for Large Language Models that aims to avoid “training set leakage” (contamination) by continuously introducing fresh, recently sourced tasks. Each question in LiveBench is paired with a verifiable ground-truth answer, so that model outputs can be scored automatically — no external judge (human or AI) is needed. [Leaderboard Open LiveBench Leaderboard](https://livebench.ai "Open LiveBench Leaderboard") [Code Open LiveBench Code](https://github.com/livebench/livebench "Open LiveBench Code") [Data Open LiveBench Data](https://huggingface.co/collections/livebench/livebench-67eaef9bb68b45b17a197a98 "Open LiveBench Data") [Paper Open LiveBench Publication](https://arxiv.org/abs/2406.19314 "Open LiveBench Publication") HELM Subjects: Chat models, Reasoning models, Multimodal models Topics: capabilities safety robustness fairness efficiency Stanford's Holistic Evaluation of Language Models provides transparent, reproducible evaluations across capabilities, safety, multimodality, and specialist domains. The project entered maintenance mode in June 2026, so its established methodology and archived results are more useful than it is for comparing the newest models. [Leaderboard Open HELM Leaderboard](https://crfm.stanford.edu/helm "Open HELM Leaderboard") [Code Open HELM Code](https://github.com/stanford-crfm/helm "Open HELM Code") [Paper Open HELM Publication](https://arxiv.org/abs/2211.09110 "Open HELM Publication") Humanity's Last Exam Subjects: Reasoning models, Multimodal models Topics: expert knowledge reasoning calibration multimodality Humanity's Last Exam measures expert-level knowledge and reasoning using difficult questions from mathematics, the sciences, humanities, and other disciplines. Its leaderboard reports both accuracy and calibration, including a multimodal subset with diagrams and figures. [Leaderboard Open Humanity's Last Exam Leaderboard](https://labs.scale.com/leaderboard/humanitys_last_exam "Open Humanity's Last Exam Leaderboard") [Code Open Humanity's Last Exam Code](https://github.com/centerforaisafety/hle "Open Humanity's Last Exam Code") [Data Open Humanity's Last Exam Data](https://huggingface.co/datasets/cais/hle "Open Humanity's Last Exam Data") [Paper Open Humanity's Last Exam Publication](https://arxiv.org/abs/2501.14249 "Open Humanity's Last Exam Publication") MMLU-Pro Subjects: Chat models, Reasoning models Topics: knowledge reasoning science humanities MMLU-Pro is a more difficult and robust successor to MMLU. It expands the number of answer choices, emphasizes reasoning, and covers a broad range of academic disciplines while reducing sensitivity to prompt variations and chance performance. [Leaderboard Open MMLU-Pro Leaderboard](https://huggingface.co/spaces/TIGER-Lab/MMLU-Pro "Open MMLU-Pro Leaderboard") [Code Open MMLU-Pro Code](https://github.com/TIGER-AI-Lab/MMLU-Pro "Open MMLU-Pro Code") [Data Open MMLU-Pro Data](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro "Open MMLU-Pro Data") [Paper Open MMLU-Pro Publication](https://arxiv.org/abs/2406.01574 "Open MMLU-Pro Publication") LongBench v2 Subjects: Chat models, Reasoning models, Long-context models Topics: long context document understanding reasoning retrieval LongBench v2 evaluates deep understanding and reasoning over realistic contexts ranging from thousands to millions of words. Its tasks cover documents, dialogue histories, code repositories, in-context learning, and structured data rather than simple information retrieval alone. [Leaderboard Open LongBench v2 Leaderboard](https://longbench2.github.io "Open LongBench v2 Leaderboard") [Code Open LongBench v2 Code](https://github.com/THUDM/LongBench "Open LongBench v2 Code") [Data Open LongBench v2 Data](https://huggingface.co/datasets/zai-org/LongBench-v2 "Open LongBench v2 Data") [Paper Open LongBench v2 Publication](https://arxiv.org/abs/2412.15204 "Open LongBench v2 Publication") MMMU-Pro Subjects: Multimodal models, Reasoning models Topics: multimodality visual understanding expert knowledge reasoning MMMU-Pro tests expert-level reasoning across interleaved text and images such as charts, diagrams, maps, tables, and technical figures. It strengthens the original MMMU benchmark against shortcuts that let models answer without genuinely understanding the visual information. [Leaderboard Open MMMU-Pro Leaderboard](https://mmmu-benchmark.github.io/#leaderboard "Open MMMU-Pro Leaderboard") [Code Open MMMU-Pro Code](https://github.com/MMMU-Benchmark/MMMU "Open MMMU-Pro Code") [Data Open MMMU-Pro Data](https://huggingface.co/datasets/MMMU/MMMU_Pro "Open MMMU-Pro Data") [Paper Open MMMU-Pro Publication](https://arxiv.org/abs/2409.02813 "Open MMMU-Pro Publication") Berkeley Function-Calling Leaderboard Subjects: Chat models, AI agents Topics: tool use function calling agent memory web search The Berkeley Function-Calling Leaderboard evaluates whether models can select and invoke tools accurately. Its tests span single- and multi-turn calls, relevance detection, web search, memory management, format sensitivity, latency, and cost. [Leaderboard Open Berkeley Function-Calling Leaderboard Leaderboard](https://gorilla.cs.berkeley.edu/leaderboard.html "Open Berkeley Function-Calling Leaderboard Leaderboard") [Code Open Berkeley Function-Calling Leaderboard Code](https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard "Open Berkeley Function-Calling Leaderboard Code") [Paper Open Berkeley Function-Calling Leaderboard Publication](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-184.html "Open Berkeley Function-Calling Leaderboard Publication") τ-bench Subjects: AI agents, Voice agents Topics: tool use conversation policy following knowledge retrieval τ-bench evaluates agents that must converse with users, follow policies, retrieve knowledge, and call tools in realistic enterprise workflows. Its evolving family includes multi-party interaction, knowledge-heavy tasks, and real-time voice conversations. [Leaderboard Open τ-bench Leaderboard](https://taubench.com "Open τ-bench Leaderboard") [Code Open τ-bench Code](https://github.com/sierra-research/tau2-bench "Open τ-bench Code") [Paper Open τ-bench Publication](https://arxiv.org/abs/2506.07982 "Open τ-bench Publication") SWE-bench Subjects: Code models, Coding agents Topics: bug fixing repository comprehension multilingual code multimodality SWE-bench evaluates systems on real GitHub issues that require changes to existing repositories. Its official leaderboards include Verified, Multilingual, Multimodal, Lite, and Full tracks, making results across different software-engineering settings easier to compare. [Leaderboard Open SWE-bench Leaderboard](https://www.swebench.com "Open SWE-bench Leaderboard") [Code Open SWE-bench Code](https://github.com/SWE-bench/SWE-bench "Open SWE-bench Code") [Data Open SWE-bench Data](https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified "Open SWE-bench Data") [Paper Open SWE-bench Publication](https://arxiv.org/abs/2310.06770 "Open SWE-bench Publication") LiveSWEBench Subjects: Code models, Coding agents Topics: code editing feature implementation bug fixing repository comprehension LiveSWEBench is a live, continuously updated benchmark that evaluates the software engineering (SWE) capabilities of AI agents in real-world conditions. It draws tasks from active repositories and scores agents on their ability to propose correct, executable changes. [Leaderboard Open LiveSWEBench Leaderboard](https://liveswebench.ai "Open LiveSWEBench Leaderboard") [Code Open LiveSWEBench Code](https://github.com/livebench/liveswebench "Open LiveSWEBench Code") [Data Open LiveSWEBench Data](https://huggingface.co/collections/livebench/liveswebench-67eaf012f02466f4e2a757e0 "Open LiveSWEBench Data") Terminal-Bench Subjects: Coding agents, Computer-use agents Topics: terminal use system administration coding end-to-end tasks Terminal-Bench evaluates agents on complete tasks inside real terminal environments. It covers software engineering, system administration, data processing, model training, and security work that requires more than generating an isolated answer or patch. [Leaderboard Open Terminal-Bench Leaderboard](https://www.tbench.ai/leaderboard/terminal-bench/2.0 "Open Terminal-Bench Leaderboard") [Code Open Terminal-Bench Code](https://github.com/harbor-framework/terminal-bench-2 "Open Terminal-Bench Code") [Paper Open Terminal-Bench Publication](https://arxiv.org/abs/2601.11868 "Open Terminal-Bench Publication") ```json {"@context":"https://schema.org","@graph":[{"@id":"https://promptmetheus.com/#website","@type":"WebSite","description":"Compose, test, and optimize prompts for your LLM-powered applications, agents, and workflows. Supports 100+ models, prompt versioning, automatic evaluations, and more.","name":"Promptmetheus","url":"https://promptmetheus.com/","publisher":{"@id":"https://promptmetheus.com/#identity"}},{"@id":"https://promptmetheus.com/resources/llm-benchmarks#webpage","description":"A list of benchmarks and leaderboards for Large Language Models","name":"LLM Benchmarks","url":"https://promptmetheus.com/resources/llm-benchmarks","@type":["WebPage","CollectionPage"],"breadcrumb":{"@id":"https://promptmetheus.com/resources/llm-benchmarks#breadcrumb"},"isPartOf":{"@id":"https://promptmetheus.com/#website"},"potentialAction":[{"@type":"ReadAction","target":["https://promptmetheus.com/resources/llm-benchmarks"]}],"primaryImageOfPage":{"@id":"https://promptmetheus.com/#logo"}},{"@id":"https://promptmetheus.com/#identity","@type":"Organization","name":"Promptmetheus","url":"https://promptmetheus.com","logo":{"@id":"https://promptmetheus.com/#logo"},"sameAs":["https://github.com/promptmetheus","https://discord.com/invite/7dbUb9wtGj","https://reddit.com/r/promptmetheus","https://x.com/PromptmetheusAI","https://youtube.com/@promptmetheus"]},{"@id":"https://promptmetheus.com/resources/llm-benchmarks#breadcrumb","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","name":"Home","item":"https://promptmetheus.com","position":1},{"@type":"ListItem","name":"LLM Benchmarks","position":2}]},{"@id":"https://promptmetheus.com/#logo","@type":"ImageObject","caption":"Promptmetheus","contentUrl":"https://promptmetheus.com/icon.png","inLanguage":"en-US","url":"https://promptmetheus.com/icon.png"}]} ``` ## Sitemap See the full [sitemap](https://promptmetheus.com/sitemap.md) for all pages.