MLCommons Releases New MLPerf Inference v6.0 Benchmark Results
SAN FRANCISCO, April 01, 2026 (GLOBE NEWSWIRE) — Right this moment, MLCommons® introduced new outcomes for its industry-standard MLPerf® Inference v6.0 benchmark suite. This launch consists of a number of necessary advances that make sure the benchmark suite checks present, real-world eventualities for AI deployments and delivers a complete image of AI system efficiency.
5 of the eleven datacenter checks in MLPerf Inference v6.0 are new or up to date, and the discharge additionally features a new object-detection take a look at for edge methods. The key modifications embody:
● A brand new, open-weight large-language mannequin benchmark primarily based on GPT-OSS 120B that can be utilized for arithmetic, scientific reasoning, and coding;
● An expanded DeepSeek-R1 advanced-reasoning benchmark, together with an interactive state of affairs that allows speculative decoding;
● DLRMv3, the third era of our recommender benchmark and now the primary sequential advice benchmark take a look at within the suite, which is completely modernized primarily based on beneficiant engineering contributions from Meta, a world chief in recommender methods;
● The suite’s first text-to-video era benchmark;
● A brand new vision-language mannequin (VLM) benchmark that transforms unstructured multimodal knowledge from Shopify’s intensive product Catalog into structured metadata;
● An upgraded single-shot object detection benchmark for edge eventualities primarily based on Ultralytics’ YOLOv11 Giant mannequin.
“That is essentially the most vital revision of the Inference benchmark suite that we’ve ever executed,” stated Frank Han, Technical Workers, Programs Improvement Engineering at Dell Applied sciences and MLPerf Inference Working Group Co-chair. “The choice to replace so many benchmarks on this spherical was prompted by the extraordinary enthusiasm and collaboration from our members, who contributed an unprecedented quantity of engineering effort and IP towards constructing new inference benchmarks. Including these new checks permits MLPerf Inference to raised maintain tempo with the breakneck tempo of evolution in AI fashions and methods in order that our benchmarks are related and consultant of real-world deployments.”
The open-source MLPerf Inference benchmark suite measures system efficiency in an architecture-neutral, consultant, and reproducible method. The objective is to create a degree enjoying area for competitors that drives innovation, efficiency, and power effectivity for all the {industry}. The printed outcomes present vital technical data for patrons who’re procuring and tuning AI methods.
“We thank Meta, Shopify and Ultralytics for his or her substantial collaboration with us in making these modifications to the MLPerf Inference benchmark suite and for contributing their datasets, job definitions and experience,” stated Miro Hodak, Senior Member of Technical Workers at AMD and MLPerf Inference Working Group Co-chair. “These partnerships have been important in guaranteeing that the checks embody eventualities and workloads that characterize the present state of the {industry}.”
“MLPerf Inference benchmarks play an important function in driving transparency and accountability throughout the AI {industry},” stated Glenn Jocher, CEO & Founding father of Ultralytics. “At Ultralytics, rigorous, reproducible benchmarking is central to how we develop and validate our Ultralytics YOLO fashions — guaranteeing builders and organizations could make knowledgeable choices about real-world efficiency. We’re proud to be a part of an ecosystem that holds all the area to the next normal.”
“Commerce is likely one of the most advanced domains in AI, but researchers not often have knowledge that displays that complexity,” stated Kshetrajna Raghavan, Principal Engineer, Utilized ML at Shopify. “Shopify is uniquely positioned to deal with this, sitting on the intersection of hundreds of thousands of retailers and billions of merchandise. Sharing this taxonomy permits the entire area to evolve.”
New instruments for submitters and customers
With Inference 6.0, submitters have the choice to make use of a newly obtainable harness to finish benchmark checks. The brand new system, LoadGen++, permits LLMs to run with a serving-style software program stack, which is acquainted from typical deployments in the present day. “LoadGen++ is a big improve from its predecessor, and represents an necessary funding by MLCommons that can enable us to remain nimble as we proceed to supply benchmark checks that observe the state-of-the-art,” stated Han.
As well as, the Inference 6.0 outcomes might be seen in a brand new on-line dashboard https://mlcommons.org/visualizer on the MLCommons web site. The dashboard brings new ranges of interactivity to viewing outcomes, together with superior filtering and customised efficiency graphs.
Giant-scale, multi-node methods gaining consideration
The submissions to Inference 6.0 show that know-how suppliers wish to showcase the efficiency of scaled-up, multi-node methods working real-world inference workloads. This spherical recorded a brand new excessive for multi-node system submissions, a 30% improve over the Inference 5.1 benchmark six months in the past. Furthermore, 10% of all the submitted methods in Inference 6.0 had greater than ten nodes, in comparison with solely 2% within the earlier spherical. The most important system submitted in Inference 6.0 featured 72 nodes and 288 accelerators, quadrupling the variety of nodes within the largest system within the earlier spherical.
“As extra AI purposes have moved into manufacturing and large availability, the demand for large-scale, high-performance methods to run them has grown,” says Hodak. “On the similar time, multi-node methods deliver a novel set of technical challenges past these of single-node methods, requiring configuration and optimization of system architectures, community interconnects, knowledge storage, and software program layers. Stakeholders are eagerly stepping as much as meet these challenges and run inference workloads at scale.”
The AI group continues to embrace and put money into MLPerf Inference
The MLPerf Inference 6.0 benchmark obtained submissions from a complete of 24 collaborating organizations: AMD, ASUSTeK, Cisco, CoreWeave, Dell, GATEOverflow, GigaComputing, Google, Hewlett Packard Enterprise, Intel, Inventec Company, KRAI, Lambda, Lenovo, MangoBoost, MiTAC, Nebius, Netweb Applied sciences India Restricted, NVIDIA, Oracle, Quanta Cloud Know-how, Purple Hat, Stevens Institute of Know-how, and Supermicro.
“I wish to welcome our first-time submitters, Inventec Company, Netweb Applied sciences India Restricted, and Stevens Institute of Know-how,” stated Han. “The AI ecosystem is giant and numerous, and it continues to develop and evolve quickly. On behalf of MLCommons, I wish to additionally thank our members, our contributors, and our companions together with Meta, Shopify and Ultralytics for collaborating with us to construct and shepherding ahead essentially the most complete and related efficiency benchmark suite for AI inference. Collectively, we’re guaranteeing that stakeholders in our group have helpful, real-world data that helps them to make higher choices.”
View the outcomes
To view the outcomes for MLPerf Inference v6.0, please go to the benchmark outcomes dashboard https://mlcommons.org/visualizer.
About MLCommons
MLCommons is the world’s chief in AI benchmarking. An open engineering consortium supported by over 130 members and associates, MLCommons has a confirmed document of bringing collectively academia, {industry}, and civil society to measure and enhance AI. The inspiration for MLCommons started with the MLPerf benchmarks in 2018, which quickly grew right into a set of {industry} metrics for measuring machine studying efficiency and selling transparency in machine studying methods. Since then, MLCommons has continued to make use of collective engineering to construct the benchmarks and metrics required for higher AI – in the end serving to to guage and enhance the accuracy, security, pace, and effectivity of AI applied sciences.
For extra data on MLCommons and particulars on turning into a member, please go to MLCommons.org or e mail participation@mlcommons.org.
Press Inquiries: Contact press@mlcommons.org
