The initial focus will be on the development of safety benchmarks for large language models used for generative AI — using Stanford's groundbreaking HELM framework.

Today, MLCommons®, the leading AI benchmarking organization, is announcing the creation of the AI Safety (AIS) working group. The AIS will develop a platform and pool of tests from many contributors to support AI safety benchmarks for diverse use cases.

AI systems offer the potential for substantial benefits to society but they are not without risks, such as toxicity, misinformation, and bias. As with other complex technologies, society needs industry-standard safety testing to realize the benefits while minimizing the risks.

The new platform will support defining benchmarks that select from the pool of tests and summarize the outputs into useful, comprehensible scores – similar to what is standard in other industries such as automotive safety test ratings and energy star scores.

The effort’s immediate priority will be supporting rapid evolution of more rigorous and reliable AI safety testing technology. The AIS working group will draw upon the technical and operational expertise of its members, and the larger AI community, to help guide and create the AI safety benchmarking technologies.

"The open and dynamic nature of the safety benchmarks being developed by the broad AI community creates real incentives to set and meet common goals," said Joaquin Vanschoren, Associate Professor of Machine Learning at Eindhoven University of Technology. "Anyone can propose new tests if they see unsolved safety issues. We have some of the smartest minds in the world coming together to actually solve these issues, and using benchmarks means we will have clear insights on which AI models best address safety concerns."

The initial focus will be developing safety benchmarks for large language models (LLMs), building on groundbreaking work done by researchers at Stanford University’s Center for Research on Foundation Models and its Holistic Evaluation of Language Models (HELM). In addition to building upon the HELM framework and incorporating many of its safety related tests, we expect several companies to externalize AI safety tests they have used internally for proprietary purposes, and share them openly with the MLCommons community, which will help speed the pace of innovation.

Percy Liang, the director for the Center for Research on Foundation Models (CRFM) says: “We have been developing HELM, a modular framework for evaluation, for about 2 years. I’m very excited to work with MLCommons to leverage HELM for AI safety evaluation, which is a topic that I’ve been thinking about for 7 years, but has become extremely urgent with the rise of powerful foundation models.”

The AIS working group believes that, as testing matures, standard AI safety benchmarks will become a vital element of our approach to AI safety. This aligns with responsible development and risk-based policy frameworks such as the voluntary commitments on safety, security, and trust that several tech companies made to the White House in July 2023, NIST’s AI Risk Management Framework, and the EU’s forthcoming AI Act.

MLCommons supports a broad set of stakeholders across industry and academia in developing shared data, tools, and benchmarks to more effectively build and test AI systems. “We are excited to work collaboratively with our members,” said David Kanter, MLCommons Executive Director. “Over the next year we will focus on building and deploying AI Safety benchmarks, beginning first with open source models and with the aim of applying the benchmarks more broadly to other LLMs once the initial methodology is proven.”

The AIS working group’s initial participation includes a multi-disciplinary group of AI experts including: Anthropic, Coactive AI, Google, Inflection, Intel, Meta, Microsoft, NVIDIA, OpenAI, Qualcomm Technologies, Inc., and academics Joaquin Vanstoren from Eindhoven University of Technology, Percy Liang from Stanford University, and Bo Li from the University of Chicago. Participation in the working group is open to academic and industry researchers and engineers, as well as domain experts from civil society and the public sector. Please see our AIS Working Group page for information on how to participate.

About MLCommons

MLCommons is the world leader in building benchmarks for AI. It is an open engineering consortium with a mission to make machine learning better for everyone through benchmarks and data. The foundation for MLCommons began with the MLPerf benchmark in 2018, which rapidly scaled as a set of industry metrics to measure machine learning performance and promote transparency of machine learning techniques. In collaboration with its 125+ members, global technology providers, academics and researchers, MLCommons is focused on collaborative engineering work that builds tools for the entire machine learning industry through benchmarks and metrics, public datasets and best practices.

For additional information on MLCommons and details on becoming a Member or Affiliate of the organization, please visit http://mlcommons.org and contact participation@mlcommons.org.

View source version on businesswire.com: https://www.businesswire.com/news/home/20231026101378/en/

MLCommons announces a new AI Safety working group. Comprised of experts across industry and academia, they are building a platform & pool of tests for industry-std AI safety testing. https://mlcommons.org/en/news/formation-ai-safety-working-group/