Amplitude Announces New Capabilities to Help Data Leaders Scale Experimentation

March 21, 2023 at 09:00 AM EDT

New Experiment Groups allows teams to learn, iterate, and ship faster without compromising results or the customer experience

To better support data leaders as they improve their organization’s experimentation practices, digital analytics leader Amplitude, Inc. (Nasdaq: AMPL) today announced an expanded suite of product features for its world-class experimentation offering, Amplitude Experiment.

Product and data teams leverage experimentation to continually test hypotheses for ways to acquire, engage and retain their users. While data leaders want to drive innovation within their organization, many find their experimentation programs blocked by user identity issues, inconsistent metrics, long testing periods, and duplicative data sets. Built upon Amplitude’s Digital Analytics Platform and natively integrated with Amplitude Analytics, Amplitude Experiment helps data leaders unite analytics, experimentation, and new program management capabilities to scale their experimentation programs faster and further.

“Leading organizations have identified how to scale experimentation effectively, consistently measuring the impact of every feature they ship as they aim to build better products,” explained Justin Bauer, chief product officer at Amplitude.

Still, the biggest barrier to scaling product experimentation is having data and analytics to analyze the results of experiments. Disparate solutions for analytics, A/B testing, and feature management have worsened this problem, leading to inconsistent data sources and the complexity of duplicative datasets.

“With the enhancements we're making today—and our native integration with Amplitude Analytics—we’re making it easier for teams to scale their A/B testing programs without sacrificing quality or overburdening data teams," added Bauer. "With every leader focused on ROI, Amplitude Experiment is an obvious choice. It drives business value quickly while eliminating duplicative vendor costs and reducing data spend.”

According to a report from Nucleus Research, Amplitude customers reported a return on investment within the first six months following deployment leveraging Amplitude Analytics and Amplitude Experiment together¹. By consolidating experimentation and analytics, customers saw a 20 percent reduction in costs on average. They also saw a direct impact on the bottom line metrics, including a more than 3x average increase in user signups and a 13 percent average increase in revenue.

New experimentation capabilities include:

Experiment Groups: Scale experimentation without compromising results or customer experience. Separate subsets of users from every test with Mutual Exclusion Groups to ensure users are not subjected to colliding experiments, which obscures results. With Holdout Groups, designate a set of users as a control group to understand the cumulative effects across multiple experiments.
Formula Metrics: Data analysts can build formula-based metrics that ensure consistent insights and faster, coordinated action without needing engineering teams to update data pipelines.
Controlled-experiment Using Pre-existing Data (CUPED): Automatically reduce experiment variance with pre-experiment data to reach statistical significance faster, with less traffic. With CUPED, teams can better understand how product changes impact different customer segments.
Data Quality Checklist: Trust results by easily monitoring for experiment design, implementation, instrumentation, and statistical integrity issues proactively.

“We recently adopted Amplitude Experiment to simplify A/B testing and run concurrent experiments without exposing users to more than one testing scenario at a time,” said Alexander Magnusson, manager of data analytics at Brainly. “Experiment lets us integrate testing and analytics within a single ecosystem, which makes it a no-brainer.”

Visit Amplitude at booth #1243 at the Gartner Data & Analytics Summit, going on now through March 22, 2023, or follow along on Twitter, LinkedIn, and the Amplitude blog for more updates. To learn more about Amplitude Experiment, request a custom demo today.

¹Nucleus Research Inc., Product Analytics and Experimentation Unite with Amplitude, January 2023, Alexander H. Wurm

About Amplitude

Amplitude is a leading digital analytics platform that helps companies unlock the power of their products. Almost 2,000 customers, including Atlassian, Jersey Mike’s, NBCUniversal, Shopify, and Under Armour, rely on Amplitude to gain self-service visibility into the entire customer journey. Amplitude guides companies every step of the way as they capture data they can trust, uncover clear insights about customer behavior, and take faster action. When teams understand how people are using their products, they can deliver better product experiences that drive growth. Amplitude is the best-in-class analytics solution for product, data, and marketing teams, ranked #1 in multiple categories in G2’s 2023 Winter Report. Learn how to optimize your digital products and business at amplitude.com.