Allocating experiment audiences

Overview

This topic explains how to include specific groups of contexts in an experiment audience using audience allocation.

About experiment audiences

You have the option to only include a subset of contexts in your experiments, which gives you accurate experiment results more quickly. This subset of contexts is called your "experiment audience."

For example, imagine you plan to test alternate copy for your checkout button. You target all Canadian and US contexts with the true variation for the button, which shows the new, alternate copy, but you only want to run an experiment measuring click conversions for end users in the United States.

To accomplish this, you would select the targeting rule on the flag's Targeting tab that affects US-based contexts and de-select the rule that targets contexts in Canada. This limits the end users who evaluate the flag to only those who are based in the United States.

You may want to refine your experiment audience for any of the following reasons:

• To run targeted experiments for a subset of your flag-targeted contexts.
• To exclude groups whose events you do not need to measure. For example, those affected by 'Default' rules.
• To reduce the volume of contexts in an experiment.

Create experiment audiences

You determine the initial experiment audience when you create a new experiment. You must include at least two variations in the experiment for the experiment to be valid. To learn more, read Creating experiments.

Targeting rules

You can run an experiment on a flag's default rule, or you can create a custom experiment audience by selecting a specific flag targeting rule to include in your experiment. You can target by any context attribute you collect. To learn how, read Target with flags.

Allocate audiences

When you build your experiment, you can allocate all or a percentage of the traffic that encounters a flag in an experiment. Audience allocation gives you flexibility when selecting your experiment audience and ensures accurate experiment results. LaunchDarkly analyzes only contexts that you choose to be part of the experiment.

If you decide to increase or decrease the number of contexts in an experiment, LaunchDarkly will create a new iteration of your experiment. To learn more, read Start experiment iterations.

LaunchDarkly automatically performs checks on the allocation, to make sure that actual traffic matches the allocation you set. To learn more, read Understanding sample ratios.

Change traffic allocation

If you change the amount of traffic in an experiment and start a new iteration, some of the contexts in the experiment may begin receiving different variations:

• If you increase the amount of traffic in your experiment audience, LaunchDarkly will automatically add or remove contexts to or from variations to prevent carryover bias. This is called "variation reassignment." For most experiments, we recommend that you allow variation reassignment, but you can prevent variation reassignment if you need. To learn how, read Carryover bias and variation reassignment.
• If you decrease the amount of traffic in your experiment audience, LaunchDarkly will move the appropriate percentage of contexts out of the experiment and into the control variation.
• If you start a new iteration but don't change the amount of traffic in your experiment audience, LaunchDarkly will not reassign contexts to different variations.

To learn how to change the audience for a running experiment, read Change experiment audiences.