Creating Data Export experiments

Overview

This topic explains how to set up and configure an experiment in LaunchDarkly that you plan to analyze in an external tool.

Data Export experiments let you export experiment data to a third-party tool for analysis using Data Export. Data Export experiments do not require a LaunchDarkly metric, and do not generate analysis results in the LaunchDarkly user interface (UI). This experiment type is useful when you want to use LaunchDarkly flags to manage your experiment, but have a preferred external analysis tool that allows you to import experiment data from other systems.

Configuring a Data Export experiment requires the following steps:

1. Enabling Data Export
2. Building the experiment
3. Turning on the feature flag
4. Starting an iteration
5. You will then automatically begin receiving experiment data in your Data Export files.

These steps are explained in detail below.

Prerequisites

Before you build a Data Export experiment, you must:

• enable Data Export in your LaunchDarkly account
• understand randomization units
• understand mutually exclusive experiments

Build experiments

You can view all of the experiments in your environment on the Experiments list.

To build an experiment:

1. Click Create, then choose Experiment.
2. The "Experiment details" step opens. Enter a Name.
3. Enter a Hypothesis.
4. Select the Data Export only experiment type.
5. (Optional) If you want to include the experiment in a holdout, click Advanced, then select a Holdout name.

6. Click Next. The "Choose randomization unit and attributes" step opens.
7. Choose a context kind from the Randomization unit menu. If you included the experiment in a holdout, the experiment's randomization unit is automatically set to the holdout's randomization unit.
8. Click Next. The "Choose flag variations" step opens.
9. Choose a Flag to use in the experiment, or click Create flag to create and use a new flag.
10. Click Next. The "Set audience" step opens.
11. (Optional) Check the Add experiment to layer checkbox if you want to exclude contexts in this experiment from other experiments.

12. In the "Define audience" section, choose which targeting rule to run the experiment on.

• If you want to restrict your experiment audience to only contexts with certain attributes, create a targeting rule on the flag you include in the experiment and run the experiment on that rule.
• If you don't want to restrict the audience for your experiment, run the experiment on the flag's default rule. If the flag doesn't have any targeting rules, the flag's default rule will be the only option.

13. Enter the percentage of traffic you want to include in the experiment.
14. Select which control variation you want LaunchDarkly to serve to the remaining population.

• (Optional) Advanced: For most experiments, we recommend leaving the Advanced options on their default settings. To learn more, read Carryover bias and variation reassignment.

  

15. Choose how you want to split the experiment audience:

• Equally splits the experiment traffic equally between all variations in the experiment.
• Percentage allows you to select specific percentages for each variation.

16. Click Finish. You are returned to the experiment's Design tab.

After you have created your experiment, the next steps are to toggle on the flag and start an iteration.

Turn on feature flags

For an experiment to begin recording data, you must toggle On the flag used in the experiment. To learn how, read Turning flags on and off.

You can build multiple experiments on the same flag, but you can run only one of those experiments on the flag at a time.

Start experiment iterations

After you create an experiment and toggle on the flag, you can start an experiment iteration in one or more environments.

To start an experiment iteration:

1. Navigate to the Experiments list.
2. Click on the environment section containing the experiment you want to start.

• If the environment you need isn't visible, click the + next to the list of environment sections. Search for the environment you want, and select it from the list.

3. Click on the name of the experiment you want to start an iteration for. The Design tab appears.
4. Click Start.
5. Repeat steps 2-4 for each environment you want to start an iteration in.

Experiment iterations allow you to record experiments in individual blocks of time. To ensure accurate experiment results, when you make changes that impact an experiment, LaunchDarkly starts a new iteration of the experiment.

To learn more about starting and stopping experiment iterations, read Managing experiments. Each time you start an iteration, you will automatically begin receiving experiment data in your Data Export files.