Skip to main content
To configure treatments for an experiment, you first need to select the flag to use. To select the flag, click + Add control at the top of the edit page. After you add the first treatment, click + Add treatment to add more. Add treatment After you have added a flag you can add treatments to the experiment. The first variant you add is the control group. The next variants you add are treatment groups. You can reorder the variants by dragging them left and right to select a different control. You can read more about flags and variants on the flags page.

​
Treatment Sizes

The treatment size is the size of each variant in the experiment. For example, if you run an experiment with one control group and two treatment groups, you may split your sample so that 40% of the users are randomly allocated to the control group, 40% to the first treatment group, and the remaining 20% to the second treatment group.
Divide the sample evenly between the variants whenever possible. This gives you the highest power. Do this by clicking Split weights evenly.
While an even split is optimal from a statistical perspective, there are cases where it might be difficult to achieve in practice. If you’re testing a risky change for example, you may worry about degrading the experience for your most valuable users and want to expose only a small fraction of them. In this case there are two ways to mitigate the risk and keep your treatments even:
  • You can run the test with an even split on a less risky part of the population (for example, new users, free users or target a specific country)
  • You can lower the allocation of the experiment to reduce the total population you are targeting and have an even split that would expose fewer users. See Audience for more information.
You need to be careful to ensure that the results are still generalizable. The statistical tests let you conclude causally whether a change had an effect on the metric that you observe for the population you are testing on. Testing on a subpopulation could be a good first step to evaluate the risk before testing on the rest of the population.

Create Variants

Set up flag variants for treatments

Audience Reference

Configure experiment audience

Statistical Settings

Configure experiment parameters

Launch an A/B Test

Run your experiment