Rollouts

A rollout workflow is a way to gradually roll out a change or feature to a percentage of your users to ensure it doesn’t have any negative impact on your users or your systems.

The distinguishing features of a rollout compared to other types of experiments is that they:

• can only have two treatment groups
• only use guardrail metrics
• have a variable allocation that lets you gradually roll out
• only use a sequential design where you see results continuously when the rollout is live

The main reasons for gradually rolling something out as opposed to enabling it for all users immediately are:

• It lets you ensure that the experience you are rolling out is not worsening the user experience before you roll it out to everyone.

• See comparisons between the group that has the feature you are rolling out and the group that has the current default experience for any metric.

A rollout uses the same basic functionality as an A/B test. It allocates users into two groups, a control group and a treatment group, and compares them.

The difference is that in a rollout, you want to roll out a feature gradually. You might start with a 95/5 allocation with 5% of eligible units in the treatment group to limit the impact of a potential issue. You then monitor important metrics to see if you worsen the user experience. If the experiment is going well, you increase the allocation to get more units in the treatment until you have rolled it out completely.

Related Resources

Launch a Rollout

Step-by-step rollout tutorial

Guardrail Metrics

Configure rollout metrics

Monitoring

Monitor rollout health

A/B Tests

Compare with A/B test workflow