Skip to content

Search the documentation by title, section, or page text.

Setup & Run A/B Test

This guide walks you through creating an A/B test, launching it, and stopping it once you have enough data. Before you start, make sure you have at least two fully configured Paywalls to compare, each with its Products, pricing, and durations in place.

Creating a test#

  1. Open A/B Testing in the sidebar and choose Create A/B Test.
  2. Give the test a name, and optionally describe what you are trying to learn in the goal field.
  3. Add at least two Paywalls. Search by name and select each one you want to compare.
  4. Use the Weight column to set how much traffic each variant receives. This is useful when you want to ramp a new variant in gradually rather than splitting traffic evenly.
  5. Choose Run A/B Test to launch, or Save as Draft to keep the configuration without going live yet.
  6. In the Running A/B Test window, select the Placement and Audience for the test, then choose Run.

Variants start serving immediately once the test is live.

The three statuses#

StatusWhat it meansWhat you can do
DraftCreated but not startedEdit or run it. Paywalls, weights, and goals are all still editable
LiveRunning, serving variants by weightModify or stop it. Avoid major changes while live
CompletedEnded, no longer collecting dataAnalyze the results, or duplicate it to run something similar

Save a test as a draft while you are still assembling it. Once it goes live, treat the configuration as frozen. Changing variants mid-test means the data before and after the change are not measuring the same thing.

Stopping a test#

  1. Go to A/B Testing in the sidebar and find the live test you want to finalize.
  2. Choose Stop.
  3. In the Stop the A/B Test window, choose what the audience sees next:
    • One of the test's own Paywalls, usually the winner
    • A different Paywall that was not part of the test
    • No Paywall at all for that audience
  4. Choose Stop and complete this A/B test.

The test moves to Completed and its results become available.

What's Next?#