A/B Tests in Traffic Control

You can run a real A/B test on a controlled share of an AI Pricing Model's traffic. Add the placement that hosts the test to the model's distribution, give it a weight, and that share of the model's users goes into the test while the model keeps deciding for everyone else.

This is a user-level test, not a per-request coin flip. Which users fall into the test slice is worked out from the profile ID you send, so the same person is in the test on every request. Which variant that person sees is worked out the same way, so it stays the same for them for as long as the test run lasts. And the paywall response carries the test's own identity, abTestId, so those requests are attributed to the test instead of being folded into the model's numbers.

Why run a test this way

Try creatives or price points you are not ready to hand to the model. The test placement can hold anything: a new design, a different price ladder, a seasonal offer. None of it has to be attached to the model or evaluated by it.
Validate a new idea while your current best keeps running. The model decides for its share of traffic the whole time the test is live, so you are measuring the new idea against a model that is still working, not against a pause.
Keep the majority on the model. Give the test 10, 20 or 30 percent and the model still decides everything else. You choose how much of your revenue is exposed to an unproven paywall.
Every user in the test gets a consistent experience. Nobody flips between the test and the model between app opens, and nobody flips between variants. That is what makes the result readable as a test rather than as noise.
Ramp up by moving one number. Raising the test's share is a weight edit and a save, applied from the next paywall request. Widening only adds users to the slice: everyone already in the test stays in it, so a ramp-up never discards the data you have already collected.

Before you start

The A/B test has to exist and be running. Build it in the A/B Testing section and start it, choosing the Placement and Audience it runs on. See Perform an A/B Test. Traffic Control targets a placement, never a paywall ID and never a test directly.
The placement you point at has to qualify as a test slice. Botsi checks this when you save, and accepts the placement only if all three hold:
- it has exactly one audience,
- that audience is the all-users audience, not a segment,
- and that audience is running a live A/B test.
A placement serving a plain paywall, a placement owned by another AI Pricing Model, and a placement whose test has been stopped are all refused. This is why a placement allocation is a genuine test arm rather than a general-purpose redirect: the only thing you can point it at is a running test on a clean audience.
Use a different placement from the one the model is bound to. The model's own placement is owned by the model, so it fails the check above and cannot be saved as a target.
Send a profile ID with your paywall requests. It is what makes the slice and the variant stable. See What to send, and what comes back.

Set it up

Create the A/B test on its placement and start it. Note the placement's name, you will pick it by name in a moment.
Open the model's Traffic Control tab and click + Add Subplacement in the always-on distribution at the top of the tab. That section is headed Current Distribution, or Default Distribution while a schedule is running.
In step 1 of the dialog, find the placement that hosts the test and select it. Each placement in the list carries a chip for the routing its audiences use, AI Model, A/B Test or Paywall, so an A/B Test chip is the signal you want. The chip does not distinguish a live test from a stopped one, so confirm the test is running before you save. Click Continue to Traffic Allocation.
In step 2, Traffic Allocation, set the weights. The dialog lists the existing AI Model and baseline rows alongside the new placement, so you can re-balance all of them at once. Weights must total exactly 100, and Confirm stays disabled until they do.
Back on the tab, check that the header reads Total: 100% and click Save Changes. If the placement does not qualify, the save is refused and nothing about your distribution changes. The on-screen error here reads only Failed to save distribution and does not name the reason, so if the save fails immediately after adding a placement row, check that placement against the three conditions above.

Entering a 50% model, 50% test split. The AI Model and baseline rows are permanent and cannot be removed, only re-weighted, so a two-way split is entered as three numbers: AI Model 50, baseline 0, placement 50. A weight of 0 is a valid saved value and that row is never drawn by the split. The baseline can still appear as the AI Model target's fallback, see Good to know below.

A worked example

Screenshot of the Traffic Control tab showing a Current Distribution with three allocation rows: AI Model at 50, the baseline paywall tagged Paywall at 20, and a placement tagged A/B Test at 30, above a stacked allocation bar with a Total: 100% readout and an Active badge, alongside a right-hand panel headed 'Your distribution is allocated' with a Scoped to section and an Add Subplacement button.

The distribution above gives the model half the traffic, the model's baseline paywall 20 percent, and a placement running an A/B test 30 percent. The stacked bar is the same split drawn to scale, Total: 100% confirms it can be saved, and the Active badge means this is what is serving right now. Selecting the placement row fills the right-hand panel, where Scoped to names the placement the allocation resolves through.

In practice: three in ten of the users who reach this model are in the test slice, and they are the same three in ten on every request, each seeing the same variant every time. The other seven in ten stay with the model, which splits them between the AI decision service and the baseline paywall in the ratio 50 to 20. That second split is drawn fresh on each request, so it is the volumes that hold at 50 percent and 20 percent overall rather than any individual user's arm. The numbers you configure are the shares you get.

To give the test more traffic, raise its weight, lower another row so the total is still 100, and save.

What to send, and what comes back

Send a profile ID on every paywall request. It is the subject the test slice and the variant are both computed from.

On the SDK endpoint, pass it as the profileId query parameter.
On the Web API endpoint, pass profileId in the body. You can pass customerUserId instead and Botsi will resolve it to the profile.

Without a profile ID there is nobody to keep stable. Botsi does not invent an identity, because a fabricated one would look sticky while actually being random. With no profile ID the whole allocation becomes a per-request draw: the same caller can land in the test on one request and on the model on the next, and the variant is redrawn each time as well. The AI Model row also cannot run without a profile ID and serves the baseline paywall instead. Anonymous traffic will still receive a paywall, it just will not produce readable test data.

What comes back tells you which arm served the request:

Arm that served	Fields on the response
The test slice (your placement row)	`abTestId`, plus `abTestName` on the Web API. `aiPricingModelId` is deliberately not sent, and `isExperiment` is `false`.
AI Model row	`aiPricingModelId`, and `isExperiment` is `true`.
Baseline row	`aiPricingModelId`, and `isExperiment` is `false`.

Two details worth building against. Fields that are not set are omitted from the JSON entirely rather than sent as null, while isExperiment is always present as a boolean. And abTestName is returned by the Web API only: the SDK response carries abTestId but not the name, so resolve the name from the ID if you need it there.

A request served by the test slice comes back in exactly the same shape as a request to a placement running that test directly. The redirect is invisible in the answer, which is the point: the test is a test either way.

How users are assigned

Two independent decisions happen, and both are computed from the profile ID rather than stored anywhere. Nothing is written to a table, nothing has to be migrated, and the same inputs always give the same answer.

Am I in the test slice? Seeded on the distribution the allocation belongs to. Because the seed is the distribution and not the individual allocation rows, editing weights and re-saving the distribution does not reshuffle who is in the test.
Which variant do I get? Seeded on the test's own salt, decided by the A/B test. Traffic Control decides how much traffic reaches the test and has no influence on which variant a request lands on.

What does and does not move people:

Raising the test's weight adds users, and removes none. Everyone already in the slice stays in it and new users join from the model's share.
Lowering it removes the users closest to the boundary and leaves the rest untouched.
Re-saving the distribution does not reshuffle the slice. Saved allocations are rewritten on every save, and membership survives that.
Relaunching the A/B test reshuffles variants on purpose. Starting a test mints a fresh salt, so a relaunched test is a fresh randomisation rather than a continuation. A test that is left running never reshuffles.
A schedule has its own membership. A Scheduled Distribution is a separate distribution, so while its window is open the slice is drawn independently of the always-on one. Deleting a schedule and adding it again creates a new one and reshuffles that window's membership.
With more than one test placement, order matters. Slices are laid out in row order, so changing an earlier row's weight shifts the boundaries of the ones after it. Ramping a single test up and down does not disturb anything if it is the only placement row.

Running the test for a fixed window

A Scheduled Distribution can point at the same placement, which is how you run a test only between two dates. A schedule fully replaces the always-on distribution while its window is open, and the schedule picker offers placements only, so a scheduled window splits its traffic between placements rather than including the AI Model row. Every placement in a scheduled window has to pass the same three conditions listed in Before you start. When the window closes, traffic returns to the always-on distribution on its own. See Scheduled Distributions.

Reading the results

Two systems are doing two different jobs here, and each reports on its own.

The A/B test decides which variant wins. Variant selection, and the comparison between variants, belong to the test. Its own results screen is where you read conversion, revenue and engagement per variant. See Read Test Results.
Traffic Control decides how much traffic reaches the test. The weight is the share of this model's users handed to the test. Changing it changes the sample size, never the variant split inside it.

What you can separate

Test traffic from model traffic. An answer from the test slice carries abTestId and no aiPricingModelId. Test requests are not counted under the model's identity, so they do not silently inflate the model's baseline numbers.
The model's two arms from each other. Both carry aiPricingModelId, and isExperiment tells them apart: true for a paywall chosen by the AI decision service, false for the baseline.
Arms by the paywalls they serve. The test's variants, the model's variants and the model's baseline paywall are different paywalls, so paywall-level numbers separate the arms cleanly. Keep the test's variants out of the model's own paywall list and that stays true.

What you cannot do yet

There is no single screen comparing the arms. Botsi does not currently break out Traffic Control allocations as columns, so reading the test against the model means reading the test's results and the model's results side by side, or keying your own analytics on the fields above.
Do not treat the AI arm as a fourth variant. The test's variants are assigned by the test, and the model's arms are drawn per request by the model. They are two different assignment mechanisms, so a variant-versus-AI comparison is directional, not a controlled result. The controlled comparison lives inside the test.

Which should I use

Botsi runs paywall tests in two places, and they answer different questions.

Use	When
A Traffic Control placement allocation	You want a test to run on part of an AI Pricing Model's traffic, with the model deciding the rest. The share is yours to set and change, and you can ramp it up without restarting the test.
A standalone placement A/B test	The test is the whole story for that placement. Every request matching the audience goes into the test, and no model is splitting traffic ahead of it.

It is the same A/B test object either way, configured in the same place, bucketed the same way, and reported through the same identity. The difference is how much traffic reaches it and who decides the rest.

Good to know

Stopping the A/B test does not stop it serving. The check that the test is live runs when you save the allocation, not when a paywall is served, so a stopped test keeps handing out its variants through the allocation, bucketed exactly as before. Set the weight back to 0, or remove the placement row, when the test is finished.
A baseline at 0 can still serve. The baseline paywall is also what the AI Model row falls back to when it cannot get a decision: a request that carries no profile ID, or a decision service error or empty response. A request with no profile ID never reaches the AI decision service at all, whatever the weights say.
Saving the always-on distribution writes back to the model. The paywall flagged as best keeps that flag and has its stored weight rewritten from the weights you just saved. The model's own weight field is not updated, so it goes stale after a Traffic Control edit. Scheduled saves do not do this.
Editing the model's weight rewrites part of the split. Changing the weight on the model edit screen rewrites the AI Model and baseline rows of the always-on distribution and leaves the placement row untouched, so the distribution can end up totalling more than 100 and is renormalised at serve time. Reopen Traffic Control after any model weight edit and re-check the numbers.
Deleting the placement removes the allocation. If the placement hosting the test is deleted, its allocation row goes with it, silently, leaving the distribution below 100 and the remaining rows renormalised at serve time. With more than one test placement, deleting one widens the others' shares. Reopen Traffic Control after any placement cleanup.
Changing the target placement's setup after saving does not break serving. The eligibility check runs at save time only, and serving stays forgiving if the target drifts afterwards. It will keep returning a paywall, but what it returns may no longer be a test, so re-check the allocation whenever you rework a placement it points at.