A/B Tests in Traffic Control

You can run a real A/B test on a controlled share of an AI Pricing Model's traffic. Add the Placement hosting the test to the model's distribution, give it a weight, and that share of the model's users goes into the test while the model keeps deciding for everyone else.

This is a user-level test, not a per-request coin flip. Which users fall into the test slice is worked out from the profile ID you send, so the same person is in the test on every request. Which variant that person sees is worked out the same way, so it stays the same for as long as the test run lasts. And the Paywall response carries abTestId, so those requests are attributed to the test rather than folded into the model's numbers.

Why run a test this way#

Try creatives or price points you are not ready to hand to the model. The test Placement can hold anything: a new design, a different price ladder, a seasonal offer. None of it has to be attached to the model.
Validate a new idea while your current best keeps running. The model decides for its share the whole time, so you measure the new idea against a model that is still working, not against a pause.
Keep the majority on the model. Give the test 10, 20, or 30 percent and the model still decides everything else. You choose how much revenue is exposed to an unproven Paywall.
Ramp up by moving one number. Raising the test's share is a weight edit and a save, applied from the next request. Widening only adds users, so a ramp-up never discards data you already collected.

What has to exist first#

The A/B test has to exist and be running. Build it in the A/B Testing section and start it, choosing the Placement and audience it runs on. See Setup & Run A/B Test. Traffic Control targets a Placement, never a Paywall ID and never a test directly.

Botsi checks the Placement when you save, and accepts it only if all three hold:

It has exactly one audience.
That audience is the all-users audience, not a segment.
That audience is running a live A/B test.

A Placement serving a plain Paywall, a Placement owned by another AI Pricing Model, and a Placement whose test has been stopped are all refused. That is what makes a Placement allocation a genuine test arm rather than a general-purpose redirect.

Use a different Placement from the one the model is bound to

The model's own Placement is owned by the model, so it fails the check above and cannot be saved as a target.

Setting it up#

Create the A/B test on its Placement and start it. Note the Placement's name. You pick it by name in a moment.
Open the model's Traffic Control tab and choose + Add Subplacement in the distribution at the top. That section is headed Current Distribution, or Default Distribution while a schedule is running.
Find and select the Placement. Each carries a chip for the routing its audiences use, so an A/B Test chip is the signal you want. The chip does not distinguish a live test from a stopped one, so confirm the test is running before you save. Choose Continue to Traffic Allocation.
Set the weights. The dialog lists the existing AI Model and Baseline rows alongside the new Placement, so you can rebalance all of them at once. Weights must total exactly 100, and Confirm stays disabled until they do.
Save. Check the header reads Total: 100% and choose Save Changes.

A refused save does not name the reason

If the Placement does not qualify, the save is refused and nothing about your distribution changes, but the on-screen error reads only Failed to save distribution.

If a save fails immediately after adding a Placement row, check that Placement against the three conditions above.

A two-way split is entered as three numbers. The AI Model and Baseline rows are permanent and cannot be removed, only re-weighted, so a 50/50 model-versus-test split is AI Model 50, Baseline 0, Placement 50. A weight of 0 is a valid saved value and that row is never drawn by the split.

A worked example#

Take a distribution giving the model 50%, the model's Baseline Paywall 20%, and a Placement running an A/B test 30%.

Three in ten of the users who reach this model are in the test slice, and they are the same three in ten on every request, each seeing the same variant every time. The other seven in ten stay with the model, which splits them between the AI decision service and the Baseline Paywall in the ratio 50 to 20. That second split is drawn fresh on each request, so it is the volumes that hold at 50% and 20% overall rather than any individual user's arm.

To give the test more traffic, raise its weight, lower another row so the total is still 100, and save.

What to send, and what comes back#

Send a profile ID on every Paywall request.

The response tells you which arm served the request:

Arm that served	Fields on the response
The test slice (your Placement row)	`abTestId` and `abTestName`. `aiPricingModelId` is deliberately not sent, and `isExperiment` is `false`.
AI Model row	`aiPricingModelId`, and `isExperiment` is `true`.
Baseline row	`aiPricingModelId`, and `isExperiment` is `false`.

One detail worth building against: fields that are not set are omitted from the JSON entirely rather than sent as null, while isExperiment is always present as a boolean.

A request served by the test slice comes back in exactly the same shape as a request to a Placement running that test directly. The redirect is invisible in the answer, which is the point.

How users are assigned#

Two independent decisions happen, both computed from the profile ID rather than stored anywhere.

Am I in the test slice? Seeded on the distribution the allocation belongs to. Because the seed is the distribution and not the individual rows, editing weights and re-saving does not reshuffle who is in the test.
Which variant do I get? Seeded on the test's own salt, decided by the A/B test. Traffic Control decides how much traffic reaches the test and has no influence on which variant a request lands on.

What does and does not move people:

Raising the test's weight adds users and removes none. Everyone already in the slice stays; new users join from the model's share.
Lowering it removes the users closest to the boundary and leaves the rest untouched.
Re-saving the distribution does not reshuffle the slice.
Relaunching the A/B test reshuffles variants on purpose. Starting a test mints a fresh salt, so a relaunched test is a fresh randomization rather than a continuation. A test left running never reshuffles.
A schedule has its own membership. While a scheduled window is open, the slice is drawn independently of the always-on one.
With more than one test Placement, order matters. Slices are laid out in row order, so changing an earlier row's weight shifts the boundaries of the ones after it.

Running the test for a fixed window#

A Scheduled Distribution can point at the same Placement, which is how you run a test only between two dates. A schedule fully replaces the always-on distribution while its window is open, and the schedule picker offers Placements only, so a scheduled window splits its traffic between Placements rather than including the AI Model row. Every Placement in the window has to pass the same three conditions.

Reading the results#

Two systems are doing two different jobs, and each reports on its own.

The A/B test decides which variant wins. Variant selection, and the comparison between variants, belong to the test. Its own results screen is where you read conversion, revenue, and engagement per variant. See Read Test Results.

Traffic Control decides how much traffic reaches the test. The weight is the share of this model's users handed to it. Changing the weight changes the sample size, never the variant split inside it.

What you can separate#

Test traffic from model traffic. An answer from the test slice carries abTestId and no aiPricingModelId, so test requests do not silently inflate the model's baseline numbers.
The model's two arms from each other. Both carry aiPricingModelId, and isExperiment tells them apart.
Arms by the Paywalls they serve. The test's variants, the model's variants, and the model's Baseline Paywall are different Paywalls, so Paywall-level numbers separate the arms cleanly. Keep the test's variants out of the model's own Paywall list and that stays true.

What you cannot do yet#

There is no single screen comparing the arms. Botsi does not break out Traffic Control allocations as columns, so reading the test against the model means reading both results side by side, or keying your own analytics on the fields above.

Do not treat the AI arm as a fourth variant

The test's variants are assigned by the test, and the model's arms are drawn per request by the model. Those are two different assignment mechanisms, so a variant-versus-AI comparison is directional, not a controlled result.

The controlled comparison lives inside the test.

Which should I use#

Use	When
A Traffic Control Placement allocation	You want a test on part of an AI Pricing Model's traffic, with the model deciding the rest. The share is yours to set and change, and you can ramp it up without restarting the test.
A standalone Placement A/B test	The test is the whole story for that Placement. Every request matching the audience goes into it, and no model splits traffic ahead of it.

It is the same A/B test object either way: configured in the same place, bucketed the same way, reported through the same identity. The difference is how much traffic reaches it and who decides the rest.

Good to know#

Stopping the A/B test does not stop it serving. The live check runs when you save the allocation, not when a Paywall is served, so a stopped test keeps handing out its variants through the allocation. Set the weight back to 0, or remove the row, when the test is finished.
A baseline at 0 can still serve. It is what the AI Model row falls back to when it cannot get a decision: a request with no profile ID, or a decision service error or empty response.
Saving the always-on distribution writes back to the model. The Paywall flagged as best keeps that flag and has its stored weight rewritten. The model's own weight field is not updated, so it goes stale. Scheduled saves do not do this.
Editing the model's weight rewrites part of the split. It rewrites the AI Model and Baseline rows and leaves the Placement row untouched, so the distribution can end up over 100 and is renormalized at serve time.
Deleting the Placement removes the allocation, silently, leaving the distribution below 100. With more than one test Placement, deleting one widens the others' shares.
Changing the target Placement's setup after saving does not break serving. The eligibility check runs at save time only. It keeps returning a Paywall, but what it returns may no longer be a test.