Definition
A randomly selected share of users who keep seeing the current experience while everyone else gets a change, such as a new paywall or personalized offers. Comparing the two groups shows how much of the difference the change caused.
Big enough to measure the difference you care about within a reasonable time, usually 5–20% of traffic. Smaller holdouts cost less revenue but take longer to read.
Long enough to capture the revenue that matters. For subscription apps that often means first renewals, so 4–8 weeks is common for monthly plans and longer for annual plans if you want renewal data.
In practice, yes. "Control group" is the usual term in A/B tests; "holdout" usually means a group kept on the old experience while a system such as a personalization model runs for everyone else.
A metric that tracks users who complete a specific high-value action that strongly correlates with long-term retention or monetization such as starting a trial, subscribing, or completing onboarding. Unlike soft engagement signals, hard activations typically reflect clear product-market fit at the user level.
A monetization model where core app features or content are fully locked behind a subscription and users must pay to proceed. Hard paywalls are effective for apps with clear, high-value offerings and are often paired with a free trial to mitigate upfront friction.
Users who demonstrate strong signals of purchase readiness, such as frequent sessions, return visits, or multiple paywall views. These users are prime targets for personalized offers, upsells, and onboarding nudges to drive subscription conversion.
A behavioral design framework that explains how products create habits by cycling users through four stages: Trigger, Action, Reward, Investment. In subscription apps, the hook model is often used to design daily engagement loops that drive retention and deepen value perception. This was popularized by Nir Eyal.
A strategy that combines multiple revenue streams such as subscriptions, in-app purchases (IAPs), and advertising. Hybrid monetization allows apps to capture value from both paying and non-paying users, optimizing ARPU across segments.
Botsi picks the offer and paywall each user sees, from options you approve, and measures the lift against a holdout.