A/B-тестирование
A/B Testing
A/B testing is a method of comparing two versions of a landing page, creative, or offer on randomly split traffic groups to choose the version with the better target metric.
What is A/B-тестирование (A/B Testing)?
A/B testing (split testing) is a controlled experiment: the audience is randomly divided into two groups. One group sees version A (the control, usually the current one), and the other sees version B (with a single change). The results are compared by conversion rate, cost per target action, or ROI, and a decision is made about what stays in rotation.
## How it works
Start by formulating a hypothesis and choosing one success metric, such as CR to deposit or approval rate. Change one element: a headline, a button, the order of landing page blocks, the first screen of a creative, or the bonus amount in the copy. Split traffic randomly and at the same time, otherwise season, time of day, and traffic source will distort the result. Then wait for a sufficient volume of data and check statistical significance (p-value or confidence interval).
## Where it is used
In gambling and betting, teams test pre-landers, banners, the order of bonuses, and registration copy. In crypto and fintech, they test KYC onboarding forms and the button text on confirmation steps, which affect drop-off. In dating, they test headlines and profile photos. In nutra, they test headlines, testimonials, and video formats, but it is important to respect the health claims restrictions of ad platforms.
## Common mistakes
Stopping the test at the first sign of a winner (peeking) inflates the effect. A test on a small sample produces a random winner. Changing several elements at once makes it impossible to identify the cause. Ignoring segments is also a problem: a variant may win on mobile traffic and lose on desktop. Comparing different traffic sources or GEOs makes the conclusions unreliable.
FAQ
How much traffic do I need for an A/B test?
It depends on the baseline conversion rate and the minimum effect you need to detect. The rarer the target action, the larger the sample required. Sample size is calculated in advance using a sample size calculator.
How is an A/B test different from a split test?
In everyday usage the terms often mean the same thing: both compare two variants on split traffic. Split test is sometimes used specifically when testing completely different pages with different URLs.
Can the same user be tested twice?
No. A user should be assigned to one group consistently through a cookie or ID. Otherwise the groups get mixed and the results are distorted.