# A/B testing vs personalization vs bandits

> A/B testing, personalization, and bandits all choose among variants, but they answer different questions: the best single variant for everyone, the best variant per person, or how to learn while serving. The distinction decides what you can conclude and what you can deliver.

_Last updated 2026-11-01. From The Personalization Guide, https://personalization.guide_

A/B testing, personalization, and bandits all choose among variants, but they answer different
questions. An A/B test measures which single variant is best for everyone. Personalization chooses the
best variant for each person or context. A bandit learns while it serves, shifting traffic toward the
better performers as evidence arrives. Confusing them leads teams to claim one while doing another.

## What each one is

### A/B testing

An A/B test randomly assigns users to two or more variants and compares an outcome to measure a causal
effect. Randomization is the point: it is what lets you attribute the difference to the variant rather
than to who happened to see it. An A/B test is built to reach a conclusion and then stop. Its output is
knowledge, the best single variant, which you then ship to everyone.

### Personalization

Personalization chooses a variant per person or context from data about them. Where an A/B test asks
which one is best for everyone, personalization asks which one is best for this person, right now. The
output is not a single winner but a policy that can give two people different variants from the same
logic.

### Bandits

A bandit splits traffic among variants while it learns, moving weight toward the better performers as
evidence accumulates. It balances exploration, trying options to gather evidence, against
exploitation, using the current best. That cuts the cost of showing losing variants, because poor
performers lose traffic automatically, but it makes clean causal inference harder than a fixed A/B
test. A contextual bandit also reads features of the user and the situation, so it can learn that one
option works in the morning and another at night. That makes a contextual bandit a form of in-session
personalization.

## The distinguishing test

**Distinguishing test: best single variant, best per person, or learning while serving?** An A/B test
finds the best single variant for everyone and then stops. Personalization finds the best variant for
each person or context. A bandit sits between experimentation and personalization: it trades a clean
causal estimate for lower cost while it runs, and a contextual bandit crosses fully into
personalization by varying the choice with context.

## How they fit together

These are points on a line from measuring to tailoring, not rivals. A/B testing produces the
trustworthy estimates that tell you whether a personalization policy is worth running at all. Bandits
are a practical middle ground when the cost of showing losing variants is high and you can accept
weaker statistical clarity, with off-policy evaluation used to estimate a policy's value from logged
data. Personalization is the end state where the choice is made per person. Many production systems use
all three: an A/B test to validate the approach, a bandit to allocate traffic as it learns, and a
contextual model to vary the decision by who is in front of it. Which variant actually works for whom
is a question only [measurement](/measurement) can answer.