# Reinforcement learning in personalization

> Reinforcement learning extends bandits to sequences of decisions, choosing each action for its long-term value rather than the next click. In marketing it is more often promised than practiced, because rewards are sparse and delayed and the feedback loop is hard to close.

_Last updated 2026-11-01. From The Personalization Guide, https://personalization.guide_

Reinforcement learning is a family of methods in which a system learns a policy, a rule for choosing
actions, by taking actions and observing the rewards they lead to over time. In personalization it
matters most where decisions come in sequences, and where the value of an action depends on what it
sets up next rather than on the immediate click.

## From bandits to sequences

A [bandit](/library/techniques/contextual-bandits) treats each decision as isolated: show a variant,
observe the reward, update. Reinforcement learning widens the frame to a sequence, where the system is
in some state, takes an action, moves to a new state, and collects a reward along the way. The goal is
the total reward over the whole sequence, not the next click.

The personalization cases that fit this shape are the ones with a horizon. Ordering a series of
onboarding messages to maximize whether someone is still active in a month, pacing offers so a discount
is not burned on a customer who would have stayed anyway, or sequencing a multi-step journey are all
problems where the best action now depends on what it makes possible later. A contextual bandit, which
reads features of the user and situation but still optimizes an immediate reward, is effectively the
one-step case of this larger idea.

## Why full reinforcement learning is rare in marketing

In practice, full online reinforcement learning is uncommon in marketing personalization, for reasons
that are structural rather than temporary.

- **Rewards are sparse and delayed.** The outcome that matters, a purchase or a retained subscription,
  can arrive weeks after the action that influenced it, which makes it hard to credit the right
  decision.
- **The loop is hard to close cleanly.** A firm rarely controls the whole environment a customer moves
  through, and exploring freely on real people is costly and carries real risk.
- **The data is observational.** Logged interactions are not a clean simulator, so a policy learned
  from them inherits the biases of whatever policy produced the logs.

## What is usually sold as reinforcement learning

Much of what is marketed as reinforcement learning is either bandit-like, optimizing an immediate
reward, or trained offline on logged data rather than learning live. Neither is wrong, and both can
work well, but they are narrower than the phrase suggests. The distance between "a model that updates
from outcomes" and "an agent learning a long-horizon policy online" is wide, and the same words cover
both.

## How to test a reinforcement learning claim

Two questions separate a real sequential policy from a relabeled bandit. First, how is the reward
defined: an immediate metric like a click, or a long-horizon value like retention? Second, how often
does the policy actually update: continuously online, or retrained offline on a schedule? If the reward
is immediate and the policy updates in discrete batches, the system is bandit-like whatever it is
called. As with "AI decisioning" and "agentic," treat "reinforcement learning" as a claim to check
against the mechanics, not a capability to assume.

## Where it fits

Reinforcement learning sits at the far end of the decision-logic axis, past rules, scoring, and
bandits. The further along that axis a system goes, the harder it is to measure, because a policy that
changes itself cannot be judged by a one-shot test. That is why sequential policies lean on off-policy
evaluation and persistent holdouts, covered in [how personalization is measured](/measurement). Like
every other method in the [Techniques](/library/techniques) cluster, it is a building block rather than
a destination, and most systems that use it combine it with simpler methods around it.