Amazon Payments deployed a multi-objective contextual bandit system to optimize personalization across an entire product acquisition funnel (AWS Machine Learning Blog). The approach uses Linear UCB (LinUCB), a contextual multi-armed bandit algorithm that conditions content selection on behavioral signals—payment history, transaction patterns, and similar features—rather than static segments. In a seven-week online A/B test, one customer population saw a high single-digit percentage relative lift in final-funnel conversion, while another showed no improvement, revealing that content quality, not the model, was the limiting factor (AWS Machine Learning Blog).
The system addresses what Amazon calls the "seesaw problem": optimizing one funnel stage in isolation can harm another, and optimizing only for the final outcome starves the model of signal because approvals are rare and delayed. Amazon's solution runs one LinUCB model per stage (start, submit, approve) and combines their Upper Confidence Bound scores through a weighted linear combination, allowing the system to balance exploration and exploitation across the entire customer journey simultaneously (AWS Machine Learning Blog). For commerce practitioners, this approach is significant because it shifts the paradigm from traditional A/B testing toward continuous learning systems that adapt as generative AI accelerates content variation, and it shows how bandits can handle multi-stage outcomes without waiting for tests to conclude.
Amazon built oversight into scale by vetting individual content building blocks (images and taglines) rather than reviewing every possible combination, then composing them into a large arm pool. The repository and code samples are publicly available on GitHub, enabling other teams to test the approach on synthetic data and adapt it to their own funnel architectures (AWS Machine Learning Blog).