A/B Testing

A Refresher on A/B Testing - by Amy Gallo. “It’s clear that A/B testing is not a panacea. There are more complex kinds of experiments that are more efficient and will give you more reliable data, Fung says. But A/B testing is a great way to gain a quick understanding of a question you have. And “the good news about the A/B testing world is that everything happens so quickly, so if you run it and it doesn’t work, you can try something else. You can always flip back to the old tactic.””
A/B Testing - You’re Doing It Wrong - by Justin Baker. “Effective A/B tests are about problem solving — driving long-lasting, positive value for your customers.”
An Ultimate Guide to A/B Testing on Prototypes. - by CanvasFlip. “Conversion funnels on CanvasFlip might help you understand the drop-off ratios and conversion rates. This goes a long way in determining which area needs immediate attention.”
Experiments at Airbnb - by Jan Overgoor. “Controlled experiments are a great way to inform decisions around product development. Hopefully, the lessons in this post will help prevent some common A/B testing errors.”
From Power Calculations to P-Values: A/B Testing at Stack Overflow - by Julia Silge. “Dealing with data means becoming comfortable with uncertainty, and A/B tests make this reality extremely apparent. Handling uncertainty wisely and using statistical tools like A/B tests well can give us the ability to make better decisions.”
How A/B Testing at LinkedIn, Wealthfront and eBay Made Me a Better Manager - by First Round Review. “In this interview, Shmukler shows why he uses A/B testing as a management framework, illustrating how it works to not only accelerate decisions, but also empower the teams making them. He outlines the framework’s benefits and challenges, as well as how to implement and scale it a startup. Any product or growth leader will learn from his data-driven approach to product and team management.”
How Netflix Does A/B Testing - by Jessie Chen. “Have you ever wondered why Netflix has such a great streaming experience? Do you want to learn how they completed their homepage plus other UI layout redesigns through A/B testing? If so, then this article is for you!”
How We Lost (and Found) Millions by Not A/B Testing - by Noah Lorang. “This is the story of how we made a change to the Basecamp.com site that ended up costing us millions of dollars, how we found our way back from that, and what we learned in the process.”
It’s All A/Bout Testing: The Netflix Experimentation Platform - by Steve Urban, Rangarajan Sreenivasan and Vineet Kannan. “In this post we’re going to discuss the Experimentation Platform: the service which makes it possible for every Netflix engineering team to implement their A/B tests with the support of a specialized engineering team. We’ll start by setting some high level context around A/B testing before covering the architecture of our current platform and how other services interact with it to bring an A/B test to life.”
Make Product Decisions Without Doubt — My Lessons from Twitter and Slack - by First Round Review. “A hypothesis tree helps dissect doubts. It’s simply a structure for an overarching thesis and supporting points.”
Resources to Evolve Your A/B Testing Skills - by Cara Harshman
Stepping up Your A/B Tests - by Kevin Shanahan. “This is the first of two posts that deep dive on A/B testing, expanding on a talk I gave at Google Playtime 2016 in London. In this post I share some of the learnings we’ve had after running 60+ A/B tests at Peak, looking at each step of the A/B testing cycle in turn.”
The Ultimate Guide To A/B Testing - by Paras Chopra. “This article is meant to be the best guide you will ever need for A/B testing.”
The Tenets of A/B Testing from Duolingo’s Master Growth Hacker - by First Round Review. “In this exclusive interview, Gotthilf examines the four A/B tests that have been most crucial to that growth. She shares the lessons and cautionary tales she’s gathered from each experiment — and the tenets born from them. Let’s begin.”