A/B Testing 101: What is an A/B Test?

From emails, to social media, to UX/UI, to lead gen, and more, Marketing Managers have a great bird's-eye view into a customer's journey from acquisition to registration. As a Marketing Manager that works for a fast-paced startup, I get to have a hand in all marketing campaigns and initiatives.

For almost three years I have gotten to know hundreds of attendees at our bootcamp. I get to put a face to the name, a firm handshake to that hotjar recording, and ____ to those email opens. With over 4,000 students, I've been able to understand the customer's journey from the time they give us a call inquiring about the bootcamp to enrolling for the course. However, like most , a customer's needs and wants are ever changing and it's important to keep up with

A dilemma that businesses often face is that they think they know their customer, but often times, you would be surprised that a customer can behave much differently than you would think.And when running an experiment or A/B test, you might find that the results can often be very humbling and customers can behave much different than you would think.

Sure, some things are predictable. Maybe a barista knows Jane's coffee order when she comes in every day at 3:00pm sharp and have it waiting on the counter as they watch her walking in from the parking lot. But what if Jane started a diet and couldn't order her venti java chip frappuccino or what if Jane found out she were pregnant and gave up caffeine? Businesses need to cater to the changing needs and wants of their customers, keeping in mind, that one size does not fit all.

Especially in the digital world

After reading this post, you will effectively be able to conduct a simple A/B test and online experiment.

What are some ways you can perform an A/B test and online experiment?

When deciding what to run an experiment on, you must first decide on which problem you want to solve. Do you want to increase the number of page views, session duration, click-through-rate, or form submissions? You must first call out your criteria for success before running your experiment. Below are some examples on what you can experiment on.

Email marketing

  • Email subject line
  • Call-to-action (CTA)

Website redesign

  • Main content body
  • Videos, images, rich content
  • Submission forms


  • Text paragraphs, bullets, lists
  • Headlines
  • Short copy versus long copy

Funnels and Navigation


  • Length of the forms
  • Special offer, discount, or promotion

And more! Check out this article for more ideas on things to A/B test.

Steps -

Watch this series on A/B Testing in Minutes.

Intro - about company, about metrics Dilemma/Problem What is an A/B test - High Level What can you test- ideas and suggestions Steps in experimentation Plot Twist - bet you didn't know

Meta Headers Social Media cards url anchor text infographic

Plot Twist- You're part of an experiment!

Now, you really didn't think I would write all of this and explain how an A/B test works without running an experiment on this post, did you?

Yep, that's right! You are officially a part of an online experiment. Do you think you can guess which part of this blog I'm experimenting with?

  • H5P rich content. Yes/No or multiple choice

This is a companion discussion topic for the original entry at https://blog.datasciencedojo.com/p/b4680a85-aedb-47ad-9b23-f0e494028e13/