Learn How To Define, Start, And Analyze The Results Of An A/B Test. Improve Business Performance Through A/B Testing
A/B testing is a tool that helps companies make reliable decisions based on data.
This is one of the fundamental skills you need to land a job as a data scientist or data analyst.
Do you want to become a data scientist or a data analyst?
If you do, this is the perfect course for you!
Your instructor Anastasia is a senior data scientist working at a Stockholm-based music streaming startup. She has earned two Master’s degrees in Business Intelligence and Computer Science, and grown from a recent graduate to a Senior role in just 3 years. Anastasia has performed a significant number of A/B tests for large tech companies with hundreds of millions monthly users.
By taking this course, you will learn how to:
- Define an A/B test
- Start an A/B test
- Analyse the results of an A/B test on your own
Along your learning journey Anastasia will walk you through an A/B testing process for a fictional company with a digital product. This case study unfolds throughout the course and touches on everything from the very beginning of the A/B testing process to the very end including some advanced considerations. Moreover, Anastasia takes some time to share with you her advice on how to prepare for the questions on the A/B test interview for a data scientist or data analyst position.
One strong point of differentiation from statistical textbooks and theoretical trainings is that these learning materials will teach you how to design A/B tests for digital products that have millions or hundreds of millions of users. It is a rare overview of the A/B testing process from a business, technical, and data analysis perspective.
This is the perfect course for you if you are:
– a data science student who wants to learn one of the fundamental skills needed on the job
– junior data scientists with no experience with A/B testing
– software developers and product managers who want to learn how to run A/B tests in their company to improve the product they are building
You will learn an invaluable skill that can transform a company’s business (and your career along the way).
What you’ll learn
- How to use A/B tests to improve business performance
- Define A/B tests
- Start A/B tests
- Analyze the results of A/B tests
- Measure the success of A/B tests
- How to define a hypothesis
- Design tracking for the metrics
- How to prepare for a data science interview (when you get asked about A/B tests)
- How to design A/B tests for digital products
- Advanced considerations when you run multiple A/B tests at the same time
Table of Contents
Introduction Welcome to the course on AB testing
1 Welcome to the course!
2 What does the course cover and what is AB testing
3 What are the key characteristics of an AB test
4 How AB tests are created Who takes part
Defining KPIs and metrics. Practical example Kittengram
5 How to measure success
6 Calculation of metrics for Kittengram – Practical example
Setting up and executing AB tests in practice
7 Data instrumentation and tracking
8 Calculating metrics
9 Designing the experiment
10 Set up the AB test
11 Statistical significance
12 Calculate the sample size of an AB test
13 Example of significance power calculator
14 AB test – start & analysis
15 Presenting results
16 AB test analysis process
17 Compare the activity between the groups
18 Advanced AB testing considerations
19 Important ethical considerations Privacy
Advanced questions for interview preparation
21 Question 1
22 Question 2
23 Question 3
24 Question 4
25 Question 5
26 How to prepare for the interview
27 Final recommendations