MONDAY: Final Application Deadline for Spring Bootcamp! Apply Now

What is a Monte Carlo Simulation (Part 5)

By Zachariah Miller • March 01, 2018

This is the 5th and final installment of a series by Metis Sr. Data Scientist Zach Miller dedicated to investigating how Monte Carlo can be a great tool. Part 1 introduces the concept of Monte Carlo, Part 2 teaches us how to do MC in Python, Part 3 explores how we can use it to (attempt to) outplay the odds in video poker, and Part 4 attacks the problem of trying to do particle physics simulations with Monte Carlo If you haven't read those yet, start there!

Using Monte Carlo to Measure Business Impact

In all the previous parts of this series on Monte Carlo techniques, we've focused on the techniques and how we can use them in sort of niche settings. However, in part 1, I mentioned that this is a really powerful technique for real-world applications. Today, we'll focus on one such case. Let's imagine that your company pays a consulting group to come in and optimize your website for audience retention. They run their analysis and proudly announce that they've found a way to direct 10% more traffic from your frequently asked questions page back to your main website. Essentially, cutting your bounce rate from that page by 10%. That's amazing, right? A retention upgrade of 10% is a solid step up, and much larger than most businesses will ever see. The consulting firm charges you and arm-and-a-leg for this analysis and result. Was it worth it?

Your initial instinct is that, well as long as their rate was less than 10% of my revenue, I come out on top. But it's not so clear. They've only optimized one page, and they haven't optimized on sales but on audience retention. How does this actually change your bottom line overall? Is it a 10% increase? Is it even a noticeable increase at all? How can you convert their optimization back to monetary gain? We're going to do it using Monte Carlo techniques in this notebook on Measuring Business Impact with MC

So what did we learn?

In this example, we took all of the knowledge we gained from the previous entries in the series and applied it to a real-world problem (albeit with a faux dataset). We saw that by applying this technique, we can convert between customer behavior and revenue, accounting for all the different ways that a customer might interact with our website. At the end of the day, we found that even though there was a 10% increase in audience retention, we only see a roughly 2% shift in the mean, and that result might not even be statistically significant compared to the randomness inherent in audience reactions. Monte Carlo let us go from, "WOW a 10% increase" to "Hmm, I don't know if that was worthwhile for our business."

If you haven't already, read Part 1, Part 2, Part 3, and Part 4 of Zach's series. 

Similar Posts

data science
As Data Science Grows in Colombia, Grad’s Career Flourishes

By Emily Wilson • February 13, 2020

Bootcamp graduate Mónica Ramirez was well-positioned when the data science boom hit Colombia, where she was born and raised. Read how her experience at our New York City bootcamp helped further her career at home.

data science
How to Gather Data from YouTube

By Kimberly Fessel • November 11, 2019

In this post, learn how you can gain access to three types of YouTube data: the videos themselves for use in computer vision tasks, the video transcripts for natural language processing (NLP), and video search results for hybrid machine learning efforts.

data science
Made at Metis: Clustering NBA Playstyles Using Machine Learning; Automatic Pricing on Etsy

By Emily Wilson • November 04, 2019

This post features two final projects from recent graduates of our data science bootcamp. Take a look at what's possible to create in just 12 weeks, including these projects on Clustering NBA Playstyles Using Machine Learning and Automatic Pricing for Etsy Sellers.