Demystifying Data Science Free Online Conference is July 30-31! Register Now

What is a Monte Carlo Simulation (Part 5)

By Zachariah Miller • March 01, 2018

This is the 5th and final installment of a series by Metis Sr. Data Scientist Zach Miller dedicated to investigating how Monte Carlo can be a great tool. Part 1 introduces the concept of Monte Carlo, Part 2 teaches us how to do MC in Python, Part 3 explores how we can use it to (attempt to) outplay the odds in video poker, and Part 4 attacks the problem of trying to do particle physics simulations with Monte Carlo If you haven't read those yet, start there!

Using Monte Carlo to Measure Business Impact

In all the previous parts of this series on Monte Carlo techniques, we've focused on the techniques and how we can use them in sort of niche settings. However, in part 1, I mentioned that this is a really powerful technique for real-world applications. Today, we'll focus on one such case. Let's imagine that your company pays a consulting group to come in and optimize your website for audience retention. They run their analysis and proudly announce that they've found a way to direct 10% more traffic from your frequently asked questions page back to your main website. Essentially, cutting your bounce rate from that page by 10%. That's amazing, right? A retention upgrade of 10% is a solid step up, and much larger than most businesses will ever see. The consulting firm charges you and arm-and-a-leg for this analysis and result. Was it worth it?



Your initial instinct is that, well as long as their rate was less than 10% of my revenue, I come out on top. But it's not so clear. They've only optimized one page, and they haven't optimized on sales but on audience retention. How does this actually change your bottom line overall? Is it a 10% increase? Is it even a noticeable increase at all? How can you convert their optimization back to monetary gain? We're going to do it using Monte Carlo techniques in this notebook on Measuring Business Impact with MC

So what did we learn?

In this example, we took all of the knowledge we gained from the previous entries in the series and applied it to a real-world problem (albeit with a faux dataset). We saw that by applying this technique, we can convert between customer behavior and revenue, accounting for all the different ways that a customer might interact with our website. At the end of the day, we found that even though there was a 10% increase in audience retention, we only see a roughly 2% shift in the mean, and that result might not even be statistically significant compared to the randomness inherent in audience reactions. Monte Carlo let us go from, "WOW a 10% increase" to "Hmm, I don't know if that was worthwhile for our business."
____

If you haven't already, read Part 1, Part 2, Part 3, and Part 4 of Zach's series. 


Similar Posts

data science
New: Advancing Women in Data Science Scholarship in Seattle & Chicago

By Metis • July 12, 2019

Our brand new Advancing Women in Data Science Scholarship is for women, trans, and non-binary aspiring data scientists who wish to attend our Data Science Bootcamp in either Seattle or Chicago. There are two full-tuition scholarships available in each city for the Fall 2019 bootcamp - apply by July 29th!

data science
Made at Metis: Recommendation Systems for Making Meals + Choosing Beer

By Emily Wilson • July 08, 2019

This post features two final projects from recent graduates of our data science bootcamp. Take a look at what's possible to create in just 12 weeks, including a meal recommendation app based on the produce you have at home + a recommendation engine to help you choose your next favorite beer.

data science
Sr. Data Scientist Roundup: Effects of AI Bias, Impact Hypothesis, and Tips for Hiring Your Data Team

By Emily Wilson • March 26, 2019

When our Sr. Data Scientists aren't teaching the intensive, 12-week bootcamps or corporate training courses, they're working on a variety of other projects. This monthly blog series tracks and discusses some of their recent activities and accomplishments.