Wed 7pm ET: Meet the New Bootcamps - Q&A With the Program Designers RSVP Now

Variance-Infused Thinking

By Tony Yiu • August 10, 2020

This post by Data Scientist Tony Yiu is a summary of a longer blog he published on his Medium account, which you can read in full here.

Variance is all around us. It impacts every decision and outcome; but unless we go out of our way to envision, it often passes by unnoticed until it’s too late.

History unfolds linearly - there is ever only one sequence of events that occurred. But in the moment, all the forks in the road (where randomness is included) that led to today could have produced an infinite number of alternate histories. It’s important to recognize this when we evaluate models and processes. 

It’s too easy to focus on the outcome. A happy outcome means our process must be good and a suboptimal one must mean it’s bad. This type of thinking is too simplistic. It fails to recognize the massive impact that randomness and chance often have on our decisions. You should not think twice about accepting a series of bets that pay $100 60% of the time (and you lose $100 40% of the time). But there is a 6% chance that you will find yourself out $300 after losing three times in a row. Does that make you a sucker? No, it just means that you had a run of bad luck. But the decision to accept the bet was and is a sound one.

In my article, we explore how failing to recognize the role of variance in outcomes often blinds us to the true state of the world.

_____

Before transitioning into data science, Tony Yiu spent nine years in the investments industry as a quantitative researcher, where he worked on portfolio optimization, economic simulation, and built numerous forecasting models to predict everything from emerging market equity returns to household spending in retirement. He now works as a data scientist at Solovis, where he uses his experience in statistics, finance, and machine learning to design and build risk analytics software for financial institutions. Tony is also a Metis Bootcamp graduate and we’re excited to have him back with us as a contributor to the blog, where he’ll write about data science and analytics in business and industry


Similar Posts

business resource
Metis Sr. Data Scientist Javed Ahmed Featured in BuiltIn Article

By Metis • September 03, 2020

In a recent article published on BuiltIn, writer Stephen Gossett interviews Metis Sr. Data Scientist Javed Ahmed about the differences between Data Scientists and Data Engineers. Javed breaks down the two distinct roles, helping readers grasp the central difference: design vs. implementation.

business resource
Javed Ahmed Discusses Ethical Risk and Bias at Ai X West 2020

By Shaunna Randolph • October 28, 2020

Ai X West, part of Open Data Science Conference (ODSC) West will be held in a virtual format this year on October 28, and one of our Sr. Data Scientists, Javed Ahmed is all set to present. Here, read what he'll cover and how you can register to watch him live.

business resource
VIDEO: An AI4 Panel Discussion on The State of AI in Banking

By Metis • September 23, 2020

Metis Sr. Data Scientist Javed Ahmed recently took part in a panel discussion about The State of AI in Banking during an online Ai4 event. He and the other panelists talked about upskilling, challenges related to COVID-19, and more. Watch the recorded panel discussion here.