NEW Python for Beginners Course Starts July 27! View Course

Extracting Value from Untapped Data

By Javed Ahmed • January 24, 2020

Photo by Joshua Sortino via Unsplash

This post was written by Javed Ahmed, Metis Sr. Data Scientist, Corporate Training

Are you using the right data for decision-making? Over the last few years, data generated per second has increased by 700% – and some 90% of all data has been created in just the last two years alone, presenting an ongoing challenge for analytics. While ample evidence suggests that better analytics lead to better performance, firms often limit their sources of data to just a few or avoid using external data entirely, even as they increase overall investment in analytics. 

This under-utilization of data is a key reason why most companies fall short in efforts to scale the impact of analytics and mitigate potential risks. A survey of CFOs suggests that while they believe data availability drives competitive advantage, they are often required to rely on inaccurate or incomplete data. Another recent study finds that introducing new types of data can “disrupt industries” and lead to significant improvements in performance. Incorporating new data sources represents a major opportunity. 

Why do so many miss out? The best way to incorporate data is usually not straightforward, and levels of data literacy vary both across and within firms. While managers overwhelmingly want to improve data usage, they are often limited by several factors:

  • - Availability of necessary information 
  • - (Perceived) low-quality data
  • - Ability to adapt existing tools 
  • - Lack of data literacy across functions and roles
  • - Risks to consumer privacy

Photo by Glenn Carstens Peters via Unsplash

Balancing Promise with Privacy

Even in industries where actively consuming external data is the norm, traditional approaches may need reevaluation in light of expanding sources of information. In the financial sector, both consumer and business lenders rely primarily on credit ratings in lieu of more nuanced information. As more alternative information emerges, leveraging it with machine learning approaches looks increasingly attractive. 

In 2005, Will Lansing, CEO of FICO, suggested that information from social media may be useful in credit scoring, stating: “If you look at how many times a person says ‘wasted’ in their profile, it has some value in predicting whether they’re going to repay their debt”.  However, consumer credit agencies have been discouraged from incorporating information from social media due to significant privacy concerns. New York state recently passed legislation preventing consumer credit firms from even looking at data from social media. In 2016, the Federal Reserve warned fintech lenders that information from nontraditional sources could be indirectly correlated with protected characteristics. 

An Opening for Lenders 

While the potential of untapped data has been dampened by concerns about privacy and discrimination, there are signs things are changing. Citing the potential to increase access to credit for the unbanked, major financial regulators came together last month to suggest alternative data may now be allowed and encouraged, despite some of the well-known risks. Lenders can now look at information relevant to predicting cash flow, which has the potential to benefit those who might otherwise be denied credit. However, they warn such data could require significant risks to consumer protection and require “robust compliance management.” 

While the implications are substantial for banks and fintech lenders, compliance and consumer privacy present challenges more broadly. Across industries, high-performing firms are twice as likely to actively work to mitigate privacy-related risks. As the scope for applying approaches using new data expands, so does the responsibility to ensure that consumers are protected. 

Source: McKinsey & Company

How can firms successfully leverage the value of untapped data? There is no quick fix. Even low-cost data requires managers to know how and where to use it. When some do and some don’t, coordinating a strategy is harder. To overcome gaps in data literacy, high-performing firms invest in analytics ‘translators’ who can bridge knowledge gaps across employees. This suggests Investments in data literacy and human capital, as well as the involvement of senior management in analytics (characteristics already linked with better performance), will critically determine the next set of winners.


Metis Corporate Training specializes in training teams on topics of data literacy, advanced analytics, and data science. Our courses are conducted through a real-world lens with immersive and hands-on engagements that can scale to fit the needs of any organization. 

Similar Posts

business resource
VIDEO: Metis Chief Data Scientist Discusses The Making of a Data Scientist

By Metis • June 16, 2020

In the most recent episode of Kaplan's Bold Leaders in Learning, host Brandon Busteed interviews Metis Chief Data Scientist about The Making of a Data Scientist. Watch it in full here.

business resource
IIA + Burtch Works COVID-19 Impact Analysis Survey Results

By Metis • June 11, 2020

Over the past few months, Burtch Works has teamed up with the International Institute for Analytics (IIA) to survey quantitative professionals about the ongoing impacts of the COVID-19 pandemic on analytics teams across the country. The most recent wave of results was published this week in a post titled, Have COVID-19 Analytics Impacts Peaked? Read about it here.

business resource
Understanding Natural Language Processing

By Tony Yiu • May 22, 2020

Natural language processing (NLP) is one of the trendier areas of data science. Its end applications are many – chatbots, recommender systems, search, virtual assistants, etc. In this post, Data Scientist Tony Yiu helps readers understand the basics of NLP, including why it's important and how it helps us.