There has been a lot of hype about the promises and potential of big data, but is it all hype, or is there substance behind the hype?

The last couple of years have seen huge leaps in data and analytics capabilities.

Today, there is more data than has ever been generated before.

Actually, with billions of devices and gadgets such as smartphones, wireless sensors, cameras, payments systems, digital platforms, and virtual reality applications generating data every single moment, the volume of data generated increases by 100% every 3 years.

Computational power has grown exponentially, storage capacity doubles every few years while costs are plummeting, and more and more sophisticated algorithms are constantly being developed.

The merging of these trends is creating opportunities for disruption of business models and whole industries.

Source: Dream Creation

Amount of data generated every 60 seconds. Source: Dream Creation

Some companies are already taking advantage of these trends and combining them with new, unconventional mindsets to tackle business problems in a totally new way, in many cases causing disruption and catching incumbents by surprise.

These companies, most of which are digital natives, have a great advantage, and the only way incumbents can keep up with them is by figuring out how they can transform their fundamental business by applying data and analytics.

To remain competitive, legacy organizations needs a two pronged approach.

First, they should focus on coming up with high risk, high reward strategies that will help them tap into new sources of revenue.

Such high risk, high reward strategies include developing new business models and entering new markets.

Second, these organizations should find ways of applying data analytics to identify insights to help them improve their core operations.

Organizations that are able to implement this two pronged approach will be well primed to take advantage of new opportunities and protect themselves from unexpected disruptions.

According to a report by the McKinsey Global Institute, data and analytics presents a wide range of opportunities for business, and as technologies surrounding big data continue making advances, we can expect that the potential applications and opportunities of data and analytics will continue growing.

Below, let’s take a look at some of the key insights from the report.


It is evident that we live in a world full of data. Whenever you browse the internet, interact with friends on social media, buy something online, use a taxi hailing application like Uber or Lyft, or use virtual assistants like Siri, Cortana or Alexa, you leave behind a treasure trove of data.

Unfortunately, while there is plenty of data in today’s world, companies have had a hard time using this data to drive their strategies.

A 2011 report by McKinsey looked at the potential for big data and analytics and determined that they would have the greatest impact on five major domains.

Looking at these domains today, it is evident that while progress has been made, most companies have only captured a fraction of the potential value of big data and analytics in these domains.

There is a great deal of value that is still unexploited.

The five domains are:

  • Location based services: The integration of GPS technology into smartphones has made mapping technology available to billions of people around the world. This has in turn created huge opportunities for businesses to offer services that rely on this technology. However, a lot of these opportunities remain untapped. For instance, the markets for geo-targeted mobile advertising services, location-based service applications and GPS-based navigation services and devices has only reached between 50% and 60% of the value the 2011 McKinsey report envisioned. The greatest value of location based services has gone to end consumers, mostly in the form of time and fuel savings. However, there are still opportunities for businesses to make use of location based services and data to gain new insights about their operations and improve efficiency.
  • US Retail: With retail having become highly digital, retailers have access to lots of behavioral and transactional-based data about their customers. With margins in the sector becoming thinner by the day, and with competition from digital native companies such as Amazon, there is strong incentive for retailers to mine this data and use it to find ways to improve their bottom lines. In this regard, data can give insights on almost every aspect of business, from how retailers can upsell and cross-sell to their customers to how they can optimize the entire value chain to reduce costs. As of today, only about 30% – 40% of the value envisioned by the 2011 McKinsey report has been captured by the US retail sector. And once again, end consumers have gained the greatest value of big data and analytics in the retail sector.
  • Manufacturing: The manufacturing industry has been very slow when it comes to taking advantage of the potential value of big data and analytics. Of the potential for big data and analytics that McKinsey envisioned in 2011, only about 20% – 30% has been achieved. In addition, most of the gains have only been made by a few industry leaders. Some of the main applications of big data and analytics in the manufacturing sector include design to value applications, development of digital factories, operation analytics driven by sensor data, and improved after-sales services that are reliant on predictive maintenance and real-time surveillance.
  • The EU Public Sector: In the 2011 report, McKinsey looked at how big data and analytics could be used to improve the delivery of public sector services in the European Union. The report determined that big data and analytics could improve efficiency in the delivery of government services, improve collection of taxes, reduce errors in payments transfer and potentially put an end to public sector fraud. This would potentially result in annual savings to the tune of €250 billion. Unfortunately, only about 10% – 20% of this value has been realized.
  • US Healthcare: In their 2011 report, McKinsey identified that there was huge potential for data and analytics in the healthcare sector in the United States. Today, however, only about 10% – 20% of this sector has been realized. This low uptake of big data and analytics in the US healthcare industry can be attributed to a number of challenges, including shortage of technical talent, organizations being averse to change, lack of incentives, as well as regulation challenges. Still, some progress has been made in the sector. The greatest progress has been made in the shift from manual to electronic medical records, though much of the data that lies within these records is yet to be fully utilized. Other current applications of big data and analytics in the health sector include applications in medical research and development, public health surveillance, and predictive medicine. Despite these applications, a lot of opportunities within this sector remain unutilized.


The less-than-ideal realization of the potential value of data and analytics in the domains described above should not be taken to mean that companies are slow in the uptake of big data and analytics.

On the contrary, many companies have already began deploying data and analytics.

Unfortunately, the manner in which they are doing it is preventing them from realizing its full potential. While many companies have already made huge investments in technologies that will help them deploy data and analytics, most of them have neglected the organizational changes that need to accompany these technology investments.

An effective analytics transformation strategy involves more than just huge technology investments. First, organizations need to ask themselves some key questions that will help define their strategic vision.

These questions include: How are we going to use data and analytics? How will we turn the insights gleaned from data and analytics into value? How do we measure the value generated from data and analytics?

The second thing organizations need to do is to build the underlying architecture that will support the generation and collection of data.

Moving from legacy data systems to more flexible and agile systems that support big data is actually one of the biggest challenges organizations face when undergoing an analytics transformation.

To make the most of data and analytics, organizations also need to digitize their operations.

Digitizing operations will make it much easier to capture data that they can then use to streamline these operations.

In addition, data collection is not enough. Organizations also need to obtain the analytical capabilities they will need in order to glean useful insights from the collected data.

On this front, organizations have two options. They can either outsource analytics to external specialists or build their own in-house analytical capabilities.

Once insights have been derived from data, these insights need to be incorporated into the actual workflow for them to make any impact.

This calls for a transformation of business processes, which is usually a huge challenge for many organizations.

For insights to be turned into meaningful impact, they need to be made accessible to the right personnel. These personnel also need to be empowered to make decisions based on data insights.

Legacy organizations have to make all these changes if they are to make the most of big data and analytics.

Ignoring any of the above changes keeps organizations from unlocking the full potential of data and analytics and makes them vulnerable to disruption.


Another major challenge that has kept companies from taking full advantage of the potential of big data and analytics is the shortage of analytics talent.

According to a survey by McKinsey & Company, about 50% of executives claim that it is more challenging to recruit talent for analytical roles compared to recruiting for any other role.

In addition, 40% of executives also report that retaining analytical talent is a huge challenge.

The shortage of analytical talent is particularly evident when it comes to data scientists. In their 2011 report, McKinsey had already predicted that it would get to a point where demand for data scientists would exceed supply. We have already gotten to this point.

The high demand for data scientists is evident in the fact that the average wages for data scientists increased by roughly 16% per year for the between 2012 and 2014, according to a report by Indeed.

This is considerably high compared to the 2% average annual increase in wages for all occupations.

The shortage of analytics talent is unlikely to end soon.

While more schools are adding data science programs and producing a greater number of data science graduates every year, the demand for data scientists is growing at an even greater rate.

However, there is still some hope.

Advances in AI and machine learning technology might make it possible to automate data preparation, which makes up over 50% of data analytics work. There is a chance that automation of data preparation might ease the demand for data scientists.

Organizations also need to realize that simply recruiting analytics talent will not enable analytics transformation by itself.

To enable analytics transformation, organizations also need business translators whose role is to act as a connection between the analytical talent and the organization’s business needs.

The business translator needs to have good knowledge of data science work, as well as a functional knowledge and expertise in the industry in which the organization operates.

This makes it possible for them to ask the right questions to the analytics team and help them derive insights that can actually be used to optimize business operations.

While organizations can outsource analytics capabilities, the business translator role needs to be developed from within the organization.


Already, a huge gap has developed between the average company and the relatively small group of companies who are leading on the analytics front – and the leaders are consolidating big advantages, with some even enjoying the winner-takes-it-all dynamics.

Think of some relatively new businesses whose entire business models are centered on data and analytics, such as Airbnb, BlaBlaCar, Didi Chuxing, DJI, Flipkart, Lyft, Pinterest, Snapchat, Spotify, and Uber.

Most of these companies disrupted their respective industries and hold the greatest market share mainly because of their data and analytics assets.

It is important to realize that we are in a new era.

While assets such as factories and equipment were a huge factor in competition a few decades ago, what matters most today is assets such as data, digital platforms, and analytical talent.

These assets are making it possible for new players to by-pass traditional barriers to entry and get into new markets surprisingly fast.

For instance, Amazon disrupted the whole retail sector on its own without having to build any stores.

Companies like Airbnb have revolutionized the hospitality industry without having to build any hotels, while Uber and Lyft disrupted the transport industry without having to buy any vehicles.

However, it is also good to note that some of these digital native companies have started putting up their own barriers to entry to keep out other players.

Companies with huge digital platforms are already enjoying network effects that are making it hard for other companies to enter these markets.

Others have access to a wide range of data and insights that give them very huge advantages over other players who might be interested in competing with them.

What’s more, with their data and insights, these leading firms have the ability to enter new industries with surprising ease.

For instance, using data and digital assets, Google is set to revolutionize the automotive industry with their autonomous cars, while Apple has made inroads into the financial sector with Apple Pay and Apple Card.

Data has become so important an asset that companies are willing to provide free services to customers in exchange for data, as is happening with most social media companies.


We have seen that data has become an important corporate asset, one that is driving disruptions across various industries.

However, in itself, data is not very valuable. Its value arises from how this data is used.

A piece of data may be useless to one company, yet the same piece of data can help another company come up with a new product, or even a new business model.

It is important for organizations to realize that not all data is created is equal.

There are very many categories of data, including behavioral data, geospatial data, environmental data, transactional data, public records, structured and unstructured data, and so on.

To make the most of data, organizations need to first consider how they can use these different types of data, and then determine which kind of data holds the most potential for them.

Once they figure this out, they can then come up with systems that will help the collect this type of data.


Some markets are more vulnerable to disruption by data-driven models than others, based on their characteristics.

Some of the characteristics that indicate that a market could be potentially disrupted include:

  • Industries where inefficient signaling results in underutilization of assets
  • Industries where there is a mismatch between supply and demand
  • Industries that depend on large amounts of personalized data
  • Industries where available data is fragmented or siloed.
  • Industries where bringing together data from multiple sources can create huge value
  • Industries whose core business model is centered on R&D
  • Industries where decision making could be affected by human biases
  • Industries where human constraints might limit the speed of decision making
  • Industries that stand to gain a lot by improving the accuracy of prediction

The above characteristics set the stage for disruption that is driven by one of six data-based business models. These models include:

Business models enabled by orthogonal data: As the world becomes more data-oriented, new kinds of data from new sources will keep coming up, and these new kinds of data can be applied to all kinds of problems. New players who apply these new kinds of data will disrupt incumbents who have grown comfortable making their decisions based on a single kind of standardized data. Some industries that are vulnerable to disruption by business models enabled by orthogonal data include insurance, healthcare, and talent/human capital management.

Hyperscale, real-time matching: Digital platforms have made it possible to match supply and demand of all kinds of products and services in real-time. Some of the industries that are vulnerable to disruption by hyperscale, real-time matching technologies include transportation and logistics, automotive, hospitality, and smart cities and infrastructure.

Radical personalization: As data continues being generated from all sorts of digital interactions, this data will make it possible to create very fine distinctions between different groups of people. These distinctions can then be used to micro-segment markets and personalize products and services at unprecedented scales. Some industries that could be disrupted by radical personalization include education, healthcare, retail, advertising, media, and travel and leisure.

Massive data integration capabilities: Today, we already have several kinds of data from multiple sources. However, this data is usually fragmented or siloed. Fortunately, technological advances are making it possible to break these silos and link different kinds of data from multiple sources. This integration of different kinds of data will allow organizations to derive even more insights from unrelated data, opening up a lot of potential value. Some industries that could be disrupted by massive data integration capabilities include insurance, banking, and the public sector.

Data-driven discovery: Previously, innovation was driven by creativity and human ingenuity. As we gain access to more and more data, however, this data and insights gleaned from the data will come in handy in supporting and enhancing human ingenuity to drive innovation. Some industries that could potentially be transformed include material sciences, life sciences, pharmaceuticals, and technology.

Enhanced decision making: Human decision making is usually influenced by things such as our biases and our inability to hold and process huge amounts of information. Data analytics and algorithms will allow us to make faster, better and more accurate decisions by eliminating the biases and limitations that influence our decision making. Some industries that will be potentially transformed by this enhanced decision making capabilities include insurance, healthcare, smart cities, and talent/human capital management.


Today, there is a lot of hype about the potential of data and analytics.

While companies and organizations have been using data and analytics for a while now, the potential to create value from big data is greater today than it was ten years ago.

Any organization that wants to survive and remain today competitive today has no other option but to start harnessing the capabilities of data to differentiate themselves, create value and optimize its operations.

The Age of Analytics: Competing in a Data-Driven World

Comments are closed.