Biggest Obstacles to Big Data Are Right In Front of Us

Big data analytics is a big thing. It also plays a huge role in artificial intelligence (AI). Deep learning requires analyzing a lot of data. Yet, big obstacles to big data still thrive.

The biggest obstacles to big data don’t deal with the data at all.

Source of Almost All Obstacles to Big Data

Luckily, it’s easy to spot these obstacles. They’re right in front of us. The biggest obstacles to big data are humans. They take on two forms in them:

1. Biases that prevent action on big data’s conclusions
2. Ignorance about statistics that prevents putting those conclusions in the right light

First, old habits die hard. Change is hard. Biases get in the way. These intensify if vested interests are at stake.

Second, only 9% of the U.S. adult population has the numeracy to understand statistics. It’s more than just reading data. It includes knowing how to work that data into decision making.

What Happens When Big Data Tells People They’re Wrong?

As an example look at American football. Coaches still make the same decisions even though data shows they’re unsound.

Big data, especially in the form of AI, will shake up decision making. The game of Go is an example. The world champion summed up this impact on Go’s 2500-year traditions as, “. . . computers now tell us humans are all wrong.”

So, what will happen when data tell executives and internal experts that they’re pet projects are no good? They’ll just give in?

What Happens When People Don’t Look at Big Data in the Right Light?

Statistics help us analyze data. All data come with a degree of uncertainty. Statistics measures it in many ways. Correlation and probability are but two.

Ignorance of this uncertainty leads to unsound decisions. That’s because decision makers often don’t look at data in the right light.

For instance, it’s not unusual for people to treat an 85% chance as though it were 100%. Look no further than the 2016 U.S. presidential election.

As a result the hedges, contingencies, options, planning scenarios and others that should show up in decisions, won’t. Ignorance will cause decision makers to overlook or underweight these.

How Big Data Will Disappoint Businesses

In the end, it’s very likely big data will disappoint. These obstacles to big data will show up as two general themes in businesses:

1. Implementation of findings won’t occur or won’t to the degree they need to be.
2. Results from decisions will fall short of expectations.

In short, the hype of big data will discredit big data. As a result, the true value of big data will take much longer to materialize than it should.