People, not just Hardware, Determine the Future of Big Data

September 4th, 2013 / By Jeremy Zasowski

I recently read a blog post about commercializing big data in healthcare that listed some very interesting figures:

  •  90 percent of the world’s data is less than two years old
  •  Total data collected will grow by 40% next year
  •  Per IBM’s estimates, 2.5 quintillion bytes of new data is generated each day (a quintillion is 1018, or 10 followed by 18 zeros)

Now that is a lot of data. Digital pieces (bits and bytes) of information, stored on servers, just waiting for someone to make sense of it and do something useful with it. When you get this much data, we get creative and call it “Big Data.” Some industries are already starting to use the Big Data that they are gathering to benefit themselves and their customers. Think of financial services, insurance and retail.

All of these industries are heavily investing in IT, from both a hardware and software perspective, but also in staffing. They are all hiring a lot of IT professionals and computer programmers, and they’ve been doing it for years now. This has laid the groundwork for how to collect and store all of this data. To make sense of Big Data, these industries have started hiring mathematicians and natural language processing (NLP) experts.

Consider Google’s ability to predict what you want to search for after typing in just a few letters into their search bar. Or think of Amazon’s ability to suggest items you might be interested in based on your past search and buying history, or financial firms abilities to predict stock price movements. All of these examples involve being able to analyze large amounts of data in order to find trends and essentially start to predict future outcomes. That’s not simple work and takes a lot of complex hardware, software, and analytical know-how.

Traditionally healthcare has been slow to adopt new technologies; just look at how many hospitals and healthcare organizations are still writing in paper charts. Now granted, EHR adoption is accelerating but this is just laying the groundwork for digitization. Healthcare is on the cusp of starting to collect more and more electronic data. The next step will be to make that data useful and get value from it, to be able to start making predictions for patient outcomes and suggestions for care.

This is the future of healthcare and healthcare IT, but the challenge I see is in finding resources. It is going to take specialized skill sets to be able to sort, process, analyze and make meaningful connections with all of this healthcare data. Who’s going to do that work when healthcare now has to compete with other high-paying industries for these in-demand mathematicians and natural language processing experts?

I think that for healthcare to be able to realize the full potential and value of the vast amounts of digital data that are now being collected, it’s not going to come down to just having the right hardware, software and systems in place. It’s also going to come down to having access to people with the right skill sets and knowledge to turn Big Data into something actionable and meaningful, and those folks are becoming harder and harder to find.

Jeremy Zasowswki is the Marketing Manager for 3M Health Information System’s Emerging Business Team.