Big Data - Are we and you prepared to handle it... ?

Every day we come across articles, columns or blogs about Big Data in some newspaper, magazine or management journal. Authors, like on our blog, praise its advantages and demonstrate concrete scenarios and situations where Big Data can improve things for the better. But the question we should ask ourselves is: Are we ready to handle Big Data? 

Research Company Gartner predicts that 4.4 million new jobs in the field of Big Data will be created worldwide by 2015. Despite that fact, McKinsey & Company in one of their reports recently revealed that they expect a shortage of up to 190,000 data scientists and 1.5 million managers with capabilities to use Big Data successfully by 2018. This insight is dramatic considering that data-driven companies on average show a significantly superior performance in many benchmarking studies. So how can companies, individuals and countries prepare themselves for this new era of data? 

Universities around the world are jumping on the new trend and offer courses in Big Data. My own Alma Mater, IE Business School, recently introduced an Elective in the MBA about Big Data. Other schools like Columbia University have used additional funding to establish a new institutes in this field that offer several courses around Big Data and would eventually provide entire Master degrees to interested students. Similar Masters Degrees are being offered by George Washington University, NYU and Schulich Business School. Universities are trying to fill the gap by offering relevant courses to  potential students. Nevertheless, the scale of available Big Data education is by far too little yet and traditional business schools will find it difficult to say goodbye to their preferred case study methodology, which is less adequate to teach Big Data. 

Companies on the other hand try to compensate for the trend by hiring legions of consultants from IBM, Accenture and Co. to participate in this new development and to bring new knowledge to their organizations. Despite short-term impact, it is not rare though that these kind of initiatives lose traction once the consultants leave after handing over the initiative to employees. This may be due to pure lack of skills on part of employees. The failure of such an initiative may be attributed to corporate cultures, which may not be data-driven per se. A change in culture is a required to encourage employees to use data to address business issues. Companies therefore should never lose focus to build strong in-house capabilities about Big Data and to change the culture from the top to a more data-driven organization that empowers employees to take decisions based on data rather than rigid rules. 

It is clear that not every individual can leave work and study full-time now to become proficient in Big Data. Individuals interested in Big Data can use free online courses available on platforms like Coursera or Open2Study. In addition, they could join pilot projects in their direct surroundings to gain experience in this new field of business.

Lastly, if you are still not convinced that looking into Big Data might be worth it, the average annual salary for data scientists in the U.S. is $80,000 to $220,000 based on the website www.glassdoor.com. Not bad for a bunch of geeks that by now seem to be seen more and more on main street.