Big Data Dissonance: The reality vs expectation of big data solutions
While the promise remains alive and well, big data is yet to deliver for most organisations. Gartner reports that more than 85 percent of Fortune 500 organisations will not be able to exploit big data effectively in 2015. More alarmingly, there is disconnect between what CEOs perceive big data is doing within their organisations and what the rest of the organisation is witnessing.
In a recent study of 362 executives, The Economist revealed that 47 percent of CEOs believed that all their employees have access to the data they needed, however, only 27 percent of all respondents agreed with the sentiment. Further, 43 percent of CEOs thought relevant data was captured and made available in real time compared to 29 percent of the total study.
The findings of the study raises two critical questions; why is there such perceptual dissonance, or gap, between CEOs and the rest of organisation - in terms of data And, why are data science solutions failing to meet the expectations
Cultural issues
Perceptual dissonance in this instance may occur for a myriad of reasons; employee engagement, the personality of the CEO or media hype surrounding the speed and scale to which big data can be implemented successfully. However, it's most likely that organisational culture is creating the disparity. Most businesses have yet to adopt a data centric culture whereby data capture, value, protection and utilisation is communicated and integrated enterprise wide.
Russell Glass, head of marketing products at LinkedIn, cites that cultivating a data centric culture requires top-down leadership and bottom-up engagement. While boards are excited and engaged with the prospect of big data, many boards are missing someone with the expertise to communicate the technical capabilities of the organisation; the value of developing existing data assets and the ability to drive holistic data science strategies.
Unfortunately, it's not the case whereby you can switch on the proverbial light bulb and instantly aggregate and analyse all manner of internal and external data to build a competitive strategy. Research from TDWI found 46 percent of organisations are unable to exploit big data analytics due to inadequate staffing or skills.
Data driven solutions are long-term strategies that not only require investment in technology but also various specialist analytical, technical, managerial and operational skills - at various stages of implementation, ongoing maintenance and utilisation.
Skills gap
The Economist states that organisations must invest in the technologies and tools which will enable them to house, process, organise and visualise data effectively. In implementing these technologies, businesses may need talent anywhere from business analysts to specialist infrastructure engineers, service desk analysts to experienced project managers.
Throughout the development lifecycle - from design and implementation to testing and evaluation - specialist skills from enterprise service management (ESM) and technological backgrounds are needed to integrate the right technology and tools. Even when the right talent has been identified, competition amidst an exponentially increasing skills gap is making it harder and harder for businesses to acquire the right specialist skills.
Across InterQuest Group we are seeing increasing client demand for the skilled talent needed to implement these technologies. For example, demand has increased for Project Managers (47 percent), Business Analysts (44 percent) and Service Desk Analysts (58 percent) over the last five years; these roles consistently feature in the top 10 in-demand roles across the group.
Once the necessary technologies are in place, mathematical and analytical talent capable of driving business intelligence strategies are critical if the investment is to harvest results. The core purpose of deploying a big data solution and cultivating a data driven culture is to build a competitive advantage through business intelligence. The best Data Scientists and Data Architects; able to design, co-ordinate and model data assets are crucial resources when converting data into actionable intelligence.
The dissonance between the expectation of big data solutions amongst CEOs and the reality of what big data is actually achieving within organisations, is visibly linked to the gap in skills in the jobs market. Leading tech companies, such as Amazon and Google, have cultivated data centric cultures built on niche specialist talent.
Further, organisations with technical expertise on the board prove to be far more adept at implementing business intelligence. In order to execute scaled big data solutions, organisations must equally execute sophisticated hiring strategies and implement them from the top down.
Mark Braund is CEO of specialist IT recruitment firm InterQuest Group.