Hadoop economics is driving big data push into the enterprise, says WANdisco CEO
David Richards, the chief exec of the company which is co-headquartered between Silicon Valley and Sheffield, says that fast-growing data demands mean that those who do not adopt open source tools such as Hadoop are at a competitive disadvantage.
"Imagine a world in which one bank has Hadoop and everyone else is using proprietary storage - it is not going to work," he told ComputerworldUK.
"One bank will have a massive competitive advantage because they can scale up storage at ten percent the cost of everybody else.
"The market for big data has got nothing to do with anything other than this: the requirement for data storage is growing at a sixty percent CAGR [compound annual growth rate] and budgets are not. So you can't use proprietary storage to solve that problem.
He adds: "It isn't a technologically-driven market place; it is an economically-driven market place."
WANdisco - short for Wide Area Network Distributed Computing - provides software to help ensure high availability of Hadoop applications across a geographically disparate network, and counts online insurance aggregrator Comparethemarket.com and retail data analytics firm dunnhumby among its clients.
While customer references of Hadoop use cases continue to surface there have been some questions marks over adoption levels. A recent Gartner report pointed out that more than half of the 284 companies it surveyed plan on investing in Hadoop.
Richards agrees that many early adopters are yet to move to production workloads using Hadoop, with projects often at the experimental stage."Companies are trying it out, and this 'lab' period is 12-24 months," he says. "You see a lot of companies say they are using Hadoop, and my first question is 'what is the SLA' [service level agreement] If that application goes down for two hours, what happens And if they say 'nothing' then you know it is in a lab, it is not being used for a strategic business reason."
However, he contends that this is beginning to change as enterprises go from big data pilots to running applications live in production. All of the Forbes Global 2000 companies have a big data strategy in place, he says, and the next year or so will see these trials come to fruition.
"I think there are so many companies - healthcare companies and so forth - that are coming out of the lab, that are using this for strategic purposes and we are about to see this market evolve very quickly. "
He adds: "We are just seeing the big applications coming out of the lab and going into strategic production. dunnhumby is in full production with us for example."
One of the drivers for this is a movement of the big IT vendors into the market. While early adopters have used Hadoop distributions from the likes of Cloudera, Mapr and Hortonworks, the presence of established suppliers such as IBM, HP and Oracle providing big data 'appliances' - preconfigured hardware and software - that will be easier for businesses to implement.
This will make Hadoop more enterprise-ready, Richards says.
"These are the guys the will take this technology and move it into production."