On Tuesday April 3, 2018 Cloudera had its quarterly earnings call at which time the CEO, Tom Reilly announced both disappointing earnings and a soft forward looking outlook after which the stock has since slipped from $22.24 per share to $12.95 per share… a drop of a whopping 42 percent. According to CNBC, “The company… found that it was taking more money to acquire new customers and increase spending among existing customers.”
Is this the beginning of the end of big data? Or is it just the end of the beginning?
It isn’t as if the computing world isn’t already littered with failed distributed computing technologies. How many of you have even heard of the Distributed Computing Environment or DCE? Or have you ever heard of CORBA, the Common Object Request Broker? Also, not so many years ago there were lots of software vendors that had invested in providing grid management systems for helping companies implement their own distributed computing clusters, usually in the HPC (High Performance Computing) space.
All of these technologies and approaches have more or less disappeared. The bottom line is that they all required super smart people to make them work. Or another way to put it is that they were all just too damn hard.
Then hadoop came along as an open source technology, Around the same time there was a precipitous drop in compute and storage costs and hadoop became the distributed compute and storage platform of choice. In addition, the venture investment world went crazy with a billion dollars invested to commercialize hadoop in an effort to ensure its success. And there is no doubt, that hadoop, and its offshoots like Spark, have far surpassed any previous attempts at making distributed computing work.
That said, the problem is that even with hadoop and Spark and a few billion dollars of VC money, distributed computing is still too hard. The proof point is in Cloudera’s quarterly earnings call, they revealed they were having to spend too much money and effort to get their installed base customers deployed. That in turn made it difficult for customers to increase their spend with Cloudera. This only reinforces the Gartner press release which stated that “only 15 percent of businesses reported deploying their big data project to production.”
So does that mean that hadoop/spark is dead? NOT AT ALL!!!
This time is different from before when DCE and CORBA both died rather unceremonious deaths. In the past, the assumption was that people would get trained in universities to learn how to program and use these technologies. The assumption this time isn’t that the users will get smarter, it is that the technology will get easier. As a result, in addition to the money invested directly in hadoop, there are also billions of dollars of venture capital money going into filling the complexity gap that was created by hadoop’s introduction.
While the first wave of technology in the big data space was simply laying the basic foundation, the second wave was investment in software vendors that provide point solutions to automate specific implementation issues for big data like, managing data ingestion, doing ad hoc data prep, handling production data pipelines or dealing with the generation of high performance OLAP cubes. And now a 3rd wave of investment is happening that will enable agile data engineering by automating the end to end data pipeline process from ingestion to consumption. And none of what I mention above even includes the investments by Google, Amazon and Microsoft to simplify getting a big data environment stood up in the cloud.
The Cloudera stock drop doesn’t signal the death of hadoop at all. What it does signal is a sign that expectations of “big data “ company stock performance are finally getting right sized until the industry delivers more cost effective and simpler ways to get true value out of big data deployments. The good news is that the 3rd wave of big data technologies has just arrived, so the challenges that caused last week’s negative dent in Cloudera’s stock will start to disappear. Whether that fixes Cloudera’s stock price is unknown. But it will at least accelerate time to successful production deployment from months down to days. And if Cloudera takes advantage of that next wave of technology, their stock price will too.