|
At the start of any significant IT hype cycle, predictions abound. No matter what the trend, a seemingly endless array of vendors, experts and analysts alike line up to place their bets as to how this new movement will forever change the IT and business landscape. Big data was no exception. As an explosion of immense new unstructured datasets gave root to the big data hype cycle that now dominates so much of the ongoing IT conversation, scores of experts chimed in with opinions on the many ways in which this new megatrend would change the way companies do business. While a handful of oft-predicted outcomes of the big data trend have in fact come to pass — notably the emergence of analytics as a key IT spending priority, an increase in the prevalence of data-driven decision making at the executive level, and the rise of the CMO as a key power broker in the new data economy — many of the most commonly predicted outcomes never come to fruition. In some cases, not only have those outcomes failed to materialize, but what actually has occurred is the complete opposite of what was predicted. In many ways, it’s those unexpected outcomes that represent big data’s most significant impact on the IT landscape. Let’s take a closer look at four unexpected impacts of big data playing out today. 1. Revitalization of the Data Warehouse Conventional wisdom at the start of the big data hype cycle was that the introduction of Hadoop and other modern unstructured databases would signal the beginning of the end for the traditional data warehouse. Companies were expected to significantly divest in what seemed like an aging relic compared to these emerging and more modern platform options. But rather than being the source of its demise, big data has actually helped breathe new life into aging data warehouses, driving new investment and spurring new uses cases in the process. Rather than ushering traditional data warehouses out, organizations have instead focused on blending the old with the new, and as such, data warehouses are being revamped and revitalized, enabling organizations to lower TCO while achieving greater ROI. Companies are reexamining their data warehouse to become smarter about which portions need to be kept, and which portions can be released for archiving or jettisoned altogether. New opportunities to replicate data warehouses to secondary structures in order to fuel business analytics are being explored. Many organizations have even taken a hybrid approach in which their data warehouse serves as the analytic sandbox, with Hadoop serving as more of an archive environment. Whatever the case, reports of the data warehouse’s demise were greatly exaggerated. 2. Resilience of Traditional Relational Databases The data warehouse wasn’t supposed to be the only casualty of big data. The same fate was expected to befall traditional relational databases. Believing that the maturation of Hadoop and other NoSQL sources would spur widespread migration of data away from traditional systems, experts rushed to predict a major decline in license revenue for the big relational database vendors. Traditional databases were purported to have no place in a big data world. Nothing could have been further from the truth. The mass migrations simply never materialized, and likely never will. Accessibility, high-availability, and security of unstructured databases remain significant concerns. Private cloud initiatives have made staying on RDBMSs more cost-effective than anticipated. And the evolution or repurposing of relational DBAs into Hadoop DBAs just hasn’t happened, thus creating both a skills and technology gap. Moreover, as the big data trend has evolved, it’s become clear that big data means all data, and that includes traditional relational databases. A host of recent industry and analyst reports indicate that the bulk of today’s data analytics projects are run using Oracle and SQL Server — the most traditional of traditional databases. 3. Redoubling of Heterogeneous Environments The advent of cheap, easy-to-use, open-source databases was supposed to put an end to complexity. Organizations were simply going to pack everything they could into their Hadoop clusters to create an increasingly homogenous and less complicated data environment. Or so it was predicted. Instead, the exact opposite has happened. Big data has created data environments that are more complex and heterogeneous than ever before. Whether it's unstructured data, structured data, historical data, archive data, desktop data, machine data, text data or sensor data — the list just continues to grow. Moreover, given the growing thirst for analytics capabilities emanating from lines of business, many new tools and additional layers of complexity now exist in silos throughout the organization, in some cases, without IT’s knowledge. While Hadoop has provided an outstanding solution for managing certain forms of unstructured data, innovations are still needed to provide a more robust view of the entirety of the information that flows through an organization. Vendors today are actively working with customers to help them solve the complexity challenge, and as data management technology continues to advance, the more homogenized and less convoluted world predicted at the start of the big data hype cycle may yet materialize. It’s just won’t have happened nearly as quickly as most predicted. 4. Resurgence of the QuantsIt may seem hard to believe given the enormous importance now placed on finding and developing data scientists, but at the onset of the big data movement, most experts were predicting the demise of the quants. It was believed that easy-to-use, open-source technologies would provide an everyman’s tool kit that minimized the need for sophisticated analytic tooling and expertise. To say this assumption was off the mark would be a gross understatement. Largely because of the thirst for insight and the focus on data-driven decision making brought on by big data, predictive analytics expertise — both in the form of tooling and personnel — is more important now than it’s ever been. Standing up a Hadoop cluster is one thing. Deriving ROI in the form of insight through predictive analytics is another altogether. While Hadoop has absolutely simplified the complexity of storing unstructured data, there’s another level of expertise needed to go from having data to having information. As a result, quants — and the predictive analytics tools that empower them — are now cooler than ever. |
|
|
