Earlier this week we looked at the results of a survey from Enterprise Management Associates that seemed to reveal a trend toward big data success. Today, we look at survey report with results that may be less glowing, even though its author is optimistic toward big data overall.
I have it on good authority that author Leo Tolstoy never wrote, “All successful IT projects resemble one another, but each unsuccessful IT project is unsuccessful in its own way.” But the idea came to mind in looking at the survey results in Capgemini’s new report, Cracking the Data Conundrum: How Successful Companies Make Big Data Operational.
In surveying 226 global respondents among multiple industries and roles – including business, IT, and analytics – Capgemini found some intriguing insight into where big data projects go wrong. These challenges, as noted by Jeff Hunter, vice-president and leader of the business information management group at Capgemini, provide a blueprint for enterprises that want to head in a more fruitful direction.
For instance, the survey found that 79% of organizations “have not completely integrated their data courses,” and only 35% have “robust processes for data capture, curation, validation, and retention.” Ah, yes – the bugaboos of data integration and data management: no news there.
It will also come as no surprise that 67% “do not have well-defined criteria to measure the success” of their big data projects, while other results indicate that there’s little coordination among business groups, IT, and analytics groups. What did we used to call that? Oh, yes, business/IT alignment.
So far, familiar ground. Here’s a new challenge that the Capgemini report reveals: too much “dependence on legacy systems for data processing and management.” Only 36% use the cloud for their big data and analytics platforms, and only 31% use open-source tools.
Perhaps the biggest surprise from the survey? That Hunter is actually “very optimistic” based on the results. Why? Because despite the challenges, he sees growing pockets of success. “I’m seeing the same cycle beginning as with the Internet and e-commerce. In the entirety of the community, a few companies create success. Those who have embraced big data, even those who have experienced setbacks and failures, [are gaining experience]. They’re seeing that data-driven decision-making has its place in how you run a business. Those successes are going to be a catalyst to speed up other projects.”
Thus, while there are common themes to the failures, there are also common themes to the successes, in Hunter’s eyes. As with any technology, there has to be a business directive, driven by a corporate sponsor. Some 53% of survey respondents don’t do this.
Hunter noted that Capgemini has seen success in the use of a centralized unit focusing on data, whether led by a chief data officer or not. “The person [in charge of the unit] might not have that moniker, but there’s a central person responsible for these activities,” he said.
While many enterprises are still dealing with data silos, it’s important to move away from them. It’s not just internal silos, Hunter noted. There’s now so much consumption of external data, especially social media, that enterprises need to figure out how to bring it all together.
That said, just because it’s called big data doesn’t mean you don’t have to boil the ocean. In fact, Hunger warned, “Firms that take it as a boil-the-ocean project get overwhelmed. When you try to do that as a data scientist, you’re operating in the dark in a back room. The timeframe of a boil-the-ocean project doesn’t coincide with business cycles.”
In setting up a centralized group, whether it’s called a center of excellence or something else, enterprises should give it structure. “Data-driven units should be a part of the business,” said Hunter. “They’re not a skunk works. They have to adhere to a process for governance.”
If there’s one important takeaway from the Capgemini report about how to make big data projects go right, it’s this: “There is a science to big data,” Hunter said. For him, it’s no surprise that two of the companies held up as success stories in the report – AT&T and Procter & Gamble – were founded on the precepts of scientific process and research.
“To be successful in data-driven world, there has to be science. You have to embrace that, even if you’re a very creative company. “You have to take a scientific approach, testing a hypothesis to prove its value or show that it’s a failure. Failing fast is just as valuable as succeeding.”
