10 More Powerful Facts About Big Data

Predictive analytics is fast becoming a mainstream technology tied to business outcomes, so the playing field is expanding from data science labs into the business and even corner offices.

According to TDWI 's Predictive Analytics for Business Advantages 2014 survey, business analysts will soon overtake statisticians and data scientists as the top users of predictive analytics tools. To prepare for this transformation, where data teams and business managers work closely together, here are six tips to help companies take advantage of predictive analytics:

1. Pinpoint the need
Too often, teams try to aggregate interesting data sources and see where it takes them. Data scientists and business managers need to define their problem as well as what they 're testing for and their desired outcomes. Only then should the data team start building a model.

[Want more on the mainstreaming of predictive analytics? Read Microsoft Brings Predictive Analysis To Azure.]

Companies cite "knowledge of the business" and "critical thinking" as the top two required skills for successful predictive analytics projects, according to TDWI. This underscores the importance of business teams driving the questions that data teams in turn answer. Avoid open-ended questions, which force analysts to develop a system that may not solve the business problem at hand. Targeted, specific questions leave little wiggle room for data scientists to go off course.

2. Don't underestimate office politics
One quarter of companies in TDWI's research cite "data ownership and political issues" as the number-one challenge with big data analytics. That's because companies collect data across all departments, so there's a lack of visibility into what's being collected and who is collecting it. Consequently, trying to acquire this information can be like trying to get a bone from fifteen territorial bulldogs. This task needs to be handled either by a skilled business manager heading the data project or a senior-level champion who can command everyone to "play nice."

Don't discount this threat. Many analytics projects have failed due to political grandstanding and in-fighting among business folks. When some are asked to hand over their data to another manager, they fear helping someone else to get a big win.

3. Keep it simple
Overfitting is a common data-modeling problem. Data scientists often find a sense of challenge and pride in building complex data models. But they cannot afford to lose sight of the fact that data eventually needs to be explained to business managers in a clear, concise way.

Shoot for simplicity and prioritize data volume and data quality over model complexity. Running data through powerful -- but not overwrought -- models guarantees the clearest, most accurate insight. That's not to say there isn't a place for more complex modeling. Just remember that the overarching goal is to provide value to the business. Data professionals can simplify their modeling work and ensure they'll deliver actionable information by working with business managers from the beginning of project, instead of waiting until the end.

4. Ensure that data drives the timeline
TDWI cites data integration as both the top challenge and the most important component of predictive analytics projects. When decision-makers need answers, they need them yesterday. They expect machine-based calculations to happen in seconds, not days or weeks. In reality, the timing of predictive modeling is unpredictable, so data scientists -- not business managers -- need to mandate the timeline, which starts when the data is ready, collected, organized, aggregated, and integrated. By managing expectations, you can avoid the inevitable pitfalls that plague projects where there is last-minute modeling and poor data preparation.

5. Embed data teams
There is often a difference between people who structure the collection of data and those who use it for analysis. Embedding data professionals within business teams ensures interaction between these two parties from the beginning. It also ensures that the data that's collected is useful for analysis and that minimal data-transformation work is required.

6. Increase business involvement
Data teams often figure that their job stops after they present the data, as business managers can then interpret the data and put the findings to use. The latest software helps business managers have more control over predictive analytics projects, but they still lack the quantitative muscle to spot nuances in data. Data teams have to communicate quantitative findings for business managers so they can better understand and present insights to business colleagues and senior managers.

Today's larger playing field for predictive analytics demands better coordination and collaboration between data scientists and technically minded business managers. That's no small task, but the rewards can be enormous.

InformationWeek's June Must Reads is a compendium of our best recent coverage of big data. Find out one CIO's take on what's driving big data, key points on platform considerations, why a recent White House report on the topic has earned praise and skepticism, and much more.