The Upshot, Vox and FiveThirtyEight: data journalism's golden age, or TMI?

In data, as in journalism, one is an example, two's a coincidence, and three is a trend – and so as today marks the launch of the third new US data-driven journalism site in about a month, it's probably a good time to look at the state of the enterprise.

The new kid on the block is The Upshot, a project launched by the New York Times to fill the gulf left by Nate Silver, who re-launched his FiveThirtyEight site with ESPN five weeks ago. This was shortly followed by the launch of Vox, a new vehicle from Ezra Klein, late of the Washington Post.

Day one of The Upshot looks good: it's clean, there's a coherent set of topics, and the data is as beautifully presented. Which is what you might expect from a news organization with the NYT's experience – the team, led by David Leonhardt but including many new outsiders, has established itself well.

The early reception for FiveThirtyEight and Vox was decidedly less generous. Part of this came from jealousy – Silver and Klein both enjoy, or suffer from, the wünderkind label – but the backlash may have been sparked by faintly hubristic moments at launch: a cringeworthy promotional video at Vox, and a round of interviews by Silver that threw shade at most of the media who might help direct traffic his way. (And that was before an epic climate-change fail.)

Both of those sites have since produced some decent pieces of work, along with the occasional dud, and now comes the Times, with a new polling model that is nicely presented, and an accompanying analysis of what's happened to the rich and middle-class that's excellently done. But it's fairly standard fare for explanatory journalism: the data's (relatively) easy to get, and not insurmounably hard to explain. Finding numbers nearly as good on drugs, trafficking or even homelessness isn't nearly so easy.

With so many rival sites hopping up, it's worth thinking about what we actually want the standard fare of our data journalism – or explanatory journalism, if you prefer that more marketable description – to become: should these startups stick to the proven model, or dig out new data? Should they break news or just analyze it? And how much is too much – are we being over-served, under-served, or have we now hit the Goldilocks point?

Preaching to whom?

View image in fullscreen

Perhaps the trickiest thing for any of the three sites – or any similar projects yet to come: who, exactly, is the audience? People who like being explained to?

FiveThirtyEight's topics have varied from polling to sports, equality in movies to toilet-seat covers. Vox has covered porn (twice), Obamacare, climate change, Ukraine and Edward Snowden.

But neither site truly aims to break news on the areas they cover, and therein lies a problem: are readers meant to visit their favorite "regular" news sites, then hop by and see if the newcomers have anything to add (or debunk)? Neither FiveThirtyEight nor Vox has offered quite enough (yet) on any of their specialities to become the first stop.

Here The Upshot may have an advantage: if it sticks to the narrower areas of polling and policy, there's clearly a defined audience for that – as Silver showed when his section of the Times site brought in a huge audience during the 2012 election. It's possible those droves might not like Silver's new, broader site – and they could opt for the Times' new venture instead.

Far more people are interested in the topics to which data-driven journalism can add than they are in data-driven journalism itself. No-one would launch a news site dedicated to interviews, hosting Miley Cyrus alongside Newt Gingrich and Jonathan Franzen. Perhaps standalone sites devoted to explanatory journalism will soon seem equally strange.

Preaching data to the data choir?

Specialist sites have another tricky problem: the people who read them are likely to be the ones who already have a pretty good understanding of the news. You've got to be a pretty informed (and humble) news consumer to read a news piece and then hunt down a separate site to understand it better.

David Leonhardt has at least tried to characterize his audience, though he risks making them all sound like the know-it-all at the bar who likes the sound of his voice too much.

"They want to grasp big, complicated stories — Obamacare, inequality, political campaigns, the real-estate and stock markets," he writes in an introductory note, "so well that they can explain the whys and hows of those stories to their friends, relatives and colleagues."

That's fine, but all three sites risk what economists call adverse selection: let's say you're writing basic explainers, but the only people finding them are already pretty informed. They'll find your content superficial, and they won't return. If you respond by increasing the complexity of your articles, you'll please the wonks, but alienate a little more of your audience. There is at least the risk of a vicious cycle.

All the standalone sites need to pick who they're writing for, and how they're going to do it – at the moment, the tone and complexity varies wildly from piece-to-piece. That's maybe doable in a large news organization, but it becomes tricky when you're establishing a new brand.

Making it personal

All the new explanatory sites have some brilliantly presented graphics and interactives – whether on March Madness or income inequality. These things take a lot of time and effort (FiveThirtyEight hasn't done another major one since launch day), and so it's worth looking at where that energy is focused.

All three sites have some great talent, and not just of the words-and-pictures variety. So it's a shame the new wave of data journalism hasn't gone a little further: the deep numbers and the big charts and the key explainers are your chance to personalize the news, to say what it means. On income inequality, couldn't the Times try to show people how they're personally forecasted to fare, or what percentile they'd be in in different countries? If explaining pensions – and the problem of undersaving – why not ask people how long they expect to live, and show the results versus what the actuarial tables predict?

As a bonus, you'd also collect an an anonymous dataset showing the mortality expectations of people of different ages, ethnicities and genders versus reality. There'd likely be a whole new analysis to do on that.

The web gives news organizations the chance to present and personalise information in a way they never could before. Can't they do more than just animate bar graphs?

Lots of explaining, not much real data

US political punditry, as Nate Silver has regularly and correctly said, is pretty damn bad. But US polling numbers must be one of the most analyzed set of figures anywhere: The Upshot already presents five different rival models of the 2014 Senate race. My model forecasts there'll be at least four more major news organizations with forecasting models come Election Day.

Again, data-driven journalism is easy when the data's easy to get: GDP figures, job figures, polling and surveys. What about all of the issues that need that kind of explanatory power when the data's not so easy to come by? Focusing too much on where the data is – therein lies a big risk of skewing the coverage mix, helping already-neglecting issues stay neglected ... and also over-selling how much of the world we can explain with the data approach.

The hunt for data and figures should be an aggressive one.

The Times has the advantage here, if Leonhardt can – as he's said he will – work with the newsroom. Silver has some non-journalists along for advanced analysis, though nobody with conventional reporting experience, while Klein has similarly brought along a brilliant team of bloggers, but few of their true blue newsroom compatriots.

Being accountable, showing your work

Doing original research on data is hard: it's the core of scientific analysis, and that's why academics have to go through peer-review to get their figures, methods and approaches double-checked. Journalism is meant to be about transparency, and so should hold itself to this standard – at the very least.

This standard is especially true for data-driven journalism, but, sadly, it's not always lived up to: Nate Silver (for understandable reasons) won't release how his model works, while FivethirtyEight hasn't released the figures or work behind some of their most high-profile articles.

That's a shame, and a missed opportunity: sharing this stuff is good, accountable journalism, and gives the world a chance to find more stories or angles that a writer might have missed.

Counter-intuitively, old media is doing better at this than the startups: The Upshot has released the code driving its forecasting model, as well as the data on its launch inequality article. And the Guardian has at least tried to release the raw data behind its data-driven journalism since our Datablog launched five years ago.

... and that's still a lot of white guys

Journalism is dominated by white men (like me), and data journalism is no better than the rest of the profession. This has been already been discussed at length, but there's an issue of quality as well as fairness: data journalism is driven by generating and testing hypoetheses.

The ideas that might occur to a straight white guy born to rich parents aren't the same as those that might to an Asian woman. This is not an issue of window-dressing: a more diverse team will deliver better journalism and better analysis.

There's also the issue of how you test your explanations. Let's say crime rates go up (they're actually falling): is that because of the economy? Income inequality? A failing education system? Racial tensions? Or some other explanation that you haven't thought of yet?

In reality, it'll be a mix of many of the above, and a host of other things. But sitting with the data alone won't tell you that. Putting good reporters in the field might. Either way, explanation and reporting might not be separated quite as easily as the new startups would hope.

We'll see when we've got more data on how The Upshot, Vox and FiveThirtyEight are doing.