A Smart Approach to Sifting Gold From the Twitter Stream

By combining people power and computer power, these sites divert a bit of the Twitter firehose into a quality, sippable stream.
467719872
Abstract MagnetGetty Images

Like most people who write for a living, I scrupulously maintain my feeds. Twitter. Tumblr. RSS. They're a fixture of my daily existence, both invaluable and utterly overwhelming.

The latest additions to my media diet, two websites with the oddball URLs Belong.io and Latest.is, feel different. They're essentially lists of links. But they're lists of reliably great links, ones I often don't see anywhere else.

Belong and Latest share a similar approach. Both start with a human-curated group of Twitter users and use algorithms to cull the links those people are discussing. This clever one-two punch results in a phenomenally high signal-to-noise ratio, albeit for relatively narrow bands of subject matter. By combining people power and computer power, the sites turn a bit of the Twitter firehose into a quality, sippable stream.

More broadly, they show an interesting approach to tackling the eternal problems of discovery and information overload.

Culling Links

Belong was built by Andy Baio, a well-known and widely-connected web denizen (and occasional WIRED contributor). Beyond making stuff for the Internet, Baio is an avid consumer of it. For the last 12 years, he's maintained Waxy Links, a daily digest of interesting stuff from around the web. Baio refers to it as a "tangible manifestation" of his procrastination.

As the web has channeled into feeds and streams, Baio has noticed a change in his activity. "I started finding more and more of my links through Twitter," he says. "So I started to wonder if I could automate link discovery."

Belong, which Baio maintains as a side project, examines the tweets of every person who has ever attended XOXO, an annual arts and technology festival Baio co-organizes. It gathers every link this group tweets and attempts to "normalize" them, stripping URLs of extra crud and matching slightly different links to the same bit of content.

Belong

The site ranks these links by considering their freshness, popularity and simplicity. Freshness examines how recently the link was indexed. Popularity looks at how many people are linking to it. The newer something is, and the more people talking about it, the higher the ranking.

The third metric, simplicity, is more interesting. Belong looks at the complexity of each URL, privileging short links over long ones. "The hope is that it boosts standalone sites over blog posts and news articles," Baio says. "The theory is that if someone devotes a domain or a subdomain to something, it's probably a more significant standalone work. And that's the thing I'm interested in: substantial, new independent works."

When I wrote this, the formula appeared to be working. The top link on Belong goes to Fight215.org, a site devoted to galvanizing the public around getting Congress to roll back mass surveillance. Next is a link to a video on Fusion in which rap radio DJ Jay Smooth discusses the horrific murder of an unarmed black man by a police officer in South Carolina. Then there's a link to the Gizmodo post "Seriously, Stop Demonizing Almonds."

The mix offers a succinct example of why I find Belong so useful. This is the first time I've seen Flight215.org. It's new and relevant to my interests. I don't follow Fusion, but the Jay Smooth video is thought-provoking and I'm glad I watched it. I do follow Gizmodo, but I follow dozens of publications and often fly right by their tweets. Seeing the almonds story pop up on Belong means busy people are linking to it. It's an endorsement of quality. It's like I'm overhearing someone---or many people---recommending this particular story to a friend, and am thus inclined to check it out.

Belong isn't perfect. At a rate of about a hundred links a day, there's still some skimming and scanning required. It doesn't offer an easy way to track new links throughout the day. There are many stories I'm not at all interested in, and the site misses entire swaths of topics by design. It's best to think of it as a collection of stuff that's at the periphery of the mainstream tech press. Still, within this niche, it does a remarkably good job.

Latest

The Latest, created by Per Stenius and Oskar Sundberg, two developers from Stockholm, surfaces content from a different area of the internet. The links skew more to web design and advertising, drawing from a list of some 1,000 Twitter users amassed by the two creators. "A lot of them are our friends, some are journalists, some are just completely random people we've found online," Sundberg says. The site scores links based on freshness and frequency and displays the top ten. Every day, I click on at least one or two.

A Better Newspaper

As Baio noted, this style of aggregation isn't entirely new. Blogdex aggregated links in the golden age of blogs, and Upian's Hot Links was a similar popular aggregator. Today, there are sites like MediaGazer and TechMeme, though these largely focus on news stories. There are things like Reddit, Metafilter, and Flipboard, each using a unique social discovery recipe.

And then there's Facebook. What is the News Feed if not a behemoth effort to apply algorithmic smarts to content shared by a curated group of people---your friends?

Still, Belong and Latest feel different. For one thing, they're all links---no status updates, no baby pics. They also tap into what people are already talking about. Unlike, say, Hacker News, which operates on Reddit-style up-voting of submitted links, Belong and Latest distill a conversation happening organically. They surface things that interesting people are finding interesting enough to share.

And that gets to the real differentiator: interesting people. It's no surprise the links shared by folks who attend cool arts and tech festivals are better than those shared by people I went to school with. Of course, I happen to be interested in arts and technology and design and advertising, so Belong and Latest are perfect for me. But presumably you could apply this sort of approach to any topic. You could round up all the top Republicans on Twitter and surface all the best Ted Cruz links.

It's this basic idea---using a curated group of people as a starting point for discovery---that I find interesting. It provides an overview of what's happening in a plugged-in neighborhood of the social web without you having to spend all day there. Twitter sends me a daily of things that are "popular in my network." Belong and Latest do something far more useful: They find the stuff that's popular in networks better than my own, ones I'd never have the time or patience or knowledge to cultivate otherwise.

This approach is a compelling complement to the follower model that Twitter and Facebook are built upon. As designer Mills Baker astutely observed, following can be exhausting. "If Twitter is your dynamic real-time newspaper, it's also a newspaper with a single editor: you. While that feels empowering, it's also burdensome; you are now required to tailor your timeline to your taste in perpetuity."

Belong is like a newspaper edited by hundreds of really smart people, with contributions from anyone who produces anything for the web. An algorithm moves the best stuff to the front page. It does all the finding and following and curating for me. All I have to do is show up and read.