Finding evidence that someone compromised your cyber defenses is a grind. Sifting through all of the data to find abnormalities takes a lot of time and effort, and analysts can only work so many hours a day. But an AI never gets tired, and can work with humans to deliver far better results.
A system called AI2, developed at MIT's Computer Science and Artificial Intelligence Laboratory, reviews data from tens of millions of log lines each day and pinpoints anything suspicious. A human takes it from there, checking for signs of a breach. The one-two punch identifies 86 percent of attacks while sparing analysts the tedium of chasing bogus leads.
That balance is critical. Relying entirely upon machine learning to spot abnormalities inevitably will reveal code oddities that aren’t actually intrusions. But humans can't hope to keep up with volume of work required to maximize security. Think of AI^2 ^as the best of both worlds---its name, according to the research paper, invokes the intersection of analyst intuition and an artificially intelligent system.
Most of AI2's work helps a company determine what's already happened to it can respond appropriately. The system highlights any typical signifiers of an attack. An extreme uptick in log-in attempts on an e-commerce site, for instance, might mean someone attempted a brute-force password attack. A sudden spike in devices connected to a single IP address suggests credential theft.
Other machine-learning systems dig through mountains of data looking for suspicious activity. But only AI2 uses regular input from analysts to turn that mountain into a molehill. A machine lacks the expertise to do the job alone.
“You have to bring some contextual information to it,” says research lead Kalyan Veeramachaneni. That's the role of human analysts, who recognize external variables that might explain a given outlier. An obvious, and common, example: Companies often stress-test their systems, causing irregularities that everyone expects. An unsupervised AI has trouble discerning such a test from a legitimate threat. AI2 can figure it out within weeks.
“On day one, when we deploy the system, it’s [only] as good as anyone else,” says Veeramachaneni. Instead of working in isolation, though, AI2 shows a security expert the day's 200 most abnormal events. The analyst provides feedback, identifying legitimate threads. The system uses that information to fine-tune its monitoring. The more often this happens, the fewer outliers the AI identifies, improving its ability to identify actual threats.
“Essentially, the biggest savings here is that we’re able to show the analyst only up to 200 or even 100 events per day, which is a very tiny percentage of what happens,” says Veeramachaneni.
None of this is theoretical. AI2 honed its skills reviewing three months’ worth of log data from an unnamed e-commerce platform. The dataset included 40 million log lines each day, some 3.6 billion in all. After 90 days, AI2 could detect 85 percent of attacks. Veeramachaneni says the unnamed site saw five or six legitimate threats a day during that time, and his system could pinpoint four or five.
Not a perfect sore, but Veeramachaneni says achieving an 85 percent detection rate using unsupervised machine learning would mean having analysts review thousands of events per day, not hundreds. Conversely, pulling 200 machine-identified events each day without an analyst's input yields a 7.9 percent success rate.
AI2 also can help prevent attacks by building predictive models of what might happen the following day. If hackers use the same method over the course of a few days, a business can bolster security by, say, requiring additional confirmation from customers. If you know someone's trying to swim across your moat, you can throw a few more alligators in there.
Though the tech shows great promise, it cannot replace human analysts. Security is just too important, and the threats too varied. "The attacks are constantly evolving," Veeramachaneni says. "We need analysts to keep flagging new types of events. This system doesn’t get rid of analysts. It just augments them.”
Science might one day provide an infallible security system. Until then, a combination of accuracy and efficiency remains the best anyone can hope for. And that, it turns out, means man and machine working together.
