Big data is one of the hottest buzzwords of the last several years. Companies frequently claim to have access to it, know how to analyze it, and have developed ground breaking products and technology because of it. In reality, very few companies know how to efficiently read and respond to the mountains of data that is available to them.
That's why there are a handful of startups that have begun developing tools for companies who want to do more than brag about big data. Asaf Yigal is the Co-Founder of Logz.io, a big data analytics company based in Tel Aviv, Israel. Israel (a.k.a. “Startup Nation”) is home to some of the world’s hottest tech companies and is quickly developing a reputation that rivals Silicon Valley.
I wanted to know more about how data can be manipulated, what trends are developing in big data, and what the future will look like with the technology Logz.io develops. So I sat down with Yigal and asked him about it.
Q: Describe how Logz has been able to use data-driven decision making to save capital or make significant business decisions.
Yigal: Logz.io is using data to drive every decision being made. We know that it's critical, especially for growing SaaS companies to keep a good handle on expenses as they are working to create exponential growth. We measure our users behavior and the experience they get from the system and correlate behavioral data and system performance to be able to better understand what works and what doesn't.
Q: What do companies frequently find they do not need after analyzing their software use this way?
Yigal: The general experience of releasing a new feature or product is that companies strive to understand usage or lack thereof. After analyzing software use this way most companies find where they should focus their efforts and what would make their customers more successful using their product. When lack of usage is discovered, unless you can visualize all the data in one place, you can only guess as to why that feature is being ignored. Maybe we need an explainer video, maybe the feature is not being noticed, maybe there is no value, or maybe the value is not clear. You can't really answer those questions without looking at the data.
Q: How does Logz determine what is valuable and what is not valuable? Is it simply based on usage?
Yigal: Usage is a big driver in determining what is valuable, but that's not all. We also look for adoption across customers and whether their users have adopted it as well. We can measure repeat usage, frequency, and experience - especially response times and error rates. For example, if a feature is not being used or has been used just once and we see a high error rate coming from users trying to use that feature, it does not mean it is not useful. It might mean that users give up on the feature or do not properly understand how to use it.
Q: How specific can the analyses be? Can it zero in on specific users?
Yigal: The analysis can absolutely zero in on a specific user in an anonymous way and that is what we do when we try to get insights into a single user's experience with a product or feature. But the broad analysis is done at an aggregated level, which gives us a better view across thousands of users. Only when we detect an issue do we drill down and work to better understand the cause.
Q: What is the future of this kind of technology? Is there a logical next step?
Yigal: The future of this technology is to be able to overlay it with machine learning to be able to automatically detect issues in the system that give users a poor experience. Today this analysis is being done manually and requires a lot of effort to go through the data.
Q: Is there a movement or a trend towards this kind of hyper-specific efficiency in startup culture?
Yigal: Yes, especially in SaaS companies where all data is centralized in a single location. Companies that do not analyze their customers’ data rapidly fall behind and have a hard time optimizing their resources to build a better product. We do see this as a major shift in start-up culture and see this as one of the major drivers of the success of SaaS companies over License companies.
Q: What else is Logz doing, and what can we expect to see from Logz soon?
Yigal: Logz.io is working on a new way to look at data. We realize that companies these days collect mountains of data and it's becoming unrealistic to understand what's really important and what companies should really pay attention to. The use of data today is reactive and we're developing a way to also be able to leverage data in a proactive manner.
Have you seen big data driving decisions in your workplace?