What is big data?
I’ve worked in digital information startups for the past 15 years. The notion of working with large volumes of advertising and customer data for predictive insights and better decision making is nothing new.
However, the technology, analytics and venture-capital communities have finally found a word that resonates with the idea of ever-expanding data sets — a constant in our world. That word is “big data”.
Being a marketing and word wonk, I’m quick to point out that the term big data is jargon. It’s become so ubiquitous that it risks one of two outcomes: becoming cliche, or becoming nothing more than a clunkier variation of the word “data”. That’s why the term bothers me.
Instead of dwelling on semantics, I thought it would be more worthwhile to unpack the intended meaning of big data. I’m no techie, so I turned to Sanjay Gupta, my colleague at Clickable, who leads our engineering team, including our big-data infrastructure deployments.
What is big data? According to Sanjay:
“Big data” refers to the huge volume of data that cannot be stored and processed using conventional databases in a reasonable time. While the storage problem has been solved by large offline storage systems, analysis tasks need to combine data in many ways; data read from one source may need to be combined with the data from multiple sources. This is where conventional data processing systems fail. Big-data technology tackles this problem by running logic on large data chunks in parallel and then combining them together in a manner that gives global results. We’re now mining data sets that are so large in volume that they become truly representational of the entire problem space. This leads to more accurate predictions and more valuable insights.
What is the promise of big data in marketing?
The ability to leverage big data has opened opportunities for startups to build tools and platforms that can surface insights around demand, supply, consumer behavior, segmentation, positioning and targeting. Big data is a natural fit for marketing because of the huge volumes of sample data that can drive analysis and more meaningful predictions.
Want to learn more about big data?