How Small Data Can Also Be Big Data

Gartner’s definition of big data dates back from 2001, when Doug Laney defined big data as increasing volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources). Since then the term big data has been redefined many times by many different blogs, including this blog. We define big data as seven different aspects (the 7 V’s) that are important for organizations to take into account when developing a big data strategy.

However, the original definition of big data suggests that organisations can only develop a big data strategy if they have vast volumes, terabytes or more, of data. IDC even defines big data projects as projects that contain a minimum of 100 terabyte of collected data. If we take that definition as a starting point it would mean that very few organizations could develop a big data projects as only the biggest (online) corporations collect so much data already today.

Looking at Gartner’s Hype Cycle of Emerging Technologies, it would probably take a lot longer then 5-10 years before big data hits the plateau of productivity if we take the definition of 100 Terabytes of data as a starting point. It would for sure mean that Small and Medium Enterprises, and especially the small businesses, could forget about big data. Fortunately, this is not the case and SME’s can also develop a big data strategy.

With small data I do not refer to IBM’s definition of small data. They define small data as low volumes, batch velocities and structured varieties. Small data however can be any form of data, structured and unstructured and in real-time or in batch processing. Small data simply refers to smaller volumes. Gigabytes and a few Terabytes instead of Petabytes or more.

It is true that the amount of data we create is growing rapidly, but there are still many organisations that only collect a few Gigabyte of data. Quite often these, smaller, businesses believe that therefore they cannot develop a big data strategy. Personally however, I believe that a big data project does not have to involve vast amounts of data; connecting various (smaller) data sets can give even more insights than analysing one vast big data set. Therefore, as I mentioned earlier, a small business has to look differently at big data. Simply because less data is created, does not mean that they cannot develop a big data strategy.

Especially when those smaller company-owned data sets are combined with public and social data sets, great insights can be achieved that can bring a small company forward. In addition, if a company wants to grow their small data sets to become big data sets after all, they should start looking for new data creation opportunities. Data can literally be found everywhere and it is just a matter of collecting it in order to use it.

So, small data can become big data by cleverly combining various data sets with different data formats; Combine weather data with your restaurant’s sales data to discover the impact of rain on your items sold and as such adjust your purchasing behaviour. Combine your customer data with their sentiment online to surprise them and create long-lasting relationships. Track how your customers behave through your shop and combine it with your sales data to see how you can adjust and improve your floor plan. Or combine online sales data with offline customer profiles to see how you can optimize your multi-channel approach for your small retail shop. The opportunities are endless and also small data can provide big insights.

image: small data & big data/shutterstock