The Salient Aspects of Big Data Management

Throughout history men have acted in the best interests of their fellow man and become pioneers of many innovative ideas. One such pioneering feat was brought about by a genuine need to have people connected to one another, a trend expected from a highly social species.  This foresight resulted in the development of the internet which guzzled and regurgitated information with such rapidity and in such copious volumes, users began to see the wisdom of collecting and storing it.

Throughout history men have acted in the best interests of their fellow man and become pioneers of many innovative ideas. One such pioneering feat was brought about by a genuine need to have people connected to one another, a trend expected from a highly social species.  This foresight resulted in the development of the internet which guzzled and regurgitated information with such rapidity and in such copious volumes, users began to see the wisdom of collecting and storing it.

Brief beginnings of ‘big data’ development

At the  dawning of the internet age which really picked up steam in the early 1990s information was minimal but as user numbers doubled and tripled, so did the volume of data. Computer owners found themselves faced with no other option than to depend on external storage space for the increasing traffic in words. Developers of course saw an opportunity and out came the many types of external storage devices that gobbled up the extra information.

But to argue that that was where it ended would be to say the ridiculous for as the number of users grew the more information was passed around till it got to a stage that IBM had to herald the sheer volume of a quintillion bytes  being created daily. It’s easy to forget that words do take up space but in today’s terms, this mind boggling sheer volume of data has made it more difficult for users and providers alike to find a satisfactory solution of how to store such a huge volume of printed words.

The volume of data has now become so huge a more appropriate name – Big Data – has been coined to portray this reality and with it has also emerged the corresponding reference to ‘cloud storage’ a ubiquitous no man’s land space that can act as storage space for huge volumes of information. No more are we now talking of external devices that can store megabytes, gigabytes and terabytes. We are referring to storage space that may turn out to be the only option left to us and who knows, it may well take us into outer space as well.

Collecting and putting ‘big data’ to use

Every piece of information that finds its way into computers is automatically recognized, stored and processed in the deep recesses of the machines. Every user knows that data can be collected following a deliberately calculated action on the computer. We are all familiar with signing up to become members of a certain site or sites. We can create our own accounts, download software, games or apps, surfing simple questions like steps of voter id verification as well as many other functions that we can perform to achieve what we desire.

All these conscious functions are also indirectly augmented by other off-shoots of the same action. For instance if one clicks on a certain word or sentence or spare part, they may be may prove to be valuable information to someone else like a site member or marketer and they automatically become part of the body of data that is collected, kept and processed when the need arises. All types of information that are available all become part of the cycle.

What is more amazing about data collected and stored in a computer is that apart from its own volume, parts of it can be combined with other but dissimilar data and produce such a powerful outcome it’s near impossible not to take account of it. For example, a YouTube video film choices can be combined with data about the details of a country’s population, number of duplicate Aadhar Card issued can be combined with male population in country, the behaviour of animals in the wild or certain primitive tribes and impressions, the data instantly becomes very potent.

When Big Data engenders a presence, it is not the only individual PC or Desktop computer users that take notice and take action but also the big players in business and industries ring across the board. This is to say that the same data that affects a business also interests individuals and collectively they transform a small piece of information into something universal which has the indirect effect of turning a small piece of information into Bid Data. This trend happens every day worldwide.

Big Data can be turned into a political asset for winning campaigns and elections for better or worse and government officials are very much aware of the power of effectively used information. Agencies operating worldwide use data for identifying, analysing and implementing solutions for specific or general health problems. Data on climatic conditions are used for improving farming methods and also used to inform isolated communities about where the bulk of development.

More important for the benefit of those that rely on information for their wellbeing is the need to use such data wisely to make sure that ensuing action is effective in providing solutions to those being targeted. We always tend to think in terms of collecting data as being divorced from its application when in fact, they are complementary to one other. Individuals and governments must see the need to use huge amounts of data for effectively achieving their goals.

The question of providing security for Big Data

From past experience it’s been known that even with the best security measures, breaches in the security system can and do take place. For Big Data, a breach can have very dire consequences. Why? Because stored information may include lots of data that are extremely personal and sensitive to someone. There have been numerous instances of online identity theft and other very serious acts of fraud and theft of wealth.

There is a real and urgent need to install in house analytical systems to help ensure that data is kept safe and secure. The current systems that are in place today may become obsolete a few years from now and many companies including Google have or are in the process of producing analytical tools for systematic use of available data. There is also the need to carefully monitor and protect open-source data analytical systems by eliminating theft prone weaknesses.

Summing it up

For now, the salient aspects mentioned above form only the tip of the iceberg. There are countless other issues that plague the Big Data arena and they are going to take a great amount of thinking and action. There are talks of multi-tiered storage systems, data validity, privacy risk, monitoring, input validation and above them all, cloud computing which has a long way to go before it can be said to offer the best and most efficient storage capability for Big Data.