Big Data - A synopsis

Every day we create about few quintillion bytes of data.  90% of the data in the world today has been created in the last 2 to 3 years alone. The sudden burst in growth of data can be attributed to: posts to social media sites, digital pictures and videos, purchase transaction records, cell phone GPS signals, sensors used to gather climate information to name a few. This data is BIG DATA.

BIG DATA is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications thus it outgrows your current ability to process it, store it, and cope with it efficiently.

BIG DATA is 4D in IT spatial: Size, Acceleration, Form, Accuracy (SAFA)

Size does matter after all.

Sometimes big data is measured in terabytes, petabytes, exabyte, zettabyte or more. In real word, it's usually measured in frustration, annoyance, anxiety, and money down the drain.

The challenges include capture, curation, storage, search, sharing, analysis, and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to "spot business trends, determine quality of research, prevent diseases, legal citations, combat crime, and determine real-time roadway traffic conditions

Acceleration

For time critical processes such as fraud detection in trade events, predict customer churn etc. BIG DATA must be used as it flows into your enterprise in order to maximize it's value. It's just not a race against time rather to derive potential insights that provides innovative ways of doing things.

Form

BIG DATA is varied type of data - structured and unstructured data such as text, video, audio, sensor data, click streams, log files and much more. New insights are found when these data types are analyzed together.

Accuracy

1 in 3 business leader do not trust the information they use to make decisions. How can one act upon information that they don't trust. Establishing trust in BIG DATA presents a huge challenge and the variety and number of source grows.

BIG DATA is more than simply a matter of size; it is opportunity to find insights in new and emerging types of data and content.