I’ve been working with customer data for over 20 years now. I’ve seen good data, bad data, black data, red data, big data and bigger data.
Big data is not lots and lots of data; well actually it sort of is, but that not all it is.
Big data has 3 components. These three components must provide value and insight to the user.
Let me explain…
A group of data programming geeks were consulting on site for a large airplane manufacture. They were creating a large data warehouse to store all sorts of details about a planes engine, parts and maintenance details. Each part was to be tracked and available for complex queries. It is big data but it is not Big Data, yet.
A lot of information is collected regarding an aircraft. Mandatory maintenance schedules and updates are recorded. All maintenance (scheduled or unscheduled) must be recorded, commercial and private. This is not a requirement for cars/trucks; cars don’t fall from the sky, yet.
Side Note: Should autonomous cars have mandatory maintenance schedule to be used? Like an aircraft? For example a check point for the vehicle may be required to be road worthy a scheduled intervals.
With tons of data being entered into storage, where to you begin? This is new data to the user, there needs to be exploration to find new patterns that provide insight to decision making. Add lots of information together, so that it is accessible to ask ‘what if’, ‘when if’, ‘why if’… and answering that question is BIG DATA
The geeks, analysts, programmers had already defined that an unscheduled repair is 1000x more costly than a scheduled repair. A well known statistic in transportation. What what can you do about it? How do you find new scheduled repairs that prevent costly unscheduled repairs? Using the data to define benefit or advantage is BIG DATA.
BTW – you don’ t need lots of data to have Big Data. Just Big Value.
Weather forecasts are a user of Big Data. The weather data points have been recorded for centuries. We have applied analytics to this data to create weather forecasts (aka predictive analytics). Weather forecast are a result of Big Data – using history to find patterns to forecast the weather.
Big Data can come from all sorts of sources.
You can see from the chart below ‘every minute of the day’, we as individuals create tons of social data every time we use social apps.
Data is not just created by website searches; is also also created every-time we take a digital picture, a doctor scans a patient on digital film, or a Fitbit collecting steps. This is new data that didn’t exist so long ago.
Side Note: There were pedometers when I was a child. They measured your steps the same way, by movement. Today ‘Fitbits’ are a rebirth of an old tool. The ability to report and share your metrics changed the industry.
If you are digital health geek, you create your own Big Data. Tracking your vitals, exercise, food intake and sleep patterns, gives you insight into goals.
