Remember the string, have phone and internet combo deals landed in our mailboxes? These offers were highly subject to change, and the type of offer and the monthly price could vary greatly between two neighboring apartments or between condominiums in the same building.
I know this because I used to be a data engineer and built extract-transform-load (ETL) data pipelines to facilitate this type of provisioning. One of my jobs involves decoding encrypted data feeds, removing rows or columns with missing data, and mapping the fields into our internal data models. Our statistics team then used clean, up-to-date data to model the best offer for each family.
This was about ten years ago. If you take that process and run it on steroids for 100x larger datasets, you’ll reach the level that medium and large organizations are dealing with today.
Every step of the data analysis process is ripe for disruption.
For example, a single video conference call can generate logs that require hundreds of storage tables. The cloud has fundamentally changed the way business is done because of the affordable, unlimited storage and scalable computing resources you can get.
To put it simply, this is the difference between an old and a modern stack.
Image Credits: Ashish Khakran, TomVest Ventures
Why are today’s data leaders thinking about the modern data stack?
Self-service analytics
Citizen-developers want access to critical business dashboards in real-time. They want to automatically update dashboards built on their activity and customer data.
For example, the product team can use real-time product usage and customer renewal data for decision making. The cloud makes data truly accessible to everyone, but compared to legacy, static, on-demand reports and dashboards, self-service analysis is required.
