When this blog started, it was based on my 2008 Teradata Partners User Conference presentation on the future data explosion and its impact on the Enterprise Data Warehouse: “’Long Tails,’ ‘Black Swans,’ and Their Impact on EDW and AEI.” One could foresee a definite future negative impact on the existing EDW platforms trying to support all this new volume and types of data (social, sensor, etc.).
I thought I would post here the last four design issue slides from this presentation:
Fail-over capacity (Dual Active)
Need flexibility to generate queries to react to unknown events
Mixed workload planning for access capacity crush
Since 2008, the technology has kept up and all of this data is being accommodated today with the varied technologies available to us. However the vision of a single view of the data is still paramount, based on my two decades of experience with Integrated Data Warehouses for Teradata. This overall requirement is addressed by Teradata Unified Data Architecture (UDA): still a single view of the data, but on different platforms that most cost effectively store and process the data, with all the data accessible from a single query across a high-speed interconnect
So where are we going from here? As always, the industry will apply the most cost effective hardware and software to the value of the data being captured. And as the cost of the platforms steadily decrease, the amount of data being captured and stored for analysis will increase. Data that was too costly to collect and store for analysis three years ago is now available on the web for free.
Expect the difficulty of supporting this vision to continue to be in the complexity of connecting all of the disparate data stores into a single view for the end user, be they a data scientist or a CEO.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.