Hadoop threat to data warehouse vendors

Database
Enthusiast

Hadoop threat to data warehouse vendors

A friend pointed me to this article and comment on it : 

http://wikibon.org/wiki/v/Data_Warehouse_Vendors_Moving_to_Contain_the_Hadoop_Threat

I saw a recent announcement between Cloudera and Hadoop :  http://www.datanami.com/2014/10/09/teradata-edges-closer-hadoop/

Experts ,do we have to understand  : Teradata failed to predict about its customer needs of "data being offloaded from their warehouses to Hadoop" .

Did it ignore hadoop till last month ? Also any rough estimate how much % of data will be offloaded to hadoop in near future and do we continue to see this trend growing ?

2 REPLIES
Enthusiast

Re: Hadoop threat to data warehouse vendors

My opinion:

Not at all a threat. Rather, it is creates a plethora of opportunities.There are tens of thousands of writes-up on the web.
But those who made comments, may not have gone deep or not looked from all angles. They may have looked from few angles.

If someone works on hadoop, then MR is difficult.I used to develop and run hive queries and it takes time. I dont want to mention which client but I found many issues.

There are many functions that exist but do not serve the business functionality. It is faster in Aster and it is SQL-MR.Java codes, basic ok, but complex one can

really take time to develop. Take the pain to develop complex java udf , then we will know how hard it is.
Data loading, staging and transformation may stand competitive to each other, because of the set up of block. In fact, they can complement each other, because some

functionalities are equally good or slightly better in hadoop than in Teradata.

Teradata has been there since 1976 and from there it has been evolving through so many business cases.Now it can support upto 234 PB and 2048 nodes.I dont think

organization may need that much. Many databases have parallelism but not comparable to Teradata, right from start and so on...It supports cloud platform too besides

being mpp, running on Linux/MF. It has quite a number of tools and utilities for all use-cases:bteq,tpt,mload,fload,fexport,tpump,assistant, odbc, jdbc,.net

provider,olde db, plug in for eclipse.Temporal features make management of business data easier over time. Where can we get in Hadoop/Big data utilities or tools or

assets like ARCHMAIN,MDS,Miner, profiler,SNAP™ framework,nPath,data labs, Aggregate Designer, PEMA's R-integration to TD, Analytic data set generator, Unity

Ecosystem manager---director,mover,loader and the like. Teradata's LDMs such as CLDM,ulDM,HCLDM,FSLDM.. are in many places.Applications TVA, TDE, MDM,DCM,Marketing and

Campaign Manamegement.

TD workload management tools, TASM, Viewpoint and other tools like Analyst pack,Utility pack,.The values obatined from each of these tools are tremendous. Even with

Workload management, OLTP, OLCP and OLAP can be housed together under the same environment with the advancement of Technologies in memory such as SSD, TD's Intelligent

memory, FSG where we can fine tune IO,CPU usage, memory usage. Compression mechanisms Autocomp, ALC, BLC,MVC.Teradata's BAR solution ,DSA, even taking care of

incremental backup. TD connects well with Attunity, Protegrity, informatica,Ab Initio, SAP DS..

Workload specific platforms Active EDW,Appliance for SAS,Aster Big Analytics Appliance, Appliance for hadoop, Data mart appliance,Integrated Big data platform.....

Data discovery workbench can give exuberant business-values from structured and multi-structured data.SNAP™ framework for discovery has an egde
over other Graph SQL DB like Neo4j in the sense that it can be SQL or graph form for analytics and the storage in AFS is compatible with HDFS
and this storage is portable in any environment taking advantage of Query Grid. Columnar can be switched on.JSON,GeoJSON,advanced XML processing, Geospatial, z co-

ordinateintroduction besides elementary ones and specific datatypes have covered well in almost all evenues. The name-value pair processing can be done in
a more efficient way thru udf. Teradata® Database 14.10 offers SQL-H™ to access Hortonworks Hadoop data dynamically from the database. Cloudera connects well
with Teradata. Teradata acquired Revelytic for hadoop metadata management and Hadapt for Big data and SQL-on -hadoop.

Many newcomers like Spark seem to be really sparkling in the market, with the cost being low.

As a package Teradata is galaxy way ahead of others.

Having said so many things, can we sacrifice Data governance, MDM.....? Experienced Resources, training are few and far between in Big data/hadoop.
Technical support is very important.
Big data may even become bigger. We are still human , having maximum two eyes, brain ......Data has to be brought
to an acceptable and business-value form, well-understood by human-being. There is no use of data even if it is
Geopbyte, if it cannot be converted to a business opportunity or value. A simple analogy of a big apple.
We cannot eat it at one gulp or chew it at one go. We have to eat part by part and throw away unwanted things- digestive system.
Same thing for data too. It is meant for a reason and for human being.

Should one risk a multi-million dollar business, with lots of dependencies at the hands of a newcomer? No. I feel for now, hadoop complements
Teradata. Hadoop is not a threat.
Enthusiast

Re: Hadoop threat to data warehouse vendors

Hadoop was not designed to be a production database.  Don't have to worry about anyone using Hadoop as their data warehouse single source of the truth.  To many issues with transaction recovery, backups, etc.  Hadoop is great for many things but ultimately the result gets loaded into an EDW.  Hadoop works very well with Teradata but will not replace Teradata.

Joe