We were trying to establish connectivity between Aster 6 and Hadoop 2.1 for which we tried two approaches as mentioned below:
Approach 1: Connectivity through SQL-H
Issue with this approach: While making the host entry in Aster AMC I do not see HDP 2.1 version in the drop down.
retail_sales=> select * from load_from_hcatalog ( ON mr_driver SERVER ('220.127.116.11' ) USERNAME ('hive') DBNAME ('default') TABLENAME ('sample_07') ) ;
ERROR: SQL-MR function LOAD_FROM_HCATALOG failed: Failed to load hadoop classes : sqlh config not found for server 18.104.22.168
Approach 2: Connectivity through Sqoop
Issue with this approach : I faced some transaction isolation level error and it looks like the jar used for this connection has set the isolation level which is not compatible with the PostgreSQL/Aster database
sqoop import --connect jdbc:ncluster://153.65.160.
14/08/05 05:09:59 INFO sqoop.Sqoop: Running Sqoop version: 22.214.171.124.1.1.0-385
14/08/05 05:09:59 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.
14/08/05 05:09:59 INFO manager.SqlManager: Using default fetchSize of 1000
14/08/05 05:09:59 INFO tool.CodeGenTool: Beginning code generation
14/08/05 05:09:59 INFO mule.Mule: sslEnabled : false, ssoEnabled : false
14/08/05 05:09:59 INFO mule.Mule: Authenticating user 'beehive' for database 'retail_sales'
14/08/05 05:09:59 INFO mule.MuleInterface: Authenticating using passwd for user beehive for database retail_sales
14/08/05 05:10:00 INFO mule.Mule: Successfully authenticated
14/08/05 05:10:00 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: [AsterData][JDBC](11975) Unsupported transaction isolation level: 2.
It looks like connectivity between Aster 6 and HDP 2.1 is not supported yet. But can anybody confirm if this holds true or we do have a work around it?
Was anyone able to get this issue resolved ? We too want to implement a similar usecase (Aster <-> HDP) data transfer.
So wanted to know, what is best possible option.
There is a new version for Aster 6 which gives the HDP 2.1 dropdown in the AMC while making the configuration of the host. Not sure when will this version be available for common use but this does support the HDP2.1 with Aster 6
From Aster manual:
Unsupported JDBC Settings
Most JDBC option-setting methods in the Aster Database driver exhibit the standard JDBC
behavior, but there are exceptions. Please note:
• setReadOnly(): Not supported; Aster Database does not allow changing connection type
• setTransactionIsolation(): Not supported; Aster Database does not allow changing
transaction isolation levels.
Sqoop seems to always set the Transaction Level (level configurable) explicitly yet Aster won't allow it to be set. So...we need Hortonworks to allow a config to not set it...or Aster needs to accept it (even it it doesn't respect - maybe just a warning).