Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

Teradata Studio
Enthusiast

Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

Hello All,

I've a desktop Centos installation with Hadoop 2.1, TDCH 1.3.1 and Studio 15.0 installed, (benefits are that everything is on the one OS) and connectivity shouldn't be a problem but...

Upon wizard completion it is finding WebHDFS and the hadoop user fine (changing them to other values provides the appropriate errors, and "dev" user or "hdfs" user seem happy), but Oozie cannot be connected to (although it's perfectly accessible and was used for the "./configureOozie.sh nn=hdp21.localdomain" command was used without error. /etc/hosts is updated

Any recommended ways to diagnose this?

10 REPLIES
Teradata Employee

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

Rupert, Can you verify that you are running Studio version 15.00.01. It is the latest release available on the download page.

Also, if you are running the Hadoop sandbox, you may be hitting the following problem: You need to edit your client host file to include the IP of the sandbox and sandbox.hortonworks.com.

The Windows host file is here: C:\Windows\System32\Drivers\etc\hosts

And an example addition for my sandbox (your IP will be different) is:  192.168.67.132        sandbox.hortonworks.com

Enthusiast

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

Running the linux x64 version on the Centos 6.5 Desktop VM with HDP 2.1 natively installed (custom setup).  So hosts file is at /etc/hosts and Studio is latest Linus version direct from site. So the host name is effectivly "localhost".

Enthusiast

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

I should be clear both connectivity and hdp installation is within the vm - so no external windows client.

The following is the specification of the Hadoop services used by the Oozie workflows:

{

        "Distribution":"HDP",

        "DistributionVersion":"2.1",

        "TeradataConnectorForHadoopVersion":"1.3.2",

        "WebHCatalog":"hdp.localdomain",

        "WebHCatalogPort":50111,

        "WebHDFS":"hdp.localdomain",

        "WebHDFSPort":50070,

        "JobTracker":"hdp.localdomain",

        "JobTrackerPort":8050,

        "NameNode":"hdp.localdomain",

        "NameNodePort":8020,

        "Oozie":"hdp.localdomain",

        "OoziePort":11000,

        "HiveClientMetastorePort":9083

}

However just to test, I then attempted a connection from my windows host to the VM using TDStudio15.00.01 and the same error occurs:

java.lang.Exception: The Oozie service cannot be reached, check the host name and port.

    at com.teradata.datatools.hadoop.connectivity.HadoopConnection.open(HadoopConnection.java:129)

    at com.teradata.datatools.hadoop.connectivity.HadoopConnectionFactory.createConnection(HadoopConnectionFactory.java:30)

    at org.eclipse.datatools.connectivity.internal.ConnectionFactoryProvider.createConnection(ConnectionFactoryProvider.java:83)

    at org.eclipse.datatools.connectivity.internal.ConnectionProfile.createConnection(ConnectionProfile.java:359)

    at org.eclipse.datatools.connectivity.ui.PingJob.createTestConnection(PingJob.java:76)

    at org.eclipse.datatools.connectivity.ui.PingJob.run(PingJob.java:59)

    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)

Enthusiast

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

To test that it is the same problem accessing remotely from a windows box I applied changes of

192.168.100.21 hdp hdp.localdomain (note that on a subsequent install I changed the address from hdp21/hdp21.localdomain) for c:\windows\system32\driver\etc\hosts and once again I got the same error.

Enthusiast

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

I downloaded the sandbox and applied the same from method from windows using TDStudio15.00.01 (win_x64) and poking into the vm and that worked. So the vanilla Ambari install of HDP2.1 does not play currently. Given that my HDP 2.1 instance has all JVM's running, I thought it was likely a config setting that hasn't been bound to a persistant value. But I ran an "env" command in the sandbox and noted no OOZIE_XXX variables in either mine or the sandbox, so that doesn't help.

I there a list of CLI commands I can ping to test why oozie is not visible?

Enthusiast

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

Tere seems to be no real difference between the ports of the HDP2.1 instance and the Sandbox 2.1:

The following is the specification of the Hadoop services used by the Oozie workflows:

{

        "Distribution":"HDP",

        "DistributionVersion":"2.1",

        "TeradataConnectorForHadoopVersion":"1.3.2",

        "WebHCatalog":"sandbox.hortonworks.com",

        "WebHCatalogPort":50111,

        "WebHDFS":"sandbox.hortonworks.com",

        "WebHDFSPort":50070,

        "JobTracker":"sandbox.hortonworks.com",

        "JobTrackerPort":8050,

        "NameNode":"sandbox.hortonworks.com",

        "NameNodePort":8020,

        "Oozie":"sandbox.hortonworks.com",

        "OoziePort":11000,

        "HiveClientMetastorePort":9083

}

Teradata Employee

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

Hey Rupert.

You can check /etc/oozie/conf/oozie-site.xml to see the base URL of oozie under the oozie.base.url property.  To check the status of the Oozie system through CLI, you can enter the following (substitute your URL if needed):

oozie admin -oozie http://localhost:11000/oozie -status

Enthusiast

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

Seems fine:

oozie admin -oozie http://localhost:11000/oozie -status

SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

log4j:WARN No appenders could be found for logger (org.apache.hadoop.security.authentication.client.KerberosAuthenticator).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

System mode: NORMAL

Enthusiast

Re: Teradata Studio 15.0 connecting to Hadoop fails with Oozie connectivity failure

I have also looked at the core-site.xml settings to see if any of these can be changed with hadoop.proxyuser.{username}.groups = * but this hasn't helped either.