AWS instance unreachable for Teradata Database Developer (Single Node)

Teradata Database on AWS

AWS instance unreachable for Teradata Database Developer (Single Node)

I've launched the AWS Teradata Database Developer (Single Node) , and can't connect with ssh to start the DB ; not sure why i can't ssh.

The AWS instance state shows the instance is unreachable.  My security group has ssh TCP port 22 open, my pem file is chmod 400.

When i view the instance system logs it appears to be forcing fsck , i also see a message mount: devpts already mounted or /dev/pts busy.

I accepted adding both the EBS and ephemeral disk. The AMI i launched was from the marketplace GCA_DBS_15100106_LocalStorage_TRIAL

Any ideas how to remedy this ?

Thanks in advance for your help, 

Tim.


 

System Log: i-8c7b1f51



 

 

[    0.000000] ACPI: RSDP 00000000000ea020 00024 (v02    Xen)
[ 0.000000] ACPI: XSDT 00000000fc00f5a0 00054 (v01 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: FACP 00000000fc00f260 000F4 (v04 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: DSDT 00000000fc0035e0 0BBF6 (v02 Xen HVM 00000000 INTL 20090123)
[ 0.000000] ACPI: FACS 00000000fc0035a0 00040
[ 0.000000] ACPI: APIC 00000000fc00f360 000D8 (v02 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: HPET 00000000fc00f4b0 00038 (v01 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: WAET 00000000fc00f4f0 00028 (v01 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: SSDT 00000000fc00f520 00031 (v02 Xen HVM 00000000 INTL 20090123)
[ 0.000000] ACPI: SSDT 00000000fc00f560 00031 (v02 Xen HVM 00000000 INTL 20090123)
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal 0x00100000 -> 0x007b1200
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[3] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x0000009e
[ 0.000000] 0: 0x00000100 -> 0x000f0000
[ 0.000000] 0: 0x00100000 -> 0x007b1200
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 7889610
[ 0.000000] Policy zone: Normal
Fast TSC calibration using PIT
Detected 2494.025 MHz processor.
kdb version 4.4 by Keith Owens, Scott Lurndal. Copyright SGI, All Rights Reserved
* Found PM-Timer Bug on the chipset. Due to workarounds for a bug,
* this clock source is slow. Consider trying other clock sources
bounce pool size: 64 pages
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
doing fast boot
Disabling lock debugging due to kernel taint
Unable to read sysrq code in control/sysrq
Creating device nodes with udev
ata_id[274]: HDIO_GET_IDENTITY failed for '/dev/.tmp-block-3:0'

mount: devpts already mounted or /dev/pts busy
mount: according to mtab, devpts is already mounted on /dev/pts

Boot logging started on /dev/ttyS0(/dev/console) at Wed Jul 20 00:06:35 2016

Waiting for device /dev/disk/by-label/ROOT-BE1 to appear: ok

fsck from util-linux 2.19.1

[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/hda2

ROOT-BE1 has gone 187 days without being checked, check forced.


 

System Log: i-8c7b1f51



 

 

[    0.000000] ACPI: RSDP 00000000000ea020 00024 (v02    Xen)
[ 0.000000] ACPI: XSDT 00000000fc00f5a0 00054 (v01 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: FACP 00000000fc00f260 000F4 (v04 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: DSDT 00000000fc0035e0 0BBF6 (v02 Xen HVM 00000000 INTL 20090123)
[ 0.000000] ACPI: FACS 00000000fc0035a0 00040
[ 0.000000] ACPI: APIC 00000000fc00f360 000D8 (v02 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: HPET 00000000fc00f4b0 00038 (v01 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: WAET 00000000fc00f4f0 00028 (v01 Xen HVM 00000000 HVML 00000000)
[ 0.000000] ACPI: SSDT 00000000fc00f520 00031 (v02 Xen HVM 00000000 INTL 20090123)
[ 0.000000] ACPI: SSDT 00000000fc00f560 00031 (v02 Xen HVM 00000000 INTL 20090123)
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal 0x00100000 -> 0x007b1200
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[3] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x0000009e
[ 0.000000] 0: 0x00000100 -> 0x000f0000
[ 0.000000] 0: 0x00100000 -> 0x007b1200
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 7889610
[ 0.000000] Policy zone: Normal
Fast TSC calibration using PIT
Detected 2494.025 MHz processor.
kdb version 4.4 by Keith Owens, Scott Lurndal. Copyright SGI, All Rights Reserved
* Found PM-Timer Bug on the chipset. Due to workarounds for a bug,
* this clock source is slow. Consider trying other clock sources
bounce pool size: 64 pages
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
doing fast boot
Disabling lock debugging due to kernel taint
Unable to read sysrq code in control/sysrq
Creating device nodes with udev
ata_id[274]: HDIO_GET_IDENTITY failed for '/dev/.tmp-block-3:0'

mount: devpts already mounted or /dev/pts busy
mount: according to mtab, devpts is already mounted on /dev/pts

Boot logging started on /dev/ttyS0(/dev/console) at Wed Jul 20 00:06:35 2016

Waiting for device /dev/disk/by-label/ROOT-BE1 to appear: ok

fsck from util-linux 2.19.1

[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/hda2

ROOT-BE1 has gone 187 days without being checked, check forced.


Tags (1)
10 REPLIES

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

Please use Public IP address or Public DNS of instance for ssh connection.

Note that "ec2-user" is the user you need to use for ssh connection as shown below,

:~# ssh -i private-key-file ec2-user@public-DNS or public-IP

For more instructions on remote login to Teradata Instance on AWS, please refer Chapter - 4, Configuring Teradata Software for AWS from Teradata Database on AWS Getting Started Guide manual.

You can get manual @ http://www.info.teradata.com/edownload.cfm?itemid=160780004

Teradata Employee

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

When you state that "The AWS instance state shows the instance is unreachable" this sounds like a problem with the deploy from AWS side.    I would drop the instance and redeploy if that is the case or contact Amzon support.

Teradata Employee

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

Also double check that you opened port 22 to your IP address or your compainies IP address. When it is a company IP address it is usually autodetected in the drop down menu. It is usually not the exact IP address you see from ipconfig.  Is there any indication that your system is visible to ping or ssh?

Teradata Employee

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

To connect the instance, please check follows from AWS instance console.

1. The Instance State should "running" state

2. The Status Checks shoule "2/2 checks passed" state

    - Need to wait with patience until state has completed. Refresh browser to get the latest state.

3. You should have the valid Key file (.pem) that specified in Key Name.

4. You should set valid Public IP in your terminal application.

Displayed syslog message "ROOT-BE1 has gone 187 days without being checked, check forced." is working as designed.

(Forced check filesystem is enabled when reboot)

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

Thank you for the replies, i tried them all and was still having an issue.  However, after leaving the instance up for about an hour while trying to debug and write this reply, finally my ssh attempt connected.  I restarted the instance too, but after about an hour it seemed to finish the startup and allow ssh.

The system logs shows that it made it past the fsck and other checks.  I wasn't patient enough i guess ;)

For about an hour the system showed 1/2 checks passed, the failed one is Instance Status Checks: These checks monitor your software and network configuration for this instance. Instance reachability check failed at July 20, 2016 at 7:01:00 PM UTC-6 (26 minutes ago).  

The ssh error was  $ ssh -i MYkey1.pem ec2-user@<mypublicIp>ssh: connect to host <mypublicIp> port 22: Bad file number

When i ssh'd to the other systems the connection was fast, when i ssh to a unused IP the error was fast to appear.  when i tried to ssh to the TDD instance ssh hung for about a minute before generating the Bad file number error.  

For about an hour when i pinged the TDD instance from a test system, in same AZ & region, it was unreachable 100% packet loss.

When i tracerouted to the TDD instance from a test system, in same AZ & region, it was not visible, until after about an hour.

I think it was stuck on a bunch of disk checks or other housekeeping, and i didn't wait long enough for it to start.

Fan

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

Have the same issue and I think @tisenhart is right. It's fs check that takes so long, I didn't have the issue before. I would like to deploy an instance via an automation tool, so this is not going to work. Teraform should redeploy AMIs that don't require fs checks.

thanks 

Teradata Employee

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

Thank you call for feedback,  redeployment of AMIs are under way to address this.

Fan

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

Any way you could add cloud init to the image? :-)

Enthusiast

Re: AWS instance unreachable for Teradata Database Developer (Single Node)

The same issue for me. Any Teradata ami's at AWS could not pass checks after launching. It goes on for 9 days. Instances that had been  launched before 9 days ago are working fine.

Please let us all know when this issue will be address.