Issue on post-deploy scripts

HSeganfredo · ‎04-12-2018

After deploying the TD on Developer Tier template, scripts fail.

Image is Teradata_Database_16.20.05.01_SLES11_SP3_on_VMware_20180219161644.ova.

Checking the generated deploy_tdc.deploy.2018-04-10T16.09.03.0698095-03.00.txt file I got:

***********************************************************************************************
*
*   10/04/2018 16:22:33   - Configure Teradata DBS...
*
***********************************************************************************************
10/04/2018 16:22:33 - TDput Configure Teradata Operation on ( SBCDF25F ) and will take approximately 20 minutes...
( SBCDF25F ) SystemName( BACEN04) SystemFamily(000CLV) TotalAmps(4) AmpsPerNode(4)
AVISO: The version of VMware Tools on VM 'SBCDF25F' is out of date and may cause Invoke-VMScript to work improperly.
invoke-vmscript : 10/04/2018 16:34:33 Invoke-VMScript  A general system error occurred: vix error codes = (3016, 0).

No C:\Users\A4113406\AppData\Local\Temp\tdvm20180410160849629\tdc.Binarydeploy.ps1:3966 caractere:38
+ ... pt_Output = invoke-vmscript -scripttext "sh /root/tvme/ConfigureTerad ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Invoke-VMScript], SystemError
    + FullyQualifiedErrorId :
Client20_VmGuestServiceImpl_GetProcessOutputInGuest_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.InvokeVmScript
invoke-vmscript : 10/04/2018 16:34:33 Invoke-VMScript  A general system error occurred: vix error codes =
(3016, 0).

No C:\Users\A4113406\AppData\Local\Temp\tdvm20180410160849629\tdc.Binarydeploy.ps1:3966 caractere:38
+ ... pt_Output = invoke-vmscript -scripttext "sh /root/tvme/ConfigureTerad ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Invoke-VMScript], SystemError
    + FullyQualifiedErrorId : Client20_VmGuestServiceImpl_GetProcessOutputInGuest_ViError,VMware.VimAutomati
   on.ViCore.Cmdlets.Commands.InvokeVmScript

10/04/2018 16:34:33 Status of step:
Success: Sysinit and DIP complete.
Copy-VMGuestFile : 10/04/2018 16:35:34 Copy-VMGuestFile  A general system error occurred: vix error codes = (3016, 0).

No C:\Users\A4113406\AppData\Local\Temp\tdvm20180410160849629\tdc.prevalidation.ps1:3277 caractere:13
+             Copy-VMGuestFile -LocalToGuest -Destination "/root/tvme/" ...
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Copy-VMGuestFile], SystemError
    + FullyQualifiedErrorId :
Client20_VmGuestServiceImpl_DirectoryExistsInGuest_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.CopyVMGuestFile
Copy-VMGuestFile : 10/04/2018 16:35:34 Copy-VMGuestFile  A general system error occurred: vix error codes =
(3016, 0).

No C:\Users\A4113406\AppData\Local\Temp\tdvm20180410160849629\tdc.prevalidation.ps1:3277 caractere:13
+             Copy-VMGuestFile -LocalToGuest -Destination "/root/tvme/" ...
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Copy-VMGuestFile], SystemError
    + FullyQualifiedErrorId : Client20_VmGuestServiceImpl_DirectoryExistsInGuest_ViError,VMware.VimAutomatio
   n.ViCore.Cmdlets.Commands.CopyVMGuestFile

invoke-vmscript : 10/04/2018 16:36:34 Invoke-VMScript  A general system error occurred: vix error codes = (3016, 0).

No C:\Users\A4113406\AppData\Local\Temp\tdvm20180410160849629\tdc.prevalidation.ps1:3204 caractere:30
+ ... pt_Output = invoke-vmscript -scripttext       "/usr/pde/bin/pdestate ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Invoke-VMScript], SystemError
    + FullyQualifiedErrorId :
Client20_VmGuestServiceImpl_RunScriptInGuest_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.InvokeVmScript
invoke-vmscript : 10/04/2018 16:36:34 Invoke-VMScript  A general system error occurred: vix error codes =
(3016, 0).

No C:\Users\A4113406\AppData\Local\Temp\tdvm20180410160849629\tdc.prevalidation.ps1:3204 caractere:30
+ ... pt_Output = invoke-vmscript -scripttext       "/usr/pde/bin/pdestate ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Invoke-VMScript], SystemError
    + FullyQualifiedErrorId : Client20_VmGuestServiceImpl_RunScriptInGuest_ViError,VMware.VimAutomation.ViCo
   re.Cmdlets.Commands.InvokeVmScript

False

10/04/2018 16:36:34 Status of step:
Error: Error Teradata could not be properly configured. It should have started.

Deployment started - 10/04/2018 16:09:03
Deployment finished - 10/04/2018 16:36:34

Any ideas on the root cause?

ArnieChazen · ‎04-12-2018

I noticed the date says 10/04/2018 is the ESXi and/or PC running in another language by chance? For example that date format in the US is October 4, 2018 but in Mexico or Brazil it would be April 10, 2018. So either the date is set wrong or the date format is a non-US date format. Also, try running the following commands in powerCLI:

$theVM = Get-VM SBCDF25F

Invoke-vmscript -vm $theVM -GuestUser root -GuestPassword iumb123 -ScriptText "echo Hello World"

HSeganfredo · ‎04-16-2018

Hi, I was able to get into the VM, after creating a new user and customizing some network and SSH settings. Some questions:

1) Where can I find the proper log files for the post deploy scripts to find out what really happened?

2) Date/time strings at the VM are in US standard time, ~ # date Mon Apr 16 18:57:47 EDT 2018

2.1) Does the local datetime string (workstation where the VMWare PowerCLI runs) makes any difference? Here is my output:

PowerCLI Z:\Teradata\Teradata for VMWare> date

segunda-feira, 16 de abril de 2018 19:59:28

3) We may post the files we did edit for the deploy. In specific, the file common.IT.properties.json was customized to have the following line. This could make any impact?

"OSTimeZone": "America/Sao_Paulo",

Any ideas?

ArnieChazen · ‎04-16-2018

1) Where can I find the proper log files for the post deploy scripts to find out what really happened?

[ANSWER] The post deployment scripts are locatd in /root/tvme on the Teradata nodes. The pre-deployment logs are on the PC you deployed from under the "logs" subdirectory.

2) Date/time strings at the VM are in US standard time, ~ # date Mon Apr 16 18:57:47 EDT 2018

2.1) Does the local datetime string (workstation where the VMWare PowerCLI runs) makes any difference? Here is my output:

PowerCLI Z:\Teradata\Teradata for VMWare> date

segunda-feira, 16 de abril de 2018 19:59:28

[ANSWER] Normally the language should not matter but I have seen some failures due to the haredware device names that the script is looking for.

3) We may post the files we did edit for the deploy. In specific, the file common.IT.properties.json was customized to have the following line. This could make any impact?

"OSTimeZone": "America/Sao_Paulo",

[Answer] The "America/Sao_Paulo" timezone string seems to be a valid timezone string so that should be ok.

Did you run the previous command I mentioned ?

ArnieChazen · ‎04-16-2018

Previous commands I mentioned to run from the PC you deploy from. note you may need to change the root GuestPassword iumb123 to the actual password if you changed it.

$theVM = Get-VM SBCDF25F

Invoke-vmscript -vm $theVM -GuestUser root -GuestPassword iumb123 -ScriptText "echo Hello World"

HSeganfredo · ‎04-17-2018

HSeganfredo · ‎04-17-2018

I was able to get log into the VM using SSH, replaced the /etc/.../ssd_config with one that allows root logins and got to change the eth0 ip addressing. Inside the VM I did notice the user tvme was not created (there is no /home/tvme directory nor and /etc/passwd entry).

HSeganfredo · ‎04-17-2018

Nothing peculiar on the /root/tvem directory. See below. Seems unchanged.

ArnieChazen · ‎04-17-2018

It looks like your /root/tvme directory is missing alot of files. Below is what I have in my directory. You might want to try re-deploying again and see if you hit the same errors or not.

dbs140:~ # cd /root/tvme
dbs140:~/tvme # ls -ltr
total 900
--wxrw--wt 1 root root     77 Dec 7 07:42 tvmeinfo
-rwxr-xr-x 1 root root     39 Dec 7 07:43 base_os.txt
-rwxr-xr-x 1 root root 47251 Dec 7 07:43 SLES11_SP3_16.20_pkgs.json
-rwxr-xr-x 1 root root 502218 Dec 7 07:44 pdeperfchk_sp3
-rw-r--r-- 1 root root   3798 Mar 7 09:42 readme.txt
-rw-r--r-- 1 root root   1324 Mar 7 09:42 node.IT.properties.2nodedbs140-141.json
-rw-r--r-- 1 root root   1227 Mar 7 09:42 common.IT.properties.2nodedbs140-141.json
-rw-r--r-- 1 root root 12429 Mar 7 09:43 all.zip
-rw-r--r-- 1 root root 44432 Mar 7 09:43 tpa.zip
-rw-r--r-- 1 root root    276 Mar 7 09:45 map_table.fix
-rw-r--r-- 1 root root   8986 Mar 7 09:50 LinuxTimeZoneStrings.txt
-rw-r--r-- 1 root root   6712 Mar 7 13:50 verifyCopiedFiles.sh
-rw-r--r-- 1 root root   6050 Mar 7 13:50 tdcLockdown.sh
-rw-r--r-- 1 root root 65428 Mar 7 13:50 tdc-rebuild.py
-rw-r--r-- 1 root root   1163 Mar 7 13:50 setTimeZone.sh
-rwxr-xr-x 1 root root 15652 Mar 7 13:50 setDbsFlags.sh
-rw-r--r-- 1 root root   4440 Mar 7 13:50 networkValidate.sh
-rw-r--r-- 1 root root 15378 Mar 7 13:50 networkConfigure.sh
-rw-r--r-- 1 root root     76 Mar 7 13:50 mpplist
-rw-r--r-- 1 root root    880 Mar 7 13:50 hosts_tpa.txt
-rw-r--r-- 1 root root    453 Mar 7 13:50 genPublicKeys.sh
-rw-r--r-- 1 root root   1384 Mar 7 13:50 fix_map_table.sh
-rw-r--r-- 1 root root    520 Mar 7 13:50 enableRssForViewpoint.sh
-rw-r--r-- 1 root root   5690 Mar 7 13:50 dsu_setup.sh
-rw-r--r-- 1 root root   5256 Mar 7 13:50 configureTCHN.sh
-rw-r--r-- 1 root root   5140 Mar 7 13:50 configureNTP.sh
-rw-r--r-- 1 root root   1019 Mar 7 13:50 UpdatePdiskLinks.sh
-rw-r--r-- 1 root root    962 Mar 7 13:50 TemporalTimeZoneStrings.txt
-rwxr-xr-x 1 root root   7952 Mar 7 13:50 SetSystemTimeZone.sh
-rw-r--r-- 1 root root   4333 Mar 7 13:50 RebuildVconfig.sh
-rw-r--r-- 1 root root   2553 Mar 7 13:50 GetFailedNodes.sh
-rw-r--r-- 1 root root   6775 Mar 7 13:50 ConfigureTeradata.sh
-rw-r--r-- 1 root root     57 Mar 7 13:50 .gitignore
-rw-r--r-- 1 root root    187 Mar 7 16:57 ntp.stdout
-rw-r--r-- 1 root root    107 Mar 7 16:57 ntp.restart.log
-rw-r--r-- 1 root root      2 Mar 7 16:57 ntp.exitcode
-rw-r--r-- 1 root root   7988 Mar 7 17:14 SetSystemTimeZone.log
-rw-r--r-- 1 root root   4096 Mar 7 17:17 viewpointUser.btq
-rw-r--r-- 1 root root 22578 Mar 7 17:25 state.json
dbs140:~/tvme #

ArnieChazen · ‎04-18-2018

The next recommended actions are as follows:

From the tdcdeployer script run the following steps manually in this order:

1) Reboot (to reboot the newly deployed VM)

2) Copyfiles step

3) Run the Deploy option again (it will pick up where it left off)

If this does not help then we may want to start from the begininig again.