CMICAutoscalingGroup Error When Using Cloud Formation in AWS

Teradata Database on AWS
AC2
Fan

CMICAutoscalingGroup Error When Using Cloud Formation in AWS

We are receiving a CMICAutoscalingGroup group error when using Cloud Formation to spin up a Dev/Test environment.  We have tried this multiple times and validated that:

 

* The VPC/subnet are configured correctly (Can see both in and out; We all opened all TCP ports)

* Setup Security Group with TCP (and at the end set all to open)

* Made sure that SNS, AutoScaling and other permissions as assigned as full for the user creating the environment

We have tried creating a new VPC and different versions of the database and different regions

 

The error occurs about 1/2 hour to 45 minutes into the stack starting.  Below are events from the log.s

 

Errors:

The following resource(s) failed to create: [CMICAutoscalingGroup]. . Rollback requested by user.
Received 0 SUCCESS signal(s) out of 1. Unable to satisfy 100% MinSuccessfulInstancesPercent requirement
Failed to receive 1 resource signal(s) for the current batch. Each resource signal timeout is counted as a FAILURE.

 

Logs:

ROLLBACK_COMPLETEAWS::CloudFormation::StackTeradataEcosystemTestdevEnterprise
DELETE_COMPLETEAWS::IAM::RoleDBMppRole
DELETE_IN_PROGRESSAWS::IAM::RoleDBMppRole
DELETE_COMPLETEAWS::EC2::SecurityGroupDBSecurityGroup
DELETE_COMPLETEAWS::EC2::SecurityGroupEcosystemSecurityGroup
DELETE_COMPLETEAWS::IAM::InstanceProfileDBMppInstanceProfile
DELETE_IN_PROGRESSAWS::EC2::SecurityGroupDBSecurityGroup
DELETE_IN_PROGRESSAWS::IAM::InstanceProfileDBMppInstanceProfile
DELETE_IN_PROGRESSAWS::EC2::SecurityGroupEcosystemSecurityGroup
DELETE_COMPLETEAWS::AutoScaling::LaunchConfigurationDBLaunchConfig
DELETE_IN_PROGRESSAWS::AutoScaling::LaunchConfigurationDBLaunchConfig
DELETE_COMPLETEAWS::AutoScaling::AutoScalingGroupDBAutoscalingGroup
DELETE_COMPLETEAWS::IAM::RoleCMICRole
DELETE_COMPLETEAWS::EC2::SecurityGroupCMICSecurityGroup
DELETE_IN_PROGRESSAWS::IAM::RoleCMICRole
DELETE_COMPLETEAWS::IAM::InstanceProfileCMICInstanceProfile
DELETE_IN_PROGRESSAWS::EC2::SecurityGroupCMICSecurityGroup
DELETE_IN_PROGRESSAWS::IAM::InstanceProfileCMICInstanceProfile
DELETE_COMPLETEAWS::AutoScaling::LaunchConfigurationCMICLaunchConfig
DELETE_IN_PROGRESSAWS::AutoScaling::LaunchConfigurationCMICLaunchConfig
DELETE_COMPLETEAWS::AutoScaling::AutoScalingGroupCMICAutoscalingGroup
DELETE_COMPLETEAWS::EC2::SecurityGroupVPSecurityGroup
DELETE_COMPLETEAWS::EC2::SecurityGroupIngressDBSecurityGroupSelfRule
DELETE_COMPLETEAWS::EC2::SecurityGroupIngressCMICSecurityGroupSelfRule
DELETE_COMPLETEAWS::EC2::SecurityGroupIngressEcosystemSecurityGroupSelfRule
DELETE_IN_PROGRESSAWS::EC2::SecurityGroupVPSecurityGroup
DELETE_IN_PROGRESSAWS::EC2::SecurityGroupIngressDBSecurityGroupSelfRule
DELETE_IN_PROGRESSAWS::EC2::SecurityGroupIngressEcosystemSecurityGroupSelfRule
DELETE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupDBAutoscalingGroup
DELETE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupCMICAutoscalingGroup
DELETE_IN_PROGRESSAWS::EC2::SecurityGroupIngressCMICSecurityGroupSelfRule
ROLLBACK_IN_PROGRESSAWS::CloudFormation::StackTeradataEcosystemTestdevEnterprise
CREATE_FAILEDAWS::AutoScaling::AutoScalingGroupCMICAutoscalingGroup
UPDATE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupCMICAutoscalingGroup
CREATE_COMPLETEAWS::AutoScaling::AutoScalingGroupDBAutoscalingGroup
CREATE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupDBAutoscalingGroup
CREATE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupCMICAutoscalingGroup
CREATE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupCMICAutoscalingGroup
CREATE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupDBAutoscalingGroup
CREATE_IN_PROGRESSAWS::AutoScaling::AutoScalingGroupDBAutoscalingGroup
CREATE_COMPLETEAWS::AutoScaling::LaunchConfigurationCMICLaunchConfig
CREATE_COMPLETEAWS::AutoScaling::LaunchConfigurationDBLaunchConfig
CREATE_IN_PROGRESSAWS::AutoScaling::LaunchConfigurationCMICLaunchConfig
CREATE_IN_PROGRESSAWS::AutoScaling::LaunchConfigurationDBLaunchConfig
CREATE_IN_PROGRESSAWS::AutoScaling::LaunchConfigurationDBLaunchConfig
CREATE_IN_PROGRESSAWS::AutoScaling::LaunchConfigurationCMICLaunchConfig
CREATE_COMPLETEAWS::IAM::InstanceProfileDBMppInstanceProfile
CREATE_COMPLETEAWS::IAM::InstanceProfileCMICInstanceProfile
CREATE_IN_PROGRESSAWS::IAM::InstanceProfileDBMppInstanceProfile
CREATE_IN_PROGRESSAWS::IAM::InstanceProfileDBMppInstanceProfile
CREATE_IN_PROGRESSAWS::IAM::InstanceProfileCMICInstanceProfile
CREATE_IN_PROGRESSAWS::IAM::InstanceProfileCMICInstanceProfile
CREATE_COMPLETEAWS::IAM::RoleDBMppRole
CREATE_COMPLETEAWS::IAM::RoleCMICRole
CREATE_COMPLETEAWS::EC2::SecurityGroupIngressCMICSecurityGroupSelfRule
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupIngressCMICSecurityGroupSelfRule
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupIngressCMICSecurityGroupSelfRule
CREATE_COMPLETEAWS::EC2::SecurityGroupIngressEcosystemSecurityGroupSelfRule
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupIngressEcosystemSecurityGroupSelfRule
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupIngressEcosystemSecurityGroupSelfRule
CREATE_COMPLETEAWS::EC2::SecurityGroupCMICSecurityGroup
CREATE_COMPLETEAWS::EC2::SecurityGroupIngressDBSecurityGroupSelfRule
CREATE_COMPLETEAWS::EC2::SecurityGroupVPSecurityGroup
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupIngressDBSecurityGroupSelfRule
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupIngressDBSecurityGroupSelfRule
CREATE_COMPLETEAWS::EC2::SecurityGroupDBSecurityGroup
CREATE_COMPLETEAWS::EC2::SecurityGroupEcosystemSecurityGroup
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupEcosystemSecurityGroup
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupVPSecurityGroup
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupDBSecurityGroup
CREATE_IN_PROGRESSAWS::IAM::RoleDBMppRole
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupCMICSecurityGroup
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupEcosystemSecurityGroup
CREATE_IN_PROGRESSAWS::IAM::RoleCMICRole
CREATE_IN_PROGRESSAWS::IAM::RoleDBMppRole
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupVPSecurityGroup
CREATE_IN_PROGRESSAWS::IAM::RoleCMICRole
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupCMICSecurityGroup
CREATE_IN_PROGRESSAWS::EC2::SecurityGroupDBSecurityGroup
CREATE_IN_PROGRESSAWS::CloudFormation::StackTeradataEcosystemTestdevEnterprise
2 REPLIES
Teradata Employee

Re: CMICAutoscalingGroup Error When Using Cloud Formation in AWS

 

How to troubleshoot AWS Launch failures

1) Launch a new instance from the CFT and in the advanced section just before launch set "Rollback on Failure" to "no". Then complete your new launch.
 
2) If the launch fails go into the EC2 web console:
CloudWatch -> Logs -> filter on Teradata/DBS/<stack name>

If you don't see any log entries (though make sure you have the stack name named right), then it's a EC2 (Amazon)  issue (not a Teradata issue) and you should open a help desk ticket with Amazon. If you do see log entries, please attach

 

Also I recently created article KCS009151 which may or may not be the same issue, but if you are able to get the cloudwatch logs there will be more details usually in those logs that will get you on the right path.

 

 
   
Creating a new one node AWS (Amazon Web Services) TD 16.10  system using CFT (Cloud Formation Templates) hangs during the creation of the auto scaling group​
 
 
In the example below there was a timeout limit of 15 minutes set and we hit that timeout, but if run without a timeout value it runs for about 2 hours timeout and rolls back.

 

 
   
The site did not check the "Enable auto-assign public IPv4 address"  when they created the subnet. There used to be an option to enable Auto-assign IPs when you were having the CFT launch into a new VPC (Virtual Private Cloud) , but it is no longer there.

 

 
   
Make sure the  "Auto-assign IPs" checkbox is checked  in the Subnet and retry the launch. Retry the launch making sure to go into the advanced section and turn off the "Rollback on Failure" so that you can review the complete logs in the event it fails for a different reason. If the deployment is successful you are done. If the deployment fails take a look at the CloudWatch or CloudTrail Logs.
 
 
To look at CloudWatch logs From the EC2 web console, go to:

CloudWatch -> Logs -> filter on Teradata/DBS/<stack name>
 
There should be some details in the CloudWatch log telling you why the deployment failed. If there is no CloudWatch log then you will need to open a helpdesk ticket with Amazon. If the errors in the CloudWatch log are not sufficient to solve the problem open a Teradata incident for assistance.
AC2
Fan

Re: CMICAutoscalingGroup Error When Using Cloud Formation in AWS

Thanks. I am aware of the Rollback option and will get the logs. I belive auto assign IP was set. Will double check.