TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

Tools & Utilities
Highlighted

TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

Hi,

 

I am encountering an error when I try to migrate data from TD onto s3 (via the axsmod).

 

$EXPORT: sending SELECT request
$EXPORT: entering End Export Phase
$EXPORT: Total Rows Exported: 1
$EXPORT: Total Rows Discarded: 0
04:46:35 S3AXM [30037]: !Error! Unable to close S3 connection: AuthorizationHeaderMalformed: The authorization header is malformed; a non-empty Access Key (AKID) must be provided in the credential.
04:46:35 S3AXM [30037]: Return value for failing opcode Op_Close is pmrcBadParm: Bad parameter passed to API
$FILE_WRITER[1]: TPT19402 pmClose failed. Bad parameter passed to Access Module API (2)
$FILE_WRITER[1]: TPT19309 Fatal error closing file.
$FILE_WRITER[1]: TPT19015 TPT Exit code set to 12.

 

As per TTU 16.20 we can use IAM roles where the Access Id and keys will be ignored. Will you be advise the fix to this issue ?

Thank you,

 

Asad


Accepted Solutions
Teradata Employee

Re: TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

If this issue has been raised by an incident, I will let the proper channels work the issue.

If you would like for me to assist you, please send me the script and the job variable file. And the entire console output.

You should be able to provide the job variable file as long as you remove anything that might be sensitive.

From the developer:

 

If the module parameter “S3Role” is set to a role name and the node on which the application is running is authorized to use that role name, then the ID and Key are ignored. Since the portion of the TPT output that contains the connector’s logging wasn’t included, I can’t comment on what might be wrong. I need ALL of the early output that includes the version numbers, the module parameters, etc.

-- SteveF
1 ACCEPTED SOLUTION
6 REPLIES
Teradata Employee

Re: TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

Version of TPT?

Script?

Job variable file?

 

-- SteveF

Re: TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

Teradata Parallel Transporter Export Operator Version 16.20.00.01

 

'MODULE_NAME=AWS_S3_ACCESS_MODULE'   Version: MODULE_VERS=16.20.00.01.

 

I cant share the job variable file but I have used the job variable file which is used in tpt s3 axsmod export example in the TTU utilities ver 16.

 

The TPT script is the same as well. 

 

I have raised an incident at T@Ys in this regard that has all of the information (Incident # RECHQRSP3).

 

I dont know whether you can access the portal and see the log files , etc there.

 

I havent received an appropriate response so far.

 

Thank you,

 

Asad

Teradata Employee

Re: TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

If this issue has been raised by an incident, I will let the proper channels work the issue.

If you would like for me to assist you, please send me the script and the job variable file. And the entire console output.

You should be able to provide the job variable file as long as you remove anything that might be sensitive.

From the developer:

 

If the module parameter “S3Role” is set to a role name and the node on which the application is running is authorized to use that role name, then the ID and Key are ignored. Since the portion of the TPT output that contains the connector’s logging wasn’t included, I can’t comment on what might be wrong. I need ALL of the early output that includes the version numbers, the module parameters, etc.

-- SteveF
Teradata Employee

Re: TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

You are in luck; the developer went out and looked at the incident and believes the problem is:

This probably means that the Role "XXXXX" was not valid for the node that this job was running on.

-- SteveF

Re: TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

The developer will need to be more specific on what he means here.

 

If he is saying that the IAM role has not been allocated to the EC2 Instance then I can confirm that it has been.

 

The issue as I see it is that during the pipe transfer , the export occurs , a connection is made but when the connection tries to close at the s3 side it looks for Access Id and secret key values which are coming in as empty.

For some reason the .so code doesnt is not catering for empty Access id and secret key values when it tries to close the connection.

 

That is what these lines in the log suggest :-

 

07:47:48 S3AXM [15267]: !Error! Unable to close S3 connection: AuthorizationHeaderMalformed: The authorization header is malformed; a non-empty Access Key (AKID) must be provided in the credential.
07:47:48 S3AXM [15267]: Return value for failing opcode Op_Close is pmrcBadParm: Bad parameter passed to API
$FILE_WRITER[1]: TPT19402 pmClose failed. Bad parameter passed to Access Module API (2)

 

But since you are the experts , I will wait for you to advise.

 

Thank you,

 

Asad

Teradata Employee

Re: TPT to s3 Load error - The authorization header is malformed; a non-empty Access Key - IAM roles

We will need a more detailed trace, here is the info:

You can accomplish an API trace by setting an extra module parameter to the name of a logfile to be created:

 

S3LogAPI=/tmp/s3_api_trace.log

 

The trace doesn’t need to go in /tmp, but the directory permissions  of the target directory must allow the creation of a file by the user running TPT.

 

This file is ASCII and can be gzipped before sending.  When not using roles, the Access ID and Access Key may appear in plaintext in this file.   When using roles only time-limited Access IDs and Keys obtained through the API exchange appear and it shouldn’t be necessary to “sanitize” the file, but the customer should skip the file for keys anyway.   The time-limited keys generated by the API exchange should not be deleted.

 

 

-- SteveF