Redshift Target Access Issues for Schema Conversion Tool CLI

0

Hi all,

I am trying to use the SCT CLI to automate some of our workflows for our migration from Teradata to Redshift and am running into some access issues. It seems that the SCT CLI defaults to using an AWS profile when you are adding a target redshift server, but our workflow just uses a db superuser and a secret manager value for the password (no AWS account is used when we use the SCT UI, I want to replicate this in the CLI, but I can't figure out how to not use an AWS account).

Any thoughts / recommendations?

asked a year ago389 views
2 Answers
1

Hi, I would strongly recommend you to follow this step-by-step example to get familiar with the SCT CLI tooling and then adapt to your own use case with RedShift.

Blog post: https://aws.amazon.com/blogs/database/convert-database-schemas-and-application-sql-using-the-aws-schema-conversion-tool-cli/

I'd also recommend to read this blog, #6 of an interesting series, describing to use SCT to convert / migrate to RedShift: https://aws.amazon.com/blogs/big-data/part-6-accelerate-your-data-warehouse-migration-to-amazon-redshift/

On you question re. AWS CLI: if you use it, it requires a profile (with access key and secret key). The IAM user corresponding to this pair of key will need proper IAM authorizations to access the services that you use: sct, sm, etc. To best understand how CLI works: https://dev.to/aws-builders/aws-cli-do-it-right-1c3n

Best,

Didier

profile pictureAWS
EXPERT
answered a year ago
0

The AWS Schema Conversion Tool (SCT) CLI does provide flexibility in terms of how you can authenticate to your target Redshift cluster. If you're trying to avoid using AWS profiles for authentication and instead want to use database credentials directly, you can follow these steps:

  • Database Credentials: Ensure that you have the necessary database credentials (username and password) for your Redshift cluster.
  • Secret Manager: If you're using AWS Secrets Manager to store the Redshift password, you'll need to retrieve it programmatically using AWS SDK or AWS CLI. Here's a sample command to retrieve a secret value using AWS CLI:
aws secretsmanager get-secret-value \
--secret-id YOUR_SECRET_NAME \
--query SecretString --output text
  • SCT CLI Command: When using the SCT CLI to add a target Redshift server, you can specify the --db-user and --db-password options to provide the database credentials. Here's a sample command:
sct-cli add-target \
--db-type redshift \
--server-name REDSHIFT_ENDPOINT \
--db-name DATABASE_NAME \
--db-user DB_USERNAME \
--db-password DB_PASSWORD
  • Bypass AWS Profile: If the SCT CLI is defaulting to an AWS profile and you want to bypass it, you can try setting the AWS_PROFILE environment variable to a non-existent profile before running the SCT CLI. This way, the CLI won't be able to use any AWS profile. For example:
export AWS_PROFILE=non_existent_profile
  • Documentation: Always refer to the official AWS documentation or the SCT CLI help command to get more details on the available options and how to use them.

  • Consider Raising a Support Ticket: If you're still facing issues or if the behavior of the SCT CLI is not as expected, consider raising a support ticket with AWS. They might provide insights specific to your account setup or any recent changes in the tool's behavior.

  • Alternative Tools: If you're open to using other tools, consider AWS Data Migration Service (DMS) which also supports migrating from Teradata to Redshift. DMS provides a different set of CLI commands and might offer the flexibility you're looking for.

Remember, when working with database credentials, always ensure they are kept secure and not hard-coded in scripts or exposed in logs.

profile picture
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions