AWS Glue Redshift annotation

0

Hello everyone.

I am trying to generate a classic script through Glue Studio to get the annotations of my source, transformations and sink in one of my jobs, so far I have been able to get without problem the annotations of my source and transformations but I can't get the annotations of the sink. My sink is simply to write data in a Redshift table, then I am putting in Glue Studio "Data Target - Redshift" but at the moment of generating the classic script I get the following error: "Error generating the script [gluestudio-service.[MASKED].amazonaws.com] createScript: InvalidInputException: Invalid DataSink: DataSink(name=Amazon Redshift, classification=DataSink, type=Redshift, inputs=[node-2], isSinkInStreamingDAG=false)".

Has anyone else had this happen to you?.

Or do you have the correct annotation for Redshift sink?.

asked 10 months ago57 views
1 Answer
0

Hello,

I have tested by generating a sample script by just selecting S3 as input/source and Redshift as target/sink and I do not see annotation generated in the script for Redshift where as a regular Glue ETL job generated annotation as below:

=================

@type: DataSink

@args: [catalog_connection = "redshiftconnection", connection_options = {"dbtable": "testtable", "database": "sample"}, redshift_tmp_dir = TempDir, transformation_ctx = "datasink4"]

@return: datasink4

@inputs: [frame = dropnullfields3]

datasink4 = glueContext.write_dynamic_frame.from_jdbc_conf(frame = dropnullfields3, catalog_connection = "redshiftconnection", connection_options = {"dbtable": "testtable", "database": "sample"}, redshift_tmp_dir = args["TempDir"], transformation_ctx = "datasink4")

=================

Confirmed with Glue expert that this is normal behavior with Glue Studio and please raise a support ticket with us with your use case and if it's a blocker, we will raise a PFR for it with our Glue service team.

SUPPORT ENGINEER
answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions