How can I run a post action script while writing to redshift from aws glue?

0
        df.toDF().write.format("jdbc").\
        option("url", "").\
        option("dbtable", f"public.{tableName}_staging").\
        option("user", "").\
        option("password", "").\
        option("postactions", f"DROP TABLE IF EXISTS $staging").\
        mode('append').save()

I have mention the post actions script in the option but it is still not executed, is there a different way to execute the post action script from redshift in aws glue notebook I used the script from https://stackoverflow.com/questions/49735489/upsert-from-aws-glue-to-amazon-redshift

질문됨 일 년 전351회 조회
1개 답변
0

Please note postaction is a feature of Glue DynamicFrame while you are converting there to a standard Spark DataFrame (toDF()) and using the standard JDBC driver instead of the Glue native Redshift connector.

My advice is that you write that DynamicFrame using a sink provided by GlueContext (like in the example you reference). Otherwise you would have to open your own connector to Redshift to run that DROP DDL (e.g. opening a JDBC connection or a Python library like psycopg2)

profile pictureAWS
전문가
답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠