1 Answer
- Newest
- Most votes
- Most comments
0
I understand that, your use case is to update the schema of the data catalog. So, for that you tried using the below code:
silver_target = self.glue_context.getSink( path=silverLakeLocation + table_name, connection_type="s3", updateBehavior="UPDATE_IN_DATEBASE", partitionKeys=["pyear"], enableUpdateCatalog=True, transformation_ctx="silver_target")
Please note that, the method that you have used above is used for creating new tables and I do not think that it would be updating the existing schema. I request you to refer this documentation in which it is stated that you need to make use of the following code for updating your schema:
additionalOptions = {
"enableUpdateCatalog": True,
"updateBehavior": "UPDATE_IN_DATABASE"}
additionalOptions["partitionKeys"] = ["partition_key0", "partition_key1"]
sink = glueContext.write_dynamic_frame_from_catalog(frame=last_transform, database=<dst_db_name>,
table_name=<dst_tbl_name>, transformation_ctx="write_sink",
additional_options=additionalOptions)
job.commit()
Alternatively, you can directly go ahead and update the schema of your Glue catalog table from the AWS Glue console itself.
- -> Login to your AWS Glue console and click on tables.
- -> Open your required table and then Actions > Edit schema
- -> Select the required column to be deleted and then click on Delete button.
I hope this answers your question.
Relevant content
- asked 2 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 years ago