1 Answer
- Newest
- Most votes
- Most comments
0
Hi ,
please note that , as by the documentation page you linked, what you are experiencing is the default behaviuor:
You can also set the updateBehavior
value to LOG
if you want to prevent your table schema from being overwritten, but still want to add the new partitions. The default value of updateBehavior
is UPDATE_IN_DATABASE
, so if you don’t explicitly define it, then the table schema will be overwritten.
The code should look like:
additionalOptions = {
"enableUpdateCatalog": True,
"updateBehavior": "LOG"}
additionalOptions["partitionKeys"] = ["partition_key0", "partition_key1"]
sink = glueContext.write_dynamic_frame_from_catalog(frame=last_transform, database=<dst_db_name>,
table_name=<dst_tbl_name>, transformation_ctx="write_sink",
additional_options=additionalOptions)
job.commit()
Relevant content
- asked 7 months ago
- asked 7 months ago
- Accepted Answerasked 4 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated a year ago
Hi,
Thanks for your answer, but I tried with "updateBehavior": "LOG", and the table description and column comments were updated.
thank you for the feed back I will research it a bit more