By using AWS re:Post, you agree to the Terms of Use
/Lake Formation Governed tables: truncate and reload?/

Lake Formation Governed tables: truncate and reload?

0

Doing a POC with LF Governed table for a small dimension. I'd like to purge and reload the table each time it runs. I have tried to include a purge_table command within a governed table transaction but I receive the below error. I've tried adding a transactionId param to the purge_table command directly or within the options dict and neither has worked. Any ideas? Or anyone found a good way to truncate and reload a governed table?

"An error occurred while calling o119.purgeTable. Specify Either Transaction Id or Query AsOf Time"

asked 4 months ago40 views
1 Answers
0

Hello,

I tried to purge the table and reload but unable to reload the table. Could you please share the purge_table command you are using for us to test ? Also, we recommend you to please open a support ticket with us so that we can get on a screen-share and assist you by checking your Lake Formation configuration and parameters used.

SUPPORT ENGINEER
answered 4 months ago
  • Thanks for your reply. See below. I'll also open a support ticket. Thank you

    dest_path = <s3 path> db = <database name> tbl = <table name>

    tx_id = glue_context.start_transaction(False)

    sink = glue_context.getSink( connection_type="s3", path=dest_path, enableUpdateCatalog=True, transactionId=tx_id ) sink.setFormat("glueparquet") sink.setCatalogInfo( catalogDatabase=db, catalogTableName=tbl )

    try: glue_context.purge_table(db, tbl, options={'transactionId':tx_id}) sink.writeFrame(glue_data_frame) glue_context.commit_transaction(tx_id) except Exception: glue_context.cancel_transaction(tx_id) raise job.commit()

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions