Update/Create Governed Table from Glue job

Hello,

Answering to your queries:

1. Is it possible to create or update Governed table from Glue job ?

Yes, it is possible for your to create and update Governed table from Glue job using the CreateTable and UpdateTable API calls. In addition, there are restrictions on the UpdateTable API operation. You can't update the table type, change partition keys, or change the table location.

2. How crawler can create Governed table ?

No, crawler does not support Governed tables.

Please refer to https://docs.aws.amazon.com/lake-formation/latest/dg/governed-table-restrictions.html for complete details on the restrictions. You can view the answers for your above queries in this link as well. It also provides the details on what APIs are not allowed on the governed tables. I hope this information helps and answers your queries.

Thank you!

You cannot add the TableType option in the setCatalogInfo() method. If you know the column names at this stage, you could use createTable or updateTable methods suggested by the other answerer.

As you have mentioned that you would not know the column names, my suggestion is to use the AWS Wrangler library.

awswrangler.s3.to_json

wr.s3.to_json(
    df=df,
    path=f"s3://{bucket}/{database}/{table}/",
    dataset=True,
    database=database,
    table=table,
    table_type="GOVERNED",
    transaction_id=transaction_id)