How to update the columns of a Glue table defined in a CloudFormation template?

Question

I've defined a Glue table in a CloudFormation template and now I want to add a column. When I update the template with the new column CloudFormation reports: "Table name cannot be equal to null or empty". I did not specify a name for the table in the template, so when CloudFormation originally created the table is used the resource name with a random suffix. I've tried to add the actual deployed table name into the template, but then CloudFormation reports: "Table already exists."

How do I get CloudFormation to update the table columns? Do I need to create a fresh table and define the table name in the template?

Example template:
```
    {
        "Description": "CloudFormation stack for testing Glue Table updates",
        "Resources": {
            "TestGlueBucket": {
                "Type": "AWS::S3::Bucket",
                "Properties": {
                    "BucketName": "qwerty-test-glue-bucket",
                    "PublicAccessBlockConfiguration": {
                        "BlockPublicAcls": "True",
                        "BlockPublicPolicy": "True",
                        "IgnorePublicAcls": "True",
                        "RestrictPublicBuckets": "True"
                    }
                }
            },
            "TestGlueDatabase": {
                "Type": "AWS::Glue::Database",
                "Properties": {
                    "CatalogId": {
                        "Ref": "AWS::AccountId"
                    },
                    "DatabaseInput": {
                        "Description": "Test database"
                    }
                }
            },
            "TestGlueTable": {
                "Type": "AWS::Glue::Table",
                "Properties": {
                    "CatalogId": {
                        "Ref": "AWS::AccountId"
                    },
                    "DatabaseName": {
                        "Ref": "TestGlueDatabase"
                    },
                    "TableInput": {
                        "TableType": "EXTERNAL_TABLE",
                        "Parameters": {
                            "classification": "parquet",
                            "typeOfData": "file"
                        },
                        "PartitionKeys": [
                            {
                                "Name": "partition_0",
                                "Type": "string",
                                "Comment": "Year"
                            },
                            {
                                "Name": "partition_1",
                                "Type": "string",
                                "Comment": "Month"
                            },
                            {
                                "Name": "partition_2",
                                "Type": "string",
                                "Comment": "Day"
                            }
                        ],
                        "StorageDescriptor": {
                            "Location": {
                                "Ref": "TestGlueBucket"
                            },
                            "InputFormat": "org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat",
                            "OutputFormat": "org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat",
                            "SerdeInfo": {
                                "SerializationLibrary": "org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe",
                                "Parameters": {
                                    "serialization.format": "1"
                                }
                            },
                            "Columns": [
                                {
                                    "Name": "data_timestamp",
                                    "Type": "timestamp"
                                },
                                {
                                    "Name": "data_type",
                                    "Type": "string"
                                },
                                {
                                    "Name": "data_value",
                                    "Type": "double"
                                }
                            ]
                        }
                    }
                }
            }
        }
    }
```

**Addition [2025-05-30 13:03 UTC]**: Using the CLI `aws glue update-table` command works completely fine with the following `--table-input` structure

```
{
    "Name": "testgluetable",
    "TableType": "EXTERNAL_TABLE",
    "Parameters": {
        "classification": "parquet",
        "typeOfData": "file"
    },
    "PartitionKeys": [
        {
            "Name": "partition_0",
            "Type": "string",
            "Comment": "Year"
        },
        {
            "Name": "partition_1",
            "Type": "string",
            "Comment": "Month"
        },
        {
            "Name": "partition_2",
            "Type": "string",
            "Comment": "Day"
        }
    ],
    "StorageDescriptor": {
        "Columns": [
            {
                "Comment": "Timestamp in UTC format",
                "Name": "data_timestamp",
                "Type": "timestamp"
            },
            {
                "Name": "data_type",
                "Type": "string"
            },
            {
                "Name": "data_value",
                "Type": "double"
            },
            {
                "Name": "created_at",
                "Type": "timestamp"
            }
        ]
    }
}
```

re:Post Agent · Answer

To update the columns of a Glue table defined in a CloudFormation template, you need to include the table name in your TableInput properties. When you're updating an existing table, you must specify the same table name that was created during the initial deployment.

The issue you're encountering is related to how CloudFormation handles updates to AWS Glue tables. When you don't specify a name in the TableInput properties, CloudFormation generates one using the resource name with a random suffix. During updates, you need to maintain this name.

Here's how to fix your template:

1. First, retrieve the actual name of the deployed table from the AWS Glue console or using AWS CLI.

2. Then, add this name to your TableInput properties in the CloudFormation template:

```
"TableInput": {
"Name": "your-actual-table-name",
"TableType": "EXTERNAL_TABLE",
...
}
```

3. When you update the stack with this modified template, CloudFormation will update the existing table rather than trying to create a new one.

This approach allows you to add, modify, or remove columns while maintaining the same table. The key is ensuring the Name property in TableInput matches exactly with the deployed table name.

If you're still encountering issues, you might need to use the VersionId parameter in your update operation, which specifies the version ID at which to update the table contents. However, this is typically not required for simple column additions.  
**Sources**  
[UpdateTable - AWS Glue](https://docs.aws.amazon.com/glue/latest/webapi/API_UpdateTable.html)  
[Work with CSV data in AWS Glue - Amazon Athena](https://docs.aws.amazon.com/athena/latest/ug/schema-csv.html)  
[Examples of CloudFormation stack operation commands for the AWS CLI and PowerShell - AWS CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/service_code_examples.html)

How to update the columns of a Glue table defined in a CloudFormation template?

Add your answer

Relevant content