Breaking Changes Checklist: Migrating AWS Glue Jobs to Version 5.0/5.1

AWS Glue versions 0.9, 1.0, and 2.0 reached end of life on April 1, 2026. To continue running your ETL workloads with support, you must migrate your jobs to a supported version. AWS Glue 5.0 and 5.1 are the recommended migration targets.

This article provides a comprehensive checklist of breaking changes organized by category. Use it to assess the scope of migration work required for your specific jobs based on their current Glue version.

Target version reference:

Version	Spark	Python	Java	S3 Connector
5.0 (GA)	3.5.4	3.11	17	EMRFS
5.1 (GA)	3.5.6	3.11	17	S3A

Critical constraints

Before you begin your migration, review these constraints that affect your migration path:

Two-hop requirement (Glue 0.9 and 1.0 only)

Jobs running on Glue 0.9 or 1.0 cannot migrate directly to 5.x. The AWS Encryption SDK 1.x is incompatible with version 2.x. You must first upgrade these jobs to Glue 2.0, 3.0, or 4.0 and run them at least once to bridge the encryption format before upgrading to 5.x.

S3A default connector (Glue 5.1 only)

Glue 5.1 uses S3A instead of EMRFS as the default S3 connector. If you don't explicitly set spark.hadoop.fs.s3a.endpoint.region, S3A defaults to us-east-2. This causes timeouts for jobs running in other AWS Regions or within a VPC.

Resolution: Add the following Spark configuration to your job:

--conf spark.hadoop.fs.s3a.endpoint.region=<your-region>

Iceberg V3 cross-engine compatibility (Glue 5.1 only)

Glue 5.1 can write Iceberg format V3 tables. Amazon Athena cannot read V3 tables. If your downstream consumers query Iceberg tables from Athena, pin your table format to V2 or use Glue 5.0 until Athena adds V3 support.

Applicability matrix

Use the following matrix to determine which breaking changes apply based on your source Glue version. Check only the changes that apply to your source-to-target pair.

Change ID	Description	0.9 to 5.x	1.0 to 5.x	2.0 to 5.x	3.0 to 5.x	4.0 to 5.x
C01	Spark version upgrade	Yes	Yes	Yes	Yes	Yes
C02	Scala 2.11 to 2.12	Yes	Yes	Yes	-	-
C04	Python upgrade to 3.11	Yes	Yes	Yes	Yes	Yes
C05	Python 2 removal	Yes	Yes	Yes	-	-
C08	Java 8 to 17	Yes	Yes	Yes	Yes	Yes
C11	YARN removal	Yes	Yes	-	-	-
C13	MaxCapacity to Workers	Yes	Yes	-	-	-
C15	Log4j to Log4j2	Yes	Yes	Yes	Yes	-
C22	Parquet timestamp rebase	Yes	Yes	Yes	Partial	-
C25	Encryption SDK two-hop	Yes	Yes	-	-	-
C33	S3A default (5.1 only)	Yes	Yes	Yes	Yes	Yes
C43-C47	JDBC driver upgrades	Yes	Yes	Yes	Yes	Yes
C53	Dropped connectors	Yes	Yes	Yes	Yes	Yes
C59	Iceberg DROP TABLE semantics	Yes	Yes	Yes	Yes	Yes
C62	Iceberg V3 cross-engine	5.1	5.1	5.1	5.1	5.1
C64	Worker model required	Yes	Yes	-	-	-

Runtime, engine, and language changes

C01 - Spark version upgrade (all source versions)

Glue 5.0 uses Spark 3.5.4 and Glue 5.1 uses Spark 3.5.6. Review the Apache Spark migration guides for each minor version between your current Spark version and 3.5.x. Test SQL behavior changes such as type casting, null handling, and partition pruning.

C02 - Scala 2.11 to 2.12 (from 0.9, 1.0, 2.0)

Recompile all Scala jobs and custom JARs against Scala 2.12. Binary compatibility is not maintained between Scala minor versions.

C04 - Python upgrade to 3.11 (all source versions)

Fix the following Python 2/3 incompatibilities in your scripts:

• print statements (use print() function)
• unicode and xrange (removed in Python 3)
• dict.keys(), dict.values(), filter(), map() return views/iterators instead of lists
• / performs true division (use // for integer division)
• String handling differences (str vs bytes)

C05 - Python 2 removal (from 0.9, 1.0, 2.0)

Spark 3.3 and later drops Python 2 support entirely. All scripts must be Python 3 compatible.

C06 - Python 3.6 removal (from 1.0, 2.0)

Spark 3.3 and later drops Python 3.6 support. Update code that uses deprecated stdlib modules or f-string syntax not available in 3.7+.

C07 - Scala untyped UDFs disabled (from 0.9, 1.0, 2.0, 3.0)

Untyped Scala UDFs are no longer permitted. Re-implement them as typed UDFs.

C08 - Java 8 to 17 (all source versions)

Java 17 restricts reflective access to internal APIs. Add --add-opens flags to your Spark configuration for any libraries that use reflection. Review and update any deprecated Java APIs that were removed between Java 8 and 17.

C09 - AWS SDK for Java v1 to v2 (all source versions, Scala/Java only)

Glue 5.0 ships both AWS SDK v1 (1.12.569) and v2 (2.29.52). Glue 5.1 ships SDK v2 (2.35.5). Migrate custom Scala or Java code to SDK v2.

C10 - Boto3 upgrade (all source versions)

Glue 5.0 ships boto3 1.34.131 and Glue 5.1 ships boto3 1.40.61. Remove any boto2 (legacy) usage.

C11 - YARN removal (from 0.9, 1.0)

Remove all spark.yarn.* configuration properties from your job parameters. These are no longer recognized.

C12 - HDFS removal (from 0.9, 1.0)

HDFS is not available. Use Amazon S3 for all intermediate data storage.

C13 - Resource manager model (from 0.9, 1.0)

Drop the MaxCapacity parameter from your job definitions. Set WorkerType (for example, G.1X, G.2X) and NumberOfWorkers instead.

Logging changes

C15 - Log4j to Log4j2 (from 0.9, 1.0, 2.0, 3.0)

Rename log4j.properties to log4j2.properties and update the syntax to Log4j2 format. For example:

# Old Log4j format
log4j.rootLogger=WARN, console

# New Log4j2 format
rootLogger.level = WARN
rootLogger.appenderRef.console.ref = console

C16 - Custom log4j properties removed (all source versions)

Custom log4j.properties files uploaded to S3 are not supported in Glue 5.0 and later. Use the Log4j2 equivalent or configure logging through Spark configuration.

C17 - Continuous logging arguments removed (all source versions)

Remove the following job arguments:

• --enable-continuous-cloudwatch-log
• --continuous-log-logGroup
• --continuous-log-logStreamPrefix
• --continuous-log-conversionPattern
• --enable-continuous-log-filter

Continuous logging is now the default behavior.

C18 - Log group consolidation (all source versions)

All job logs now write to /aws-glue/jobs/error. Update any CloudWatch alarms, metrics filters, or log consumers that depend on the previous log group structure.

C19 - Job insights log stream moved (all source versions)

The job-insights-rca-driver log stream now writes to the error log group. Update references if you consume this stream directly.

Parquet, timestamps, and datetime changes

C22 - Parquet timestamp rebase (from 0.9, 1.0, 2.0; partial for 3.0)

Set the following Spark properties to maintain backward compatibility with existing Parquet data:

--conf spark.sql.legacy.parquet.int96RebaseModeInRead=CORRECTED
--conf spark.sql.legacy.parquet.int96RebaseModeInWrite=CORRECTED
--conf spark.sql.legacy.parquet.datetimeRebaseModeInRead=CORRECTED

Use LEGACY instead of CORRECTED if your downstream consumers expect the pre-Gregorian calendar behavior.

C23 - Proleptic Gregorian calendar (from 0.9, 1.0, 2.0)

Dates before October 15, 1582 may shift due to the calendar change in Spark 3.0 and later. Validate any historical date data in your tables.

C24 - Timestamp partition keys (from 0.9, 1.0, 2.0)

If you use timestamp columns as partition keys, repartition them to string type to avoid unexpected behavior.

Security and encryption changes

C25 - AWS Encryption SDK two-hop (from 0.9, 1.0)

This is a hard migration constraint. You cannot upgrade from 0.9 or 1.0 directly to 5.x. Run your jobs on an intermediate version (2.0, 3.0, or 4.0) at least once to bridge the encryption format before upgrading to 5.x.

C26 - Lake Formation fine-grained access control requires DataFrames (all source versions)

Spark-native fine-grained access control (FGAC) works with DataFrames only. DynamicFrame-based jobs that rely on FGAC must be converted to use DataFrames.

C29 - Lake Formation FTA native path for open table formats (all source versions)

DynamicFrame with FTA (fine-grained table access) does not work with Iceberg, Delta Lake, or Hudi tables. Use DataFrames for open table format operations under Lake Formation governance.

File system and S3 connector changes

C33 - Default S3 connector changed to S3A (Glue 5.1 only)

Glue 5.1 uses S3A instead of EMRFS by default. Without explicit region configuration, S3A defaults to us-east-2 and causes timeouts in other Regions or VPC environments.

Resolution: Add this to your job configuration:

--conf spark.hadoop.fs.s3a.endpoint.region=<your-region>

C34 - Opt back to EMRFS (non-breaking, optional)

If you prefer to continue using EMRFS on Glue 5.1, set the following properties:

--conf spark.hadoop.fs.s3.impl=com.amazon.ws.emr.hadoop.fs.EmrFileSystem
--conf spark.hadoop.fs.s3n.impl=com.amazon.ws.emr.hadoop.fs.EmrFileSystem
--conf spark.hadoop.fs.AbstractFileSystem.s3.impl=org.apache.hadoop.fs.s3.EMRFSDelegate

C37 - S3A region quirk in Glue 5.0 (if opting in to S3A on 5.0)

If you explicitly enable S3A on Glue 5.0, set the endpoint region property as described in C33.

JDBC driver changes

All JDBC drivers are upgraded in Glue 5.0 and 5.1. Review your connection strings, authentication methods, and driver-specific options.

Change ID	Database	New Version	Key Breaking Changes
C43	MySQL	8.0.33	Review connection string and authentication changes
C44	Microsoft SQL Server	10.2.0	TLS enabled by default; aadSecurePrincipalId/Secret deprecated; getAADSecretPrincipalId API removed
C45	Oracle	23.3.0.23.09	Review Oracle thin driver migration notes
C46	PostgreSQL	42.7.3	Review parameter and behavior changes
C47	Redshift	redshift-jdbc42-2.1.0.29	Upgraded from redshift-jdbc41-1.2.12.1017; JDBC 4.2 API required

Connector changes

C50 - Redshift connector upgrade (all source versions)

The Redshift connector is version 6.4.0 in Glue 5.0 and 6.4.2 in Glue 5.1. New features include three-part names, CNAME support, and ShortType to SMALLINT mapping. Review your connector options for compatibility.

C51 - MongoDB connector upgrade to 10.3.0 (all source versions)

The following property names changed:

• uri is now connection.uri
• Escape commas, asterisks, and backslashes in collection names

C52 - Snowflake connector upgrade (all source versions)

Glue 5.0 uses connector version 3.0.0 and Glue 5.1 uses 3.1.1. Key changes:

• SFUSER is no longer required when using OAUTH authentication
• Advanced Query Pushdown is removed

C53 - Athena-based custom and marketplace connectors dropped (all source versions)

Athena-based custom connectors and marketplace connectors are not supported in Glue 5.0 and later. Replace them with native Glue connectors or Spark-native alternatives.

C54 - SaaS connectors dropped (all source versions)

The following connectors are removed in Glue 5.0 and later:

• Salesforce
• Google Ads, Google Analytics, Google Sheets
• Hubspot
• Jira Cloud
• ServiceNow
• Slack
• SAP OData
• NetSuite
• Zendesk
• Stripe
• Marketo
• Facebook Ads, Instagram Ads, Snapchat Ads
• Intercom
• Zoho CRM
• Salesforce Marketing Cloud (including Pardot)

If you use any of these connectors, you must find an alternative data ingestion method before migrating.

C55 - Vectorized SIMD CSV reader dropped (all source versions)

The vectorized SIMD CSV reader is not supported in Glue 5.0 and later. Remove any opt-in configuration for this feature.

Open table format changes

C57 - Hudi upgrade (all source versions)

Glue 5.0 ships Apache Hudi 0.15.0 and Glue 5.1 ships Hudi 1.0.2. Tables created before Hudi 0.10.0 without a primaryKey table property must be recreated.

C58 - Delta Lake upgrade (non-breaking)

Glue 5.0 uses Delta Lake 3.3.0 and Glue 5.1 uses 3.3.2. No breaking changes.

C59 - Iceberg upgrade with DROP TABLE semantics change (all source versions)

Glue 5.0 uses Apache Iceberg 1.7.1 and Glue 5.1 uses Iceberg 1.10.0. The DROP TABLE command no longer deletes underlying data files. Use DROP TABLE ... PURGE if you want to delete data files along with the table definition.

C62 - Iceberg V3 cross-engine compatibility (Glue 5.1 only)

Athena cannot read Iceberg V3 tables written by Glue 5.1. If Athena is a downstream consumer of your Iceberg tables, pin the table format version to V2.

Job configuration and API changes

C63 - GlueVersion parameter required (all source versions)

Explicitly set GlueVersion to 5.0 or 5.1 in your job definition. The parameter is mandatory.

C64 - Worker model required (from 0.9, 1.0)

Drop MaxCapacity from your job definition. Use WorkerType and NumberOfWorkers instead.

C67 - ML transforms removed (from 0.9, 1.0, 2.0, 3.0)

Machine Learning transforms (FindMatches) are not supported in Glue 4.0 and later. Migrate ML workloads to Amazon SageMaker or implement the logic in custom code.

PySpark-specific changes

C72 - Arrow UDF configuration renamed (all source versions)

Rename the following Spark property:

# Old
spark.sql.execution.pythonUDF.arrow.enabled

# New
spark.sql.execution.arrow.pyspark.enabled

C73 - Array schema inference (all source versions)

Spark 3.4 and later merges schemas across all array elements. To restore the previous behavior (infer from the first element only), set:

--conf spark.sql.pyspark.legacy.inferArrayTypeFromFirstElement.enabled=true

C74 - DynamicFrame predicates on empty tables (all source versions)

If your jobs use push-down predicates with DynamicFrame on tables that can be empty, upgrade the internal PythonGluePackagingTool from version 1.1 to 3.0.

C75 - pandas minimum version (all source versions)

The minimum pandas version is 1.0.5. Update any pinned versions in your --additional-python-modules or requirements.txt.

Migration strategy recommendations

From Glue 0.9 or 1.0 (two-hop required)

1. Upgrade jobs to Glue 3.0 or 4.0 first.
2. Run all jobs at least once on the intermediate version to bridge the Encryption SDK.
3. Address Python 2 removal, YARN configuration, HDFS usage, and the worker model.
4. Then upgrade to Glue 5.0 or 5.1.

From Glue 2.0

1. Upgrade directly to Glue 5.x.
2. Focus on: Python 3.11 compatibility, Parquet timestamp rebase, Log4j2 migration, and Java 17.

From Glue 3.0 or 4.0

1. Upgrade directly to Glue 5.x.
2. Focus on: Java 8 to 17, JDBC driver changes, connector updates, and Iceberg semantics.
3. If targeting 5.1, address the S3A connector change.

General recommendations

• Test migrations with non-production workloads first.
• Start with leaf jobs that have no downstream dependencies for lowest risk.
• Set spark.hadoop.fs.s3a.endpoint.region explicitly for all Glue 5.1 migrations.
• Review all JDBC connection strings against the new driver versions.
• Verify that no dropped SaaS connectors or marketplace connectors are in use.
• If your Iceberg tables are queried from Athena, stay on V2 format or use Glue 5.0.

Related information

• AWS Glue version support policy
• Migrating AWS Glue for Spark jobs to newer versions
• Apache Spark migration guide
• AWS Glue release notes
• Using the Iceberg framework in AWS Glue