Why does AWS Glue schema registry not register a new version when adding a default to existing Avro schema field?

Question

I am trying to add a default to an existing field in an Avro schema in AWS Glue, but the change isn't registering as a new version.

Is this behavior expected? If so, why? If not, how can I go about registering the new version?

Schema v1:
```
{
  "type": "record",
  "name": "test",
  "fields": [
    {"name": "testytest","type":"string"}
  ]
}
```

Steps:
1. "Register new version"
2. Add `,"default":"default"` after `string`
3. "Register schema version"

Expected Outcome:
* New version shows up under `Schema version`

Actual Result:
* Editor closes silently and no new version is present

AWS TAM · Answer

In Avro, adding a default value to an existing field is considered a backward-compatible change. This means that the new schema version can be read by consumers using the previous version of the schema.

Backward-compatible changes do not automatically create a new schema version in Glue. This is because the new schema is still compatible with the previous version, and consumers can continue to use the previous version without any issues.

To register the new schema version with the default value, you can use the
    UpdateSchema
    API operation in the AWS Glue API. This will create a new version of the schema in the registry.

Here's an example of how you can use the
UpdateSchema
API to create a new version:

aws glue update-schema \
  --schema-id '{"SchemaName": "my-schema-name", "RegistryName": "my-registry-name"}' \
  --schema-definition file://my-schema-definition.json

Replace
"my-schema-name"
,
"my-registry-name"
, and
"my-schema-definition.json"
with the appropriate values for your use case.

Ethan_H · Answer

I'm able to reproduce this behavior that after added`,"default":"default"` the new schema is not successfully saved.  Even if I disabled Compatibility check, the new version still failed to register. I'd try to reach out to our internal team to confirm the root cause of this. In the meanwhile, please feel free to cut a support ticket from your account to engage AWS support.

Why does AWS Glue schema registry not register a new version when adding a default to existing Avro schema field?

Add your answer

Relevant content