Error trying to load data from SQL Server "decimal" type

0

I have a Glue job that's pulling data from a SQL Server view and trying to create/update a table definition in the data catalog. If I include one of the "decimal" type fields, I keep getting this:

An error occurred while calling o132.pyWriteDynamicFrame. Schema cannot be updated: data type has been changed, new data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(11,4)) does not match old data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(10,4))

It varies from DECIMAL(9,4) / DECIMAL(10,4) and the error above, so it looks like it defines the type when it sees one record, and then when it finds another record that exceeds the previously defined type, it crashes.

Any suggestions of a way for me to define everything on the database side and just let Glue create a type that's big enough for the whole table to be loaded? I tried adding "CAST ... AS DECIMAL(38,4)" on the DB side but nothing changes.

已提问 1 年前245 查看次数
1 回答
0

I found a workaround which is to use a SQL Query filtering and forcing the conversion there, by specifying all the fields in the view and using CAST ... AS DECIMAL(38,4) there. Is there a way for me to do this by default to all fields of that type just so I don't have to hardcode field names?

已回答 1 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则