Error trying to load data from SQL Server "decimal" type

0

I have a Glue job that's pulling data from a SQL Server view and trying to create/update a table definition in the data catalog. If I include one of the "decimal" type fields, I keep getting this:

An error occurred while calling o132.pyWriteDynamicFrame. Schema cannot be updated: data type has been changed, new data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(11,4)) does not match old data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(10,4))

It varies from DECIMAL(9,4) / DECIMAL(10,4) and the error above, so it looks like it defines the type when it sees one record, and then when it finds another record that exceeds the previously defined type, it crashes.

Any suggestions of a way for me to define everything on the database side and just let Glue create a type that's big enough for the whole table to be loaded? I tried adding "CAST ... AS DECIMAL(38,4)" on the DB side but nothing changes.

已提問 1 年前檢視次數 268 次
1 個回答
0

I found a workaround which is to use a SQL Query filtering and forcing the conversion there, by specifying all the fields in the view and using CAST ... AS DECIMAL(38,4) there. Is there a way for me to do this by default to all fields of that type just so I don't have to hardcode field names?

已回答 1 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南