Error trying to load data from SQL Server "decimal" type

0

I have a Glue job that's pulling data from a SQL Server view and trying to create/update a table definition in the data catalog. If I include one of the "decimal" type fields, I keep getting this:

An error occurred while calling o132.pyWriteDynamicFrame. Schema cannot be updated: data type has been changed, new data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(11,4)) does not match old data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(10,4))

It varies from DECIMAL(9,4) / DECIMAL(10,4) and the error above, so it looks like it defines the type when it sees one record, and then when it finds another record that exceeds the previously defined type, it crashes.

Any suggestions of a way for me to define everything on the database side and just let Glue create a type that's big enough for the whole table to be loaded? I tried adding "CAST ... AS DECIMAL(38,4)" on the DB side but nothing changes.

질문됨 일 년 전263회 조회
1개 답변
0

I found a workaround which is to use a SQL Query filtering and forcing the conversion there, by specifying all the fields in the view and using CAST ... AS DECIMAL(38,4) there. Is there a way for me to do this by default to all fields of that type just so I don't have to hardcode field names?

답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠