Error trying to load data from SQL Server "decimal" type

0

I have a Glue job that's pulling data from a SQL Server view and trying to create/update a table definition in the data catalog. If I include one of the "decimal" type fields, I keep getting this:

An error occurred while calling o132.pyWriteDynamicFrame. Schema cannot be updated: data type has been changed, new data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(11,4)) does not match old data type: [amountAllocated] optional int64 amountAllocated (DECIMAL(10,4))

It varies from DECIMAL(9,4) / DECIMAL(10,4) and the error above, so it looks like it defines the type when it sees one record, and then when it finds another record that exceeds the previously defined type, it crashes.

Any suggestions of a way for me to define everything on the database side and just let Glue create a type that's big enough for the whole table to be loaded? I tried adding "CAST ... AS DECIMAL(38,4)" on the DB side but nothing changes.

posta un anno fa270 visualizzazioni
1 Risposta
0

I found a workaround which is to use a SQL Query filtering and forcing the conversion there, by specifying all the fields in the view and using CAST ... AS DECIMAL(38,4) there. Is there a way for me to do this by default to all fields of that type just so I don't have to hardcode field names?

con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande