how do I stop Athena from rounding during calculations

0

Enter image description here Here is an example of the issue I'm experiencing with rounding in Athena. The first column has a datatype of decimal(13,3), the next 4 columns have datatypes of decimal(5,0), this is to mimic the source DB. The last column is calculated. in this example it would be 0.002 X 40 / 47 X 250 / 1, which should be 0.426, however as you can see instead I'm getting 0.500. I believe this is happening because it's rounding intermittently during the calculation. 0.002 X 40 = 0.080, 0.080 / 47 = 0.00170212765957... (I think is being rounded to 0.002), 0.002 X 250 = 0.500, 0.500 X 1 = 0.500 Does anybody know how to prevent this from happening? I am on Athena engine version 3.

Thanks!

asked 7 months ago225 views
1 Answer
1

Hello,

Greetings of the day!

I understand that you are seeing unexpected rounding off of your data in Athena when you were computing some data out of your decimal typed columns. Please note that, in Athena whenever you perform computation between two columns of different types then if the computation is compatible then the output would be cast into the higher datatype with higher precision. By this I mean that, when you performed the arithmetic operation between the decimal(13,3) and decimal(5,0) datatypes then the output would be coming up as the type of decimal(13,3). Now if the output is going beyond this precision then inherently it would be rounded up to fit it to decimal(13,3).

Hence, as you mentioned 0.002 X 40 = 0.080, 0.080 / 47 = 0.00170212765957... (I think is being rounded to 0.002), it is indeed being rounded up to 0.002. I have tested it on my test environment as well. As per the inherent logic 0.00170212765957 must be able to fit in decimal(13,3) but it is going beyond this thereby it is being rounded off to the nearest value which is 0.002.

You can try increasing the scale value of your decimal type to a higher number. Instead of decimal(13,3), I had tried with decimal(13,6) and it was able to give me value without much rounding off. Therefore, it might be better to set the decimal column to the required precision and scale.

Hope this helps! Thanks!

profile pictureAWS
SUPPORT ENGINEER
Chaitu
answered 7 months ago
  • Hi Chaitu, Thank you for your answer. I was hoping there was a way to only get the final answer to take on the higher precision, without rounding the intermediate calculations. But this makes sense, I appreciate your response.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions