The python script works well read/write file to S3 with my local machine by aws-vault authentication, then I wanted to deploy this script on EC2.
However, this error pops up when I run the scripts on EC2 linux instance.
What the script does are
- Call the restAPI
- Data Transformation from Json to Pandas Dataframe
- Does some aggregation as report (I think this is where the error appears)
- Write back report back to S3
Traceback (most recent call last):
File "/home/ec2-user/intl_salesOrder_query.py", line 146, in <module>
orders_since_monthBeginning[float_columns] = orders_since_monthBeginning[float_columns].astype('float64').copy()
File "/home/ec2-user/miniconda3/lib/python3.9/site-packages/pandas/core/frame.py", line 3643, in __setitem__
self._setitem_array(key, value)
File "/home/ec2-user/miniconda3/lib/python3.9/site-packages/pandas/core/frame.py", line 3687, in _setitem_array
self[k1] = value[k2]
File "/home/ec2-user/miniconda3/lib/python3.9/site-packages/pandas/core/frame.py", line 3645, in __setitem__
self._set_item_frame_value(key, value)
File "/home/ec2-user/miniconda3/lib/python3.9/site-packages/pandas/core/frame.py", line 3775, in _set_item_frame_value
raise ValueError("Columns must be same length as key")
I tried to change the data type from string to numeric so that I can do further calculation. Following are the code snippets cause error
I'm not sure that's the issue - the error message indicates that there may be a formatting/data issue: "Columns must be the same length as key".