I encountered this error when writing a large size Parquet file onto an S3 bucket, "com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool"
I included the following, but still the job stops.
[
{
"Properties": {
"fs.s3.maxConnections": "500",
}
}
]
When I reduce the size of the data, everything works fine.
Any insights would be very much appreciated. Thanks
Hi, Please tell me the code of the writing part and the size of the file you want to upload. Do you reproduce the problem 100%? Or do you reproduce it once in a while?