Timeout Error while uploading files to S3 Bucket using OpenShift Cluster

Hello Arjun,

I would like to just give general ideas of this issue since I am working with little information here. You mentioned the timeouts happen in the afternoons and at nights.

This brings up some questions like, does the write rate increase at these hours of the day or is the write consistent through out the day? If the rate of writing to bucket increases then it might just mean you are hitting default s3 limits.

For example, your application can achieve at least 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per partitioned Amazon S3 prefix and in case you are hitting these limits, you most likely will get timeout errors as Amazon S3 scales to high request rates.

If scaling is the issue here, then you can basically implement retry mechanism for these put requests. Moreover, you can design your s3 to be performant by increasing the number of prefixes.

You can increase your read or write performance by using parallelization. For example, if you create 10 prefixes in an Amazon S3 bucket to parallelize reads, you could scale your read performance to 55,000 read requests per second. Similarly, you can scale write operations by writing to multiple prefixes.

The scaling, in the case of both read and write operations, happens gradually and is not instantaneous. While Amazon S3 is scaling to your new higher request rate, you may see some 503 (Slow Down) errors. These errors will dissipate when the scaling is complete.

Read more about this here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html