Intermittent S3 403 forbidden access errors on multipart object

0

Hi team, I have a spark process that uses s3 as object storage. This application runs frequently ( multiple times a day) writing thousands of objects on a S3 encrypted bucket as part of a Hive external table. It is using multipart upload to write them in a component which is not handled by us ( to give brief context: S3 "magic" committers).

The process works for 99% of times, but from time to time, there are intermittent errors that are impacting the object write, making the application to fail.

should I provide a extended request id to troubleshoot the issue?

thank you.

1개 답변
0

Could it be that your credentials are expiring and being replaced in the middle of the multi-part upload?

profile pictureAWS
전문가
kentrad
답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠