- Newest
- Most votes
- Most comments
This is not a permission issue. If you are running in Amazon Managed Service for Apache Flink, this is a false positive.
Checkpointing and application processing are unaffected. You can safely ignore it.
You can verify that checkpointing is working, looking at the Flink Dashboard, in the Checkpoint history pane, and looking at the numberOfFailedCheckpoints metric in CloudWatch
answered 2 years ago
This warning is nothing to worry about and can be safely.
It is not related to IAM permissions and does not affect application checkpiinting.
You can verify checkpoints are working correctly from the Flink Dashboard and verifying the metric numberOfFailedCheckpoints is not actually increasing
answered a year ago
For testing, I run the flink application with the role created using the IAM role '' create or update via required policy, and get the same error.
answered 2 years ago
Hi AWS, I'm having same issue post upgrade from flink version runtime 1.15 to 1.18, is this something you can support ? We have a IAM role configured properly for our Flink application. if these buckets are managed by AWS, AWS it should manage the permissions. do we need to add some additional permission for our IAM role ? we tried adding delete permission to that additional internal bucket but no luck. Please assist !
answered a year ago
The error you're encountering suggests that the IAM role you've created and assigned to your Amazon Managed Service for Apache Flink application doesn't have sufficient permissions to perform the s3:DeleteObject action on the specified S3 bucket. This action is necessary for Flink to manage checkpoints effectively.
You're correct that Amazon Managed Service for Apache Flink uses managed S3 buckets for storing application state and checkpoints. These buckets are indeed managed by AWS, but your application's IAM role needs the appropriate permissions to interact with them.
To resolve this issue, you need to update the IAM role's policy to include the necessary S3 permissions. Specifically, you should add the s3:DeleteObject permission, along with other relevant S3 actions. Here's what you should do:
-
Open the IAM console and locate the role you created for your Flink application.
-
Edit the role's permissions policy.
-
Add the following S3 actions to the policy:
- s3:DeleteObject
- s3:GetObject
- s3:PutObject
- s3:ListBucket
-
Ensure that the Resource section of the policy includes the ARN of the S3 bucket used for checkpoints. It should look something like this: "Resource": [ "arn:aws:s3:::c8346d49dd06d3204eee7db29037b65e4e65bb57/*", "arn:aws:s3:::c8346d49dd06d3204eee7db29037b65e4e65bb57" ]
-
Save the updated policy.
After making these changes, your Flink application should have the necessary permissions to perform checkpoints successfully. If you continue to experience issues, you may need to review other aspects of your IAM configuration or consult AWS support for further assistance.
Sources
Troubleshooting Amazon Managed Service for Apache Flink identity and access - Managed Service for Apache Flink
Identity-based policy examples for Amazon Managed Service for Apache Flink - Managed Service for Apache Flink
Create and run the application (CLI) - Managed Service for Apache Flink
answered 2 years ago
I already have S3FullAccessPolicy in my permissions.
Yes I could verify in logs and metric numberOfFailedCheckpoints that there is no issues with checkpoints, but our main concern is whether such deletion process is for clean-up old checkpoints ? Not able to delete/clean-up old checkpoints as per design it sounds a concern to me.
This warning is transient. There is nothing to worry about it. Checkpoints are fully managed in Managed Flink. Checkpoints are not stored in customer's S3 bucket, not affected by the application IAM role, and there is no direct cost for them. The service takes care of clearing old checkpoints, asynchronously.