I was using aws console to set up all services
I have set up two firehose streams, the first one is perfectly fine but the second one is having this issue
Both streams are using the access role generated by aws console
After comparing both access roles, I found that two roles are slightly different
- First one has no or empty
Sid in both permissions and trusted entities
- Second one has
secretsmanager:GetSecretValue and kms:Decrypt permission while the first one doesn't (although the resource is set to %FIREHOSE_POLICY_TEMPLATE_PLACEHOLDER%)
- First one has no
Condition in trusted entities while the second one has it
I'm sure the aws:resourceArn under Statement.Condition.ArnEquals from the second stream's access role was pointing to the correct firehose stream arn
However, if I removed the condition from the second one's trusted entities, it will start working
What caused the condition checking failed?
Original trusted entities (permission denied)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "firehoseAssume",
"Effect": "Allow",
"Principal": {
"Service": "firehose.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "arn:aws:firehose:{region}:{uid}:deliverystream/{firehoseStreamName}"
}
}
}
]
}
Modified trusted entities (working fine)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "firehoseAssume",
"Effect": "Allow",
"Principal": {
"Service": "firehose.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Error log written in S3 destination under format-conversion-failed folder
{
"attemptsMade": 1,
"arrivalTimestamp": 1717385774937,
"errorCode": "DataFormatConversion.AccessDenied",
"errorMessage": "Access was denied when assuming role. Please ensure that the role specified in the data format conversion configuration has granted the Firehose service permission to assume it.",
"attemptEndingTimestamp": 1717385868087,
"rawData": {data},
"dataCatalogTable": {
"region": "{region}",
"versionId": "LATEST",
"tableName": "{table}",
"roleArn": "arn:aws:iam::{uid}:role/service-role/KinesisFirehoseServiceRole-{generatedName}",
"catalogId": "",
"databaseName": "{db}"
}
}
Thanks for accepting my answer!