I have setup an AWS Backup plan to backup all of my S3 buckets. Each bucket backup job completes successfully with one exception. That bucket backup job completes with the status of "Completed with issues". After setting up SNS Email-JSON notifications I can see multiple messages with the EventType of S3_BACKUP_OBJECT_FAILED. Reading the document Troubleshoot errors for Amazon S3 backups that fail directs me to attach the AWSBackupServiceRolePolicyForS3Backup and AWSBackupServiceRolePolicyForS3Restore policies to the backup role being used. I am using the default role and had already attached the policies. However, I verified that with the command aws iam list-attached-role-policies --role-name AWSBackupDefaultServiceRole. The output of that command is:
{
"AttachedPolicies": [
{
"PolicyName": "AWSBackupServiceRolePolicyForRestores",
"PolicyArn": "arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForRestores"
},
{
"PolicyName": "AWSBackupServiceRolePolicyForBackup",
"PolicyArn": "arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup"
},
{
"PolicyName": "AWSBackupGatewayServiceRolePolicyForVirtualMachineMetadataSync",
"PolicyArn": "arn:aws:iam::aws:policy/service-role/AWSBackupGatewayServiceRolePolicyForVirtualMachineMetadataSync"
},
{
"PolicyName": "AWSBackupRestoreAccessForSAPHANA",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupRestoreAccessForSAPHANA"
},
{
"PolicyName": "AWSBackupAuditAccess",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupAuditAccess"
},
{
"PolicyName": "AWSBackupDataTransferAccess",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupDataTransferAccess"
},
{
"PolicyName": "AWSBackupFullAccess",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupFullAccess"
},
{
"PolicyName": "AWSBackupServiceRolePolicyForS3Restore",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupServiceRolePolicyForS3Restore"
},
{
"PolicyName": "AWSBackupServiceRolePolicyForS3Backup",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupServiceRolePolicyForS3Backup"
},
{
"PolicyName": "AWSBackupOrganizationAdminAccess",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupOrganizationAdminAccess"
},
{
"PolicyName": "AWSBackupOperatorAccess",
"PolicyArn": "arn:aws:iam::aws:policy/AWSBackupOperatorAccess"
}
]
}
I do not know why the backup continues to have failed objects. How can I further diagnose this issue? There is no additional information provided in the notifications and I can find no further documentation on troubleshooting S3 backup failures.
Alright, those are indeed offline storage classes requiring the separate restore step to access, so your conclusion is almost certainly precisely correct.
For completeness, when you restore objects from the offline storage classes, the original object will remain in the archive class, while the restore operation creates a copy of its contents in the Standard storage class. If you want to move the object back to an online storage class, you'll have to copy the temporary restored object into a new object that you'll retain in an online class. More details: https://docs.aws.amazon.com/AmazonS3/latest/userguide/restoring-objects.html
Thank you. I didn't realize that and after waiting the 48 hours for the deep archive restore was wondering why it was still the offline class. I'm now copying/replacing the offline objects with the online objects and putting them in the glacier_ir class. I'm pretty confident this will clear up the very last of the errors.
After modifying the storage class on all of the failed objects the errors have cleared.
Amazon, please include better error messaging. All of this could have been avoided with a simple "invalid storage class" error.