How do I download logs from the Elastic Beanstalk console without receiving an Access Denied error?

Short description

When you request tail logs in the Elastic Beanstalk environment console or with eb logs, the most recent log entries link together. The log entries link together into a single text file and are uploaded to Amazon Simple Storage Service (Amazon S3) by an instance in your environment.

When you request bundle logs, an instance in your environment packages the full log files into a ZIP archive and uploads it to Amazon S3.

The instances in your environment must have an Elastic Beanstalk instance profile with permission (s3:Get* ,s3:List*, s3:PutObject) to write to your Amazon S3 bucket. These permissions are included in the default instance profile. If you're using a custom instance profile role, then include these permissions.

To resolve the Access Denied errors or logs stuck in download when trying to retrieve logs from your AWS Elastic Beanstalk console, check the following:

• Amazon S3 user permissions
• Amazon S3 bucket policy
• Amazon S3 bucket encrypted with KMS key
• Amazon S3 gateway endpoint policy
• Service control policies (SCP)
• Resource utilization

Resolution

Amazon S3 user permissions

Elastic Beanstalk uses user permissions to save or upload logs to your Elastic Beanstalk S3 bucket. AWS Identity and Access Management (IAM) users must have the following permissions to retrieve logs from the Elastic Beanstalk console:

• s3:PutObject
• s3:GetObject
• s3:GetBucketAcl
• s3:PutObjectAcl

Note: Your user policy must also have the s3:DeleteObject permission because Elastic Beanstalk uses your user permissions to delete the logs from Amazon S3.

Example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:GetBucketAcl",
        "s3:PutObjectAcl"
      ],
      "Resource": "*"
    }
  ]
}

Amazon S3 bucket policy

Check your Elastic Beanstalk Amazon S3 bucket policy and make sure the PutObject permission is allowed for your instance profile. The PutObject permission is automatically allowed for your default instance profile (aws-elasticbeanstalk-ec2-role). If you're using a custom instance profile, then make sure to add the PutObject permission.

Example:

{
            "Sid": "eb-ad78f54a-f239-4c90-adda-49e5f56cb51e",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::123456789012 :role/aws-elasticbeanstalk-ec2-role",
                    "arn:aws:iam::126355979347:role/custom-instance-profile-role"
                ]
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::elasticbeanstalk-ap-south-1-123456789012/resources/environments/logs/*"
        },

Amazon S3 bucket encrypted with KMS key

KMS key encryption can be added to the Amazon S3 buckets used by Elastic Beanstalk. When KMS key encryption is added, the presigned URL generated by the Elastic Beanstalk Pull Bundle Logs action on the console fails. This failure is indicated by an Access Denied error.

As a workaround, you can manually download the bundle logs from the Amazon S3 bucket locations. For more information, see Log location in Amazon S3.

Amazon S3 gateway endpoint policy

An Elastic Beanstalk environment can be created in private subnets using Amazon Virtual Private Cloud (Amazon VPC) endpoints. For this scenario, you must have an Amazon S3 gateway endpoint to communicate with instances and retrieve files such as UserdataBootstrap.sh and platform.zip. Check whether there are any user restrictions at the Amazon S3 gateway endpoint level. For more information, see Gateway endpoints for Amazon S3.

Service control policies

If your permissions are correct and you still receive an Access Denied error, then check whether an organizational policy is turned on for your account. For more information, see Service control policies (SCPs).

Resource utilization

If all permissions and policies are correctly configured, then check your resource utilization for your Amazon Elastic Compute Cloud (Amazon EC2) instance. If the server is over-utilized, such as with high CPU or memory usage, then you your logs can be stuck in download. To resolve this, change your instance type to increase your CPU and memory storage. For example, you can change it from t2.micro to t2.medium.