Skip to content

How to conditionally create or link an existing S3 bucket in CloudFormation template?

0

How to conditionally create or link an existing S3 bucket in CloudFormation template?

I am working with an AWS CloudFormation template, and I need to conditionally create an S3 bucket or link an existing one based on a parameter.

Specifically, I want to achieve the following: If a parameter (CreateBucket) is set to "true", create a new S3 bucket. If the parameter (CreateBucket) is set to "false", reference an existing S3 bucket. I need the template to work in such a way that, regardless of whether the bucket is newly created or existing, I want to apply a bucket policy (for encryption in transit) to the selected bucket.

Ex: Suppose I created an S3 bucket and an SNS topic using a CloudFormation template. The bucket has a DeletionPolicy set to Retain. When I delete the stack, the bucket will not be deleted, and CloudFormation will skip it.

Later, if I attempt to create a resource with the same name, it will throw an error stating that the resource already exists. In such cases, can CloudFormation check whether the resource already exists and, if so, map that resource to the stack? If the resource does not exist, it should create a new one. Is this possible?

##What I've tried so far:

I have used CloudFormation conditions (!Equals) to control the creation of resources based on the CreateBucket parameter. For the newly created bucket, I apply the necessary configurations like encryption and public access blocking. For the existing bucket, I set the bucket name directly. I have used the !If intrinsic function to dynamically reference the correct bucket in the policy section, but I'm encountering errors.

##The problem I am facing: I’m receiving errors when using conditions in the Condition property of resources, such as "Template format error: Every Condition member must be a string." There’s also an issue with referencing the ARN of the bucket based on the condition.

my code

AWSTemplateFormatVersion: "2010-09-09"

Parameters:
  CreateBucket:
    Type: String
    Default: "true"
    AllowedValues:
      - "true"
      - "false"
    Description: "Specify whether to create a new S3 bucket or link an existing one."

Conditions:
  ShouldCreateBucket: !Equals [!Ref CreateBucket, "true"]

Resources:
  # Create a new S3 bucket if the condition is true
  Bucket:
    Type: AWS::S3::Bucket
    Condition: ShouldCreateBucket  # Reference condition by name directly
    DeletionPolicy: Retain
    Properties:
      BucketName: !Sub "${AWS::StackName}-bucket-${AWS::AccountId}"
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: aws:kms
              KMSMasterKeyID: alias/aws/s3
      PublicAccessBlockConfiguration:
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

  # If the condition is false, refer to an existing bucket
  ExistingBucket:
    Type: AWS::S3::Bucket
    Condition: !Equals [!Ref CreateBucket, "false"]
    Properties:
      BucketName: "existing-bucket-name"  # Replace with your existing bucket's name

  BucketBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !If 
        - ShouldCreateBucket
        - !Ref Bucket
        - !Ref ExistingBucket
      PolicyDocument:
        Id: RequireEncryptionInTransit
        Version: '2012-10-17'
        Statement:
          - Principal: '*'
            Action: '*'
            Effect: Deny
            Resource:
              # ARN references the newly created or existing bucket
              - !If
                - ShouldCreateBucket
                - !GetAtt Bucket.Arn
                - !Sub "arn:aws:s3:::existing-bucket-name"
              - !If
                - ShouldCreateBucket
                - !Sub "arn:aws:s3:::${Bucket.Name}/*"
                - !Sub "arn:aws:s3:::existing-bucket-name/*"
            Condition:
              Bool:
                aws:SecureTransport: 'false'
3 Answers
0

Hi,

I'd suggest you a solution with another level of indirection. I use this solution often:

  1. Add an SSM Parameter to your template: https://docs.aws.amazon.com/fr_fr/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html
  2. Apply the condition to the Value field of the Parameter to either choose the existing bucket or create the new one
  3. Use !GetAtt on Value !GetAtt <logicalNameOfYourParameter>.Value wherever you need to use the bucket name

Best,

Didier

EXPERT

answered 2 years ago

  • Is it possible only with cloudformation template. It will only check if the resource is exist or not if its exist it will map it into cloudformation otherwise it will create that resource.?

0

To conditionally create or link an existing S3 bucket in a CloudFormation template based on a parameter, you can use a combination of Parameters, Conditions, and Resources sections. Here's how you can approach this:

  1. Define a parameter: In the Parameters section, define a parameter called 'CreateBucket' of type String with allowed values 'true' and 'false'.

  2. Create a condition: In the Conditions section, create a condition that evaluates whether to create a new bucket:

Conditions:
  ShouldCreateBucket: !Equals [!Ref CreateBucket, 'true']
  1. Define the S3 bucket resource: In the Resources section, define the S3 bucket with a condition:
Resources:
  MyS3Bucket:
    Type: AWS::S3::Bucket
    Condition: ShouldCreateBucket
    Properties:
      # Add your bucket properties here
  1. For referencing an existing bucket: Create a parameter for the existing bucket name and use it when CreateBucket is 'false'.

  2. Use the !If function to reference the correct bucket: When applying the bucket policy or referencing the bucket elsewhere in your template, use the !If function to choose between the newly created bucket and the existing one:

BucketPolicy:
  Type: AWS::S3::BucketPolicy
  Properties:
    Bucket: !If 
      - ShouldCreateBucket
      - !Ref MyS3Bucket
      - !Ref ExistingBucketName
    # Rest of your policy definition

Regarding your specific issues:

  • The error "Every Condition member must be a string" typically occurs when you're trying to use a condition directly in a resource's Condition property. Instead, use the condition in the resource definition itself, as shown above.
  • For referencing the ARN of the bucket, you can use the !GetAtt function for the new bucket and construct the ARN for the existing bucket:
BucketArn: !If 
  - ShouldCreateBucket
  - !GetAtt MyS3Bucket.Arn
  - !Sub 'arn:aws:s3:::${ExistingBucketName}'

As for your question about reusing existing resources, CloudFormation doesn't automatically check for existing resources with the same name. However, you can use a custom resource with a Lambda function to check if a resource exists and return its details if it does. This way, you can either create a new resource or use an existing one based on the Lambda function's response.

Remember to set appropriate DeletionPolicy and UpdateReplacePolicy attributes on your resources to control their lifecycle during stack updates and deletions.
Sources
AWS::S3::Bucket - AWS CloudFormation
CloudFormation template Resources syntax - AWS CloudFormation
CloudFormation template Conditions syntax - AWS CloudFormation

answered 2 years ago

0

CloudFormation isn't meant to be used this way. It is a declarative language, intended to declare the desired target state of your resources, rather than being a procedural programming language that would inspect the state of resources outside the CFN stack and take different actions depending on the results of the checks.

The specific thing you're asking about you can technically accomplish by doing mostly what you're doing already: conditionally create the bucket depending on your parameter value, and when it isn't created, reference the name of the bucket in the Bucket propery of the AWS::S3::BucketPolicy resource, rather than trying to create an AWS::S3::Bucket resource for a bucket that exists already.

However, this isn't a good use of CloudFormation. For example, if a bucket policy already exists and contains other statements besides your preferred TLS restriction, setting a new bucket policy with CFN will either fail or overwrite the existing policy. It won't add the TLS restriction statement to an existing bucket policy. Also, even if setting the bucket policy succeeds but someone manually modifies the bucket policy later, the CFN stack will go into a drifted state and may behave unpredictably if ever updated.

The proper way to use CloudFormation would be to have it manage all the resources you configure via it. If you have previously created buckets and bucket policies manually via the console, for example, you should import each one into a CFN stack and make all the configuration changes there. If you want to keep some S3 buckets managed manually, you should modify or replace the bucket policies via the CLI or console, without going through CloudFormation.

EXPERT

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.