Skip to content

How do I reference existing buckets and their attributes in a cloudformation template

0

Greetings, I have a small serverless app working in a sandbox account which I'd like to instantiate in another account using CloudFormation. I have a template working for most of the resources, but I'd like to add the CloudFront Distribution and CachePolicy -- and I'd like to reference pre-existing buckets in the target account for logging and serving UI content. To start my template I used CloudFormation's IaC Generator tool in the sandbox account. This produced a template with the Distribution, CachePolicy, logging bucket, and content bucket all created from scratch.

To modify the generated template to reuse existing buckets, I first removed the two bucket definitions. I then added two string parameters to hold the names of the buckets. Now I need to modify the template's Distribution definition to reference attributes of the existing buckets rather than reference attributes of resources created in the same template. The generated template references the newly created buckets like so:

  CloudFrontDistribution00E1QRBQFLIBMPVI00Y2bzE:   
    UpdateReplacePolicy: "Delete"  
    Type: "AWS::CloudFront::Distribution"  
    DeletionPolicy: "Delete"  
    Properties:  
      DistributionConfig:  
        Logging:  
          IncludeCookies: false  
          Bucket:  
            Fn::GetAtt:  
            - "S3Bucket00identifierdevuse1logging0100Vu0Xw"  
            - "DomainName"  
          Prefix: "cloud-front-multi-use-01"  
        Comment: "Distribution allowing access to muiti-use s3 bucket."  
        DefaultRootObject: ""  
        Origins:  
        - ConnectionTimeout: 10  
          OriginAccessControlId: ""  
          ConnectionAttempts: 3  
          OriginCustomHeaders: []  
          DomainName:  
            Fn::GetAtt:  
            - "S3Bucket00identifierdevmultiuse01007gKi0"  
            - "RegionalDomainName"  

My modified template has two parameters for bucket names:

  CatActUiS3BucketName:  
    NoEcho: "false"  
    Type: "String"  
    Description: "An ...  
  CatActLoggingS3BucketName:  
    NoEcho: "false"  
    Type: "String"  
    Description: "An ...  

And attempts to reference the attributes so (this does not work):

          Bucket:  
            Fn::Sub:  
            - "${Bucket}.s3.amazonaws.com"  
            - { Bucket: !Ref "CatActLoggingS3BucketName"}  
          Prefix:  
            Fn::Sub:  
            - "${Base}${Suffix}"  
            - { Base: !Ref "CatActBaseName", Suffix: !Ref "CatActNameSuffix"}  
          -snip-  
          DomainName:  
            Fn::Sub:  
            - "${RDN}"  
            - {RDN: !GetAtt [!Ref "CatActUiS3BucketName", "RegionalDomainName"]}  

I have not figured out a way to pull an attribute out of an existing resource identified as a string parameter to the template. What is the best way to do this? Am I thinking about this wrong? Any advice would be greatly appreciated. Thanks for reading this, Terrence

3 Answers
1

Hello.

  1. Create the two new buckets in an infrastructure template with two outputs (the logging and ui buckets), and refer to those new output buckets as imports into my template.

If I were you, I would manage the buckets for the UI and logs separately, as in option 2.
It may work to combine the UI and log files into one, but there is a risk that the log files will be exposed to the public if an operational error occurs.
Therefore, we basically create separate S3 buckets for different purposes.
To reference an existing S3 bucket, I recommend putting the bucket name in the parameter and using "Fn::Sub" to concatenate strings.
By concatenating strings, you can create domain names that can be used with "Bucket:".

Also, as you know, you cannot extract attributes from an existing resource that is identified as a string parameter in the template.
Therefore, a specification such as "{RDN: !GetAtt [!Ref "CatActUiS3BucketName", "RegionalDomainName"]}" will cause an error.

EXPERT
answered 9 months ago
EXPERT
reviewed 9 months ago
0

I think I'll take inspiration from Leo K.'s answer (https://repost.aws/questions/QUkuULC_mgSRioRecS-ESXHQ/how-to-conditionally-create-or-link-an-existing-s3-bucket-in-cloudformation-template#ANMW61rnSxR_SW4Cp9UZ7AtA) to Ravi's question. In his answer Leo states

CloudFormation isn't meant to be used this way. It is a declarative language, intended to declare the desired target state of your resources, rather than being a procedural programming language that would inspect the state of resources outside the CFN stack and take different actions depending on the results of the checks.

I was worried about creating too many buckets which I anticipated might be hard to keep track of -- the target account already has a bucket set up for logging and one for web hosting, created outside of cfn. I guess the template will help me keep track of it. I think I have two options: 1) create the two new buckets in the template and referenced only by the resources in the template; or 2) Create the two new buckets in an infrastructure template with two outputs (the logging and ui buckets), and refer to those new output buckets as imports into my template.

If you have any advice for me on this or ways to think about CloudFormation templates, I'd still appreciate it. Thanks, Terrence

answered 9 months ago
-2

To reference existing buckets and their attributes in a CloudFormation template, you'll need to use a combination of parameters and CloudFormation-supplied parameter types. Here's how you can modify your approach:

  1. Instead of using simple string parameters for your bucket names, use the AWS-specific parameter type for S3 buckets. Change your parameter definitions to:
CatActUiS3BucketName:
  Type: AWS::S3::Bucket::Name
  Description: "Name of the existing S3 bucket for UI content"

CatActLoggingS3BucketName:
  Type: AWS::S3::Bucket::Name
  Description: "Name of the existing S3 bucket for logging"
  1. To reference the bucket names and their attributes in your Distribution definition, you can use the Ref function for the bucket name and the Fn::GetAtt function for bucket attributes. Here's how you can modify your Distribution properties:
CloudFrontDistribution00E1QRBQFLIBMPVI00Y2bzE:
  Type: "AWS::CloudFront::Distribution"
  Properties:
    DistributionConfig:
      Logging:
        IncludeCookies: false
        Bucket: 
          Fn::GetAtt: 
            - CatActLoggingS3BucketName
            - RegionalDomainName
        Prefix: 
          Fn::Sub:
            - "${Base}${Suffix}"
            - { Base: !Ref "CatActBaseName", Suffix: !Ref "CatActNameSuffix"}
      Comment: "Distribution allowing access to multi-use s3 bucket."
      DefaultRootObject: ""
      Origins:
        - ConnectionTimeout: 10
          OriginAccessControlId: ""
          ConnectionAttempts: 3
          OriginCustomHeaders: []
          DomainName:
            Fn::GetAtt: 
              - CatActUiS3BucketName
              - RegionalDomainName

By using the AWS-specific parameter type AWS::S3::Bucket::Name, CloudFormation will validate that the bucket exists in your account and allow you to reference its attributes using Fn::GetAtt.

This approach allows you to reuse existing buckets while still leveraging CloudFormation's ability to reference and use their properties. It ensures that the buckets exist and provides access to their attributes without needing to create new buckets in your template.
Sources
Specify existing resources at runtime with CloudFormation-supplied parameter types - AWS CloudFormation
CloudFormation template Resources syntax - AWS CloudFormation
CloudFormation template Parameters syntax - AWS CloudFormation

answered 9 months ago
  • cfn-lint tells me that AWS::S3::Bucket::Name is not a valid type for a parameter of a cfn template.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.