Skip to content

Is it possible to use Lambda to copy ECR and S3 from one account to another?

0

Hello.

I need a way to quickly create a new account under the organization and copy S3 buckets and ECR repositories to this new account. Would Lambda be the most suitable tool for this task, or do you have other suggestions?

So far, I have successfully created an account using Lambda (NodeJS) and set up roles and some policies. However, I encountered an error when attempting to copy over ECR repositories. After some research, I discovered that Lambda might not be the best tool for this task. What would you suggest? Using CLI or something locally is not an option due to customer requirements.

I need a way to automate this process with a one-click solution, where Lambda seems ideal. During my research, I came across CodeBuild and found some examples of using it for copying ECR repositories. It seems possible to trigger CodeBuild from Lambda.

Would CloudFormation be an option for this task?

2 Answers
2
Accepted Answer

If you want to set up the base infrastructure (ECR and S3 buckets) then CloudFormation is very good option. If you need to copy the contents across then you're best off creating the resources with CloudFormation and then using some sort of code (which might be Lambda or perhaps a container-based solution) to perform the copy. What you can do is have CloudFormation send a message (via SNS or EventBridge) to indicate that the build is complete and then you can begin the copy process.

You say "I encountered an error" but you haven't said what it is so it's not possible to provide further guidance.

AWS
EXPERT
answered a year ago
EXPERT
reviewed a year ago
  • Posted below due to the char limit in the comment :) Appreciate your help.

0

@Brettski-AWS Thank you for your reply. So I have a very basic Lambda.

const sourceECR = new AWS.ECR();
    const targetECR = new AWS.ECR({
      accessKeyId: credentials.AccessKeyId,
      secretAccessKey: credentials.SecretAccessKey,
      sessionToken: credentials.SessionToken,
    });

    const sourceRepos = await sourceECR.describeRepositories().promise();
    console.log(
      `Source account has ${sourceRepos.repositories.length} repositories.`
    );

    for (const repo of sourceRepos.repositories) {
      const repoName = repo.repositoryName;

      try {
        await targetECR
          .createRepository({ repositoryName: repoName })
          .promise();
        console.log(`Created repository ${repoName} in target account.`);
      } catch (err) {
        if (err.code !== "RepositoryAlreadyExistsException") {
          throw err;
        }
        console.log(`Repository ${repoName} already exists in target account.`);
      }

      const images = await sourceECR
        .listImages({ repositoryName: repoName })
        .promise();

      console.log(`Images: ${images.imageIds.length}`);

      for (const image of images.imageIds) {
        const imageDetail = await sourceECR
          .batchGetImage({
            repositoryName: repoName,
            imageIds: [image],
          })
          .promise();

        // Step 4: Put images to the target repository
        await targetECR
          .putImage({
            repositoryName: repoName,
            imageManifest: imageDetail.images[0].imageManifest,
            imageTag: imageDetail.images[0].imageId.imageTag,
          })
          .promise();

        console.log(
          `Copied image ${image.imageTag} from ${repoName} to target account.`
        );
      }
    }

I am succesfully creating repo in new account, but after that I got to an error:

"Layers with digests '[sha256:b6057b4254346546030669b570acd43d253caf06867d842ec8b568a7a411f259,sha256:a809ba7fc93f49dd84cca1125a37c820f202d6722a674443d5c68533b4a79622,sha256:910adb6ea1d3dcdb53dcba1a5307674a5a2dd11aae88cc4f822bb0cd5cbc0834,sha256:6c7b70fd6df49d5755c5cf0d1f704aea27cc581ca8818c1109f66cc8914eeb53,sha256:3f4363f23d43be137d87d35b2cfb1b426339a0ef82dfcee031ccaa4c58b9ee9b,sha256:5e764821d4202db9fa73fde38169d124987f4aded88171f0d9f0daa5fe071147,sha256:f92d57d31a76e0d5ed1fe4e9a697e384c4018104435611d1c9ec76748d00b747,sha256:b51b2d37b1a1441a637b335057d3af087e1b50eb2aa8c3b7e9df13d217b01692,sha256:406144831b66410bcc96b7d2f803c7f402a25dddb148331fc17943a5ef1a72f9,sha256:c6a83fedfae6ed8a4f5f7cbb6a7b6f1c1ec3d86fea8cb9e5ba2e5e6673fde9f6,sha256:3eddb9bd60ff80030aba15d4fad09d69910182841eb4af0aa7096c3e9b008d75,sha256:5c30aa900b096c1999e57347eefc256ac6d0f228a2427efa4b8d9b2ec9152e74]' required for pushing image into repository with name 'rtc' in the registry with id '0144986xxxx' do not exist\"}"

When I started looking into it, I saw some suggestions that a lambda function might not be suitable for this kind of task. Since I have only been using AWS for a couple of weeks, I'd rather ask here for advice than waste too much time on something that might not work due to technical limitations.

I want to provide some more information about the images because I noticed that Lambda has some limitations in this area. I will have 5 images, each around 150MB in size. In S3, there will be 2 buckets, each around 20MB, which contain a compiled website.

Here’s the whole idea:

I currently have a CloudFormation setup using the AWS CDK, with several stacks where I configure the network, IDP, and a few applications using AppRunner or a Fargate, depending on the needs. The idea is for a Lambda function to generate a new account, copy the necessary images to ECR and S3, and once that's done, another Lambda function triggers CloudFormation to set up everything with images from its own ECR and S3. This is a business requirement to ensure each account has its own version. In the testing phase, I have this working with images from the ECR on the organisation account.

The only piece currently missing is how to automatically copy (ECR and S3) from the organisation account to the newly created account.

answered a year ago
  • You can definitely do what you want with Lambda; but I would be breaking up each step into an individual Lambda function and orchestrating them with Step Functions. That way you can build in retries and other functionality without your code becoming too complex and without running into runtime limitations of Lambda.

  • Great thank you :) Do you have any tips for the problems with layers or do you know some code samples which do something like this?

  • Could you be more specific about "problems with layers"? Layers work. If you have a question about that, best to create it as a another post.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.