Skip to content

How do I migrate my Compute and Container resources to another region?

14 minute read
Content level: Foundational
0

This article provides general guidance on migrating Compute and Containers resources from one region to another.

If you are migrating data from the Middle East (UAE) Region (me-central-1), then you might experience increased error rates as we continue making progress with restoration efforts. For additional information about recovery efforts and service updates that impact your AWS accounts, see the AWS Personal Health Dashboard. For assistance with this event, contact AWS Support through the AWS Management Console or the AWS Support Center.

This is part of a series of articles that provide general guidance on migrating resources form one Region to another. It covers the following services:

  • Amazon EC2 / Amazon EBS
  • Amazon Elastic Container Service (Amazon ECS)
  • Amazon Elastic Kubernetes Service (Amazon EKS)

For general guidance and a full list of domain and service-specific migration guides, see How do I migrate my resources to another region?

How do I copy an EBS Snapshot to another region using coldsnap and the EBS Direct APIs? compliments the snapshot based approach used to migrate some of the services in this article.

For other domains, see the following resources:

Amazon EC2 / Amazon EBS

Key Considerations and Preparation

  • Instance store volumes are not preserved in AMIs and Snapshots
  • IP addresses will change; update DNS and security group references accordingly
  • Large Snapshots/AMIs (multi-TB) can take hours to copy cross-region
  • KMS Keys are regional

Using AMIs

This is the easiest way to migrate a whole instance and all its volumes to another region.

  1. Create an Amazon EBS-backed AMI from the instance
  2. Copy an Amazon EC2 AMI to the new region
  3. Launch an EC2 instance from a custom AMI

Using Snapshots

Follow this process if you only have access to a snapshot. If you can create an AMI, follow that process instead.

  1. Create a snapshot of an EBS volume
  2. Copy an Amazon EBS snapshot to the new region
  3. Create an Amazon EBS-backed AMI from the snapshot
  4. Launch an EC2 instance from a custom AMI

Time-Based Copies for Snapshots and AMIs

Use when standard copies are too slow and you need guaranteed completion time. This has an additional cost compared to normal copy processes that are free. More details can be found on the EBS Pricing Page

  1. Calculate required throughput (Data Size ÷ Completion Duration)
  2. Determine your completion duration (minimum 15 minutes, maximum 48 hours, in 15-minute increments)
  3. Check/request throughput quota increase if needed
  4. Copy snapshot with time-based completion option
  5. Monitor copy progress
  6. Create an Amazon EBS volume from the snapshot
  7. Launch an EC2 instance from a custom AMI and attach the volume

Using AWS Elastic Disaster Recovery (DRS)

Use for ongoing replication and disaster recovery to a new region. Includes the ability to fail back to the original region once available again.

  1. Initialize Elastic Disaster Recovery in the supported target region
  2. Add source servers to AWS DRS by installing the AWS Replication Agent
  3. Configure launch settings in AWS Elastic Disaster Recovery
  4. Prepare for failover by launching a drill instance
  5. Perform a failover with Elastic Disaster Recovery to the new region

Using AWS Application Migration Service (MGN)

Use for automated lift-and-shift migration to a new region. This is a one-way migration.

  1. Initialize Application Migration Service in the target region
  2. Add source servers by installing the AWS Replication Agent
  3. Configure launch settings in MGN
  4. Launch a test instance
  5. Launch a cutover instance to the new region

Using Data Lifecycle Manager (DLM) for Automated DR Backups

Use for automated, scheduled cross-region backups as part of disaster recovery preparation. This is a lower cost option than MGN and DRS but requires more management.

  1. Create a snapshot lifecycle policy with cross-region copy rules
  2. Tag your Amazon EC2 resources for automated backup
  3. View Amazon EBS snapshot information to verify snapshots are copying to the DR region
  4. Create an Amazon EBS volume from snapshots in the DR region
  5. Launch an Amazon EC2 instance for recovery
  6. Attach an Amazon EBS volume to an Amazon EC2 instance
  7. Edit records in Amazon Route 53 to point DNS to the DR region

Amazon Elastic Container Service (Amazon ECS)

Migrate Amazon ECS to Another Region

  • Container images must be available in the target region's ECR repository before workloads can be deployed - rebuild from your CI/CD pipeline (preferred) or copy images manually. ECR cross-region replication only applies to images pushed after replication is configured, so verify existing images are present using list-images
  • Secrets Manager secrets, Systems Manager Parameter Store parameters, and KMS keys are regional and must be recreated in the target region before registering task definitions
  • IAM roles are global but verify that task execution and task role policies do not contain hard-coded source region ARNs
  • EFS file systems and EBS volumes are regional or AZ-specific - use AWS DataSync for EFS data migration and EBS snapshot copy for EBS volumes
  • Cloud Map namespaces are regional and must be recreated in the target region
  • ACM certificates are regional - request or import certificates in the target region and update load balancer listener configurations with the new ARNs before migrating HTTPS services
  • EventBridge rules and schedules that target ECS tasks are regional and must be recreated in the target region

Redeploy Using Infrastructure as Code

Use this workflow when your ECS infrastructure is defined in CloudFormation, CDK, or Terraform.

  1. Update the target region in your IaC configuration and update all region-specific ARNs (KMS keys, ECR URIs, load balancers, EFS file systems)
  2. Redeploy the infrastructure in the target region using your IaC tool
  3. Update DNS in Route 53 to point to the new load balancer
  4. Verify all services reach desired task count with passing health checks

Recreate ECS Cluster and Services Manually

Use this workflow if you do not have IaC or need to migrate services selectively.

  1. Export source configuration using describe-clusters, describe-services, describe-task-definition, describe-capacity-providers, and describe-scaling-policies to document your current configuration
  2. Recreate regional dependencies in the target region: VPC and networking, KMS keys, Secrets Manager secrets, Parameter Store parameters, CloudWatch log groups, and ACM certificates. Create the target region load balancer before creating services, as ECS services must reference the load balancer at creation time. See Create an Application Load Balancer or Create a Network Load Balancer
  3. Register task definitions by stripping read-only fields (taskDefinitionArn, revision, status, requiresAttributes, compatibilities, registeredAt, registeredBy) from the exported JSON, updating all region-specific URIs and ARNs, then using register-task-definition. See Task definitions
  4. Recreate the cluster using create-cluster and services using create-service. For EC2 launch type, use the SSM parameter /aws/service/ecs/optimized-ami/amazon-linux-2023/recommended to find the target region's ECS-optimized AMI ID before recreating the Auto Scaling group and capacity provider
  5. Reconfigure auto scaling using register-scalable-target and put-scaling-policy. See Service auto scaling
  6. Recreate EventBridge rules and schedules targeting ECS tasks using put-rule or create-schedule, updating task definition ARNs and subnet or security group IDs to target region values
  7. Update Route 53 records to the new load balancer and confirm all services reach desired task count with passing health checks

Amazon Elastic Kubernetes Service (Amazon EKS)

Migrate Amazon EKS to Another Region

  • Container images must be available in the target region's ECR repository before workloads can be deployed - rebuild from your CI/CD pipeline (preferred) or copy images manually; ECR cross-region replication only applies to images pushed after replication is configured, so verify existing images are present using list-images
  • Secrets Manager secrets, Systems Manager Parameter Store parameters, and KMS keys are regional and must be recreated in the target region before deploying workloads that reference them
  • IAM roles are global but verify that pod role and IRSA trust policies do not contain hard-coded source region ARNs - EKS Pod Identity associations are cluster-specific and must be recreated
  • ACM certificates are regional - request or import certificates in the target region and update Ingress annotations with the new ARNs before deploying HTTPS workloads
  • EFS is regional and does not support native cross-region replication - use AWS DataSync for data migration
  • EBS volumes are AZ-specific - use snapshot copy to migrate them
  • Fargate profiles are cluster-specific and must be recreated using create-fargate-profile with matching namespace selectors
  • If the cluster uses Karpenter, NodePools and EC2NodeClasses must be recreated and the interruption SQS queue must be recreated in the target region - see the Karpenter documentation
  • AWS Backup for EKS does not capture DaemonSets, Jobs, CronJobs, HorizontalPodAutoscalers, PodDisruptionBudgets, or NetworkPolicies - export these resources and any non-managed add-on Helm configurations from the source cluster before taking a backup

Deploy Using Infrastructure as Code and AWS Backup

Use this workflow when your cluster infrastructure is managed with CloudFormation, CDK, or Terraform. IaC recreates the cluster and AWS Backup restores Kubernetes state and persistent storage.

  1. Create or verify a recent backup using start-backup-job, then create a backup vault in the target region using create-backup-vault and copy the recovery point using start-copy-job. See Amazon EKS backups
  2. Update your IaC configuration for the target region (region-specific ARNs, ECR URIs, KMS keys) and redeploy. Set cluster authorization mode to API or API_AND_CONFIG_MAP (required by AWS Backup)
  3. Associate the OIDC provider, update each IRSA trust policy to reference the new OIDC provider ARN, and recreate Pod Identity associations using create-pod-identity-association. See Enabling IAM Roles for Service Accounts
  4. Restore using start-restore-job targeting the new cluster. For encrypted clusters, provide a target region KMS key ARN via encryptionConfigProviderKeyArn. Monitor with describe-restore-job. See Restoring an Amazon EKS cluster
  5. Create EFS access points in the target region and update PersistentVolume manifests and StorageClass definitions with the new file system ID and access point IDs before applying storage resources
  6. Redeploy non-managed add-ons using the Helm configurations exported before backup, updating region-specific values (ACM certificate ARNs, ECR image URIs, EFS file system IDs). Apply resources not captured by Backup (DaemonSets, Jobs, CronJobs, HorizontalPodAutoscalers, PodDisruptionBudgets, NetworkPolicies) from the exported manifests
  7. Update Route 53 records to the new load balancer and confirm all pods are Running, add-ons are active, and health checks are passing

Restore Using AWS Backup Without Infrastructure as Code

Use this workflow when IaC is not available. AWS Backup captures Kubernetes cluster state (namespaces, Deployments, Services, StatefulSets, ConfigMaps, Secrets, PVCs, CRDs, role bindings) and persistent storage. VPCs, node groups, and container images are not included and must be prepared before the restore.

  1. Create or verify a recent backup and copy it to the target region using the same process as step 1 in the IaC workflow above
  2. Export each IRSA role trust policy using get-role and Pod Identity associations using list-pod-identity-associations
  3. Export Secrets Manager secrets using get-secret-value and Parameter Store parameters using get-parameter, then recreate them in the target region. Request or import ACM certificates for HTTPS workloads
  4. Copy container images to the target region ECR (restore will fail to start pods if images are inaccessible)
  5. Recreate VPC, subnets, and security groups. See Create a VPC
  6. Create the cluster using create-cluster with the same Kubernetes version. Set authorization mode to API or API_AND_CONFIG_MAP. See Creating an Amazon EKS cluster
  7. Associate the OIDC provider, update each IRSA trust policy to reference the new OIDC provider ARN using update-assume-role-policy, and recreate Pod Identity associations using create-pod-identity-association. See IAM Roles for Service Accounts
  8. Install EKS add-ons including CSI drivers using create-addon before restoring (required for PV mounts). See Managing Amazon EKS add-ons
  9. Create managed node groups using create-nodegroup, ensuring nodes exist in each AZ where EBS volumes will be restored. Recreate Fargate profiles using create-fargate-profile with matching namespace selectors
  10. Create EFS access points in the target region and update PersistentVolume manifests and StorageClass definitions with the new file system ID and access point IDs before applying storage resources
  11. Restore using start-restore-job targeting the new cluster. For encrypted clusters, provide a target region KMS key ARN via encryptionConfigProviderKeyArn
  12. Redeploy non-managed add-ons using the Helm configurations exported before backup, updating region-specific values (ACM certificate ARNs, ECR image URIs, EFS file system IDs). Apply resources not captured by Backup from the exported manifests
  13. Update Route 53 records and confirm all nodes are Ready, pods are Running, and health checks pass. Check logs for AccessDenied errors, which indicate an IRSA trust policy still referencing the source cluster's OIDC provider ARN

Recreate Cluster Manually Using kubectl and AWS CLI

Use this workflow if AWS Backup is not configured and you need to migrate cluster infrastructure and workloads manually.

  1. Export source cluster configuration using describe-cluster (Kubernetes version, networking, encryption, logging), each node group using describe-nodegroup, and each add-on version using list-addons and describe-addon. Export each IRSA role trust policy using get-role and Pod Identity associations using list-pod-identity-associations
  2. Export non-managed add-on configurations (AWS Load Balancer Controller, external-dns, cert-manager, metrics-server) using the Helm CLI. Record release names, chart versions, and values for each release
  3. Export all workload manifests from application namespaces (excluding kube-system, kube-public, kube-node-lease) using kubectl. Export cluster-scoped resources separately: CRDs, ClusterRoles, ClusterRoleBindings, StorageClasses, and IngressClasses. For EBS-backed workloads, copy EBS snapshots to the target region using copy-snapshot
  4. Export Secrets Manager secrets using get-secret-value and SSM parameters using get-parameter, then recreate them in the target region. Request or import ACM certificates for HTTPS workloads. Copy container images to the target region ECR
  5. Recreate VPC, subnets, and security groups in the target region, then create the cluster using create-cluster with the same Kubernetes version. See Creating an Amazon EKS cluster
  6. Update kubeconfig using update-kubeconfig, associate the OIDC provider, then update each IRSA trust policy to replace the source cluster's OIDC provider ARN with the new cluster's ARN using update-assume-role-policy. Recreate Pod Identity associations using create-pod-identity-association. See IAM Roles for Service Accounts
  7. Install EKS add-ons matching versions from step 1 using create-addon, including EBS CSI and EFS CSI drivers before applying any workloads. See Managing Amazon EKS add-ons
  8. Create managed node groups using create-nodegroup matching the configuration from step 1, ensuring nodes exist in each AZ where EBS volumes are being restored. Recreate Fargate profiles using create-fargate-profile with matching namespace selectors
  9. Apply cluster-scoped resources first (CRDs, ClusterRoles, ClusterRoleBindings, StorageClasses, IngressClasses), then prepare namespace-scoped manifests by stripping read-only fields (resourceVersion, uid, creationTimestamp, generation, managedFields, status), updating ECR image URIs, EFS file system IDs and access point IDs in PersistentVolume specs and StorageClass definitions, ACM certificate ARNs in Ingress annotations, and any Secrets Manager or KMS ARNs to target region values. Recreate cluster access using create-access-entry and associate-access-policy instead of copying the aws-auth ConfigMap. Apply prepared manifests using kubectl
  10. Redeploy non-managed add-ons from the configurations exported in step 2, updating region-specific values (ACM certificate ARNs, ECR image URIs, EFS file system IDs)
  11. Update Route 53 records to the new load balancer and confirm all nodes are Ready, pods are Running, and health checks pass. Check logs for AccessDenied errors, which indicate an IRSA trust policy or Pod Identity association still referencing the source cluster's OIDC provider ARN

In collaboration with Nymus Booysen and Francesco Penta

AWS
EXPERT
published 3 months ago1.4K views