CDK deploy fails because of a Socket timed out without establishing a connection

0

I'm working on my first CDK stack, and it's frustratingly not working at all. It dies when I go to run deploy with the following error:

Building assets failed: Error: Building Assets Failed: Error: FuseArchiver: Socket timed out without establishing a connection

If I run it with cdk deploy -vv I get these last few lines after it run synthesis:

[15:21:15] Retrieved account ID xxxxxxxxx from disk cache
[15:21:15] Assuming role 'arn:aws:iam::xxxxxxxxxxx:role/cdk-hnb-xxxxxx-deploy-role-xxxxxxxxxxxx-us-east-1'.
[15:21:16] Waiting for stack CDKToolkit to finish creating or updating...
[15:21:16] [AWS cloudformation 200 0.081s 0 retries] describeStacks({ StackName: 'CDKToolkit' })
[15:22:35] [AWS ssm undefined 79.686s 6 retries] getParameter({ Name: '/cdk-bootstrap/hnxxxxxds/version' })
[15:22:35] Call failed: getParameter({"Name":"/cdk-bootstrap/hnxxxxxxfds/version"}) => Socket timed out without establishing a connection (code=TimeoutError)

 ❌ Building assets failed: Error: Building Assets Failed: Error: FuseArchiver: Socket timed out without establishing a connection
    at buildAllStackAssets (D:\Users\xxxxx\apps\nvm\v18.15.0\node_modules\aws-cdk\lib\index.js:374:115279)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async CdkToolkit.deploy (D:\Users\xxxxx\apps\nvm\v18.15.0\node_modules\aws-cdk\lib\index.js:374:143496)
    at async exec4 (D:\Users\xxxxx\apps\nvm\v18.15.0\node_modules\aws-cdk\lib\index.js:429:51795)
[15:22:52] Reading cached notices from D:\Users\xxxxx\.cdk\cache\notices.json

Building Assets Failed: Error: FuseArchiver: Socket timed out without establishing a connection
[15:22:52] Error: Building Assets Failed: Error: FuseArchiver: Socket timed out without establishing a connection
    at buildAllStackAssets (D:\Users\xxxxx\apps\nvm\v18.15.0\node_modules\aws-cdk\lib\index.js:374:115279)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async CdkToolkit.deploy (D:\Users\xxxxx\apps\nvm\v18.15.0\node_modules\aws-cdk\lib\index.js:374:143496)
    at async exec4 (D:\Users\xxxxx\apps\nvm\v18.15.0\node_modules\aws-cdk\lib\index.js:429:51795)

So it looks like it dies when it goes to talk to AWS SSM with getParameter call. I'm running this from an AWS workspace inside a VPC which should be able to talk to SSM, but can't. I've had some issues with SSM reaching my boxes in the past with permissions, but I think those are resolved.

I'm using node: v18.15.0, CDK 2.69.0, and Windows 10

  • There seem to be an open bug. Does the suggestion on this site works for you? https://github.com/aws/aws-cdk/issues/19930

  • @alatech I saw that github issue. I tried adding --asset-parallelism=false which didn't seem to help. It still received the timeout. Also this one seems to timeout while uploading to S3 bucket. My issue looks like it hits this when requesting data from the SSM service which should be a relatively small response.

  • Do you have a vpc interface endpoint in your vpc? Either way could be that the SSM is not reachable as your workload is in vpc

  • I did have a VPC interface endpoint registered on the VPC that the workspace was in, but that Workspace wasn't included in the correct security groups to enable access to the VPC endpoint. Once, researched on how to add a workspaces to specific security groups (totally unintuitive process). It finally worked.

1 Answer
0

Hello,

I reviewed your query and there could be one to many reasons responsible for the Socket Timeout error that you are facing when the call is made to get the parameter: /cdk-bootstrap/hnxxxxxds/version. One of the prime reason being not able to connect to the SSM endpoint.

In order to possibly resolve the issue please undertake the below steps:

Steps

  1. Kindly execute the below command in the Administrative powershell on the workspace, to confirm if there is connectivity to the SSM endpoints.

    Test-NetConnection ssm.us-east-1.amazonaws.com -port 443

**If the command gives the output “TcpTestSucceeded: False”, implies that the error is due to connectivity with the endpoints. Hence, proceed further.

  1. If the workspace has connectivity over the internet, please ensure that the communication is allowed over port 443 and also to ssm.us-east-1.amazonaws.com through the Windows Firewall or any antivirus software installed on your workspace.

  2. If the Workspace does not have connectivity over the internet ensure that VPC endpoint for SSM is created properly using the below steps. Also, communication should be allowed over port 443 and also to ssm.us-east-1.amazonaws.com through the Windows Firewall or any antivirus software installed on your workspace as well.


  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.
  2. In the navigation pane, choose Endpoints.
  3. Choose Create endpoint.
  4. For Service category, choose AWS services.
  5. For Service Name, select com.amazonaws.us-east-1.ssm 
  6. For VPC, choose the VPC ID for your workspace.
  7. For Subnets, choose a Subnet ID in your VPC. For high availability, choose at least two subnets from different Availability Zones within the Region. Note: If you have more than one subnet in the same Availability Zone, you don't need to create VPC endpoints for the extra subnets. Any other subnets within the same Availability Zone can access and use the interface.
  8. For Enable DNS name, select Enable for this endpoint. For more information, see: https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html
  9. For Security group, select an existing security group, or create a new one. The security group must allow inbound HTTPS (port 443) traffic from the resources in your VPC that communicate with the service.
  10. If you created a new security group, open the VPC console, choose Security Groups, and then select the new security group. On the Inbound rules tab, choose Edit inbound rules. Add a rule with the following details, and then choose Save rules:
    1. For Type, choose HTTPS.
    2. For Source, choose your VPC CIDR. For advanced configuration, you can allow specific subnets' CIDR used by your workspace.

  1. I hope either of the steps above help you resolve the issue, if not I would also request you to have a look at the IAM role: arn:aws:iam::xxxxxxxxxxx:role/cdk-hnb-xxxxxx-deploy-role-xxxxxxxxxxxx-us-east-1 to see if it has permission to undertake ssm:GetParameter: https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParameter.html

Additionally, to troubleshoot in depth and answer your question in detail, we require details that are non-public information. Please open a support case with AWS using the following: https://support.console.aws.amazon.com/support/home#/case/create

Thank you!

AWS
SUPPORT ENGINEER
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions