Questions tagged with AWS Systems Manager

I just created a few simple Systems Manager State Manager associations, all to accomplish small tasks like running inventory and scanning for patches. I had initially assumed that an association failing on a single instance would lead that instance to be marked as non-compliant, with whatever severity level was configured for the association. In one of the walkthroughs in the SSM documentation, however, I see this: > If the State Manager association fails, no compliance data is reported. For example, if Systems Manager attempts to download a Chef cookbook from an S3 bucket that the node doesn't have permission to access, the association fails, and Systems Manager reports no compliance data. Is this the case for all association failures, regardless of document? And if it is, under what scenarios would association compliance show anything other than compliant instances?

Hi All, Has anyone had any luck connecting via Fleet's Manager RDP to Microsoft Windows Server 2012 Standard instances? RDP and connectivity seems to work all as expected outside Fleet Manager, we can get on any of 2012 (not R2) boxes without any issues from any other box. No other Windows versions we use seem to have this issue, 2012 R2 and 2019 seem to work perfectly fine with Fleet's Manager RDP. Any advise will be much appreciated. Kind Regards.

Hi, I was able to configure AWS session manager to use SSH keys over session manager tunnel as it is described here -> https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html. But now i need to force user to provide SSH keys, because now, even tho i can use SSH keys to authenticate into the EC2 instance, im still able to to it without providing SSH keys, just by using `aws ssm start-session` command. As i suppose i can add some kind of policy for that, something like: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::accountid:user/test-user" }, "Action": [ "ssm:TerminateSession", "ssm:ResumeSession" ], "Condition": { "StringEquals": { "ssm:StartSession/RequireSSH": "True" ( parameters made up, by me ) } } } ] } ``` But im not sure what should be in the place of `"ssm:StartSession/RequireSSH": "True"`, Any help will be appreciated Joann

Hi, I want to forbid IAM user (for example: support-IAM ) to authenticate to the EC2 instance user ( for example : root-EC2 ), but still be able to authenticate as other EC2 instance user ( for example: support-EC2 ) , for that, as i understood i need to forbid this user to set `--tags` with value `root-EC2` (or in my case, everything **except** `support-EC2`) while using command `aws sts assume-role`, but how can i do it? P.S im using `AWS session manager` Joann

Put simply, I have written an SSM Automation. It creates a snapshot of each of the attached volumes of a targeted instance, by getting a list of Volume Ids from a DescribeInstance call. It then utilizes Rate Execution of a second runbook, in an AWS:executeAutomation action, which creates each snapshot, fanning out on those volume ids. As you can see, I have already limited MaxConcurrency of this step to 1. ``` { "name":"SnapshotAllVolumes", "action":"aws:executeAutomation", "maxAttempts": 3, "onFailure":"Abort", "inputs":{ "DocumentName": "MyCustomCreateSnapshotRunbook", "Targets": [ { "Key": "ParameterValues", "Values": [ "{{ DescribeInstance.CurrentVolumes }}" ] } ], "TargetParameterName": "VolumeId", "RuntimeParameters": { "InstanceId": "{{ InstanceId }}", "InstanceName": "{{ GetInstanceName.Name }}" }, "MaxConcurrency": "1" } ``` Where I have gotten into trouble is I am attempting to execute this runbook via Maintenance Window on ~60 instances, each averaging about 3 EBS volumes. I keep crashing into a rate limit on the above step. > Step fails when it is Executing. Fail to start automation, errorMessage: Rate exceeded. Please refer to Automation Service Troubleshooting Guide for more diagnosis details. Unfortunately, it doesn't tell me exactly which rate limit I'm hitting, but I think I can assume it is one of two: either the limit on api calls, or the limit on simultaneous SSM rate executions. Because the latter is much more restrictive(25 is the max), I think it's my most likely suspect. I've been dialing down the concurrency limit on my Maintenance Window. If my assumption about rate limit is correct, I need to stay under 25 concurrent Rate Executions. With the max concurrency of 1 on the child runbook, no individual execution of the parent runbook should lead to more than 2 simultaneous rate executions(the parent, and x*(number of volumes) consecutive executions). This means my concurrency rate for my maintenance window needs to keep below 25/2, so I've reckoned my max safe concurrency is a mere 12. Talking all this over with support, the solution recommended was to implement some sort of exponential backoff on the retries here. That way the calls that are rate limited on the first attempt are retried at different times, et al. This could be done through code if resorted to invoking a lambda that executed the child Automation, instead of executing directly in the Parent Runbook. I'd much rather not need to introduce a dependency on a Lambda here. I was wondering if anyone has a way to implement incremental backoff purely in SSM automation? Perhaps it isn't even possible?

Hi, We need to achieve: 1) Possibility to create/delete ssh key pairs for instances hosted in multiple accounts, in one place (centralization) 2) Possibility to assign permissions for the user we are ssh'ing into inside the instance, for example : IAM - admin user, should have admin ssh key, which will give full read/write permissions in virtual machine in which user is ssh'ing, and the IAM - support user, should have support ssh key, which will give less permissions. Any suggestions, in what we can look into, are welcome. As i googled a bit, the "AWS system manager" - "session manager" looks like it would work for us, but does it work for instances hosted in multiple accounts? Joann

Apologize to all for the duplicate post. I created my login under the wrong account when I initially posted this question. I’m able to generate a new OpsItem for any EC2, SecurityGroup, or VPC configuration change using an EventBridge rule with the following event pattern. { "source": "aws.config", "detail-type": "Config Configuration Item Change", "detail": { "messageType": "ConfigurationItemChangeNotification", "configurationItem": { "resourceType": "AWS::EC2::Instance", "AWS::EC2::SecurityGroup", "AWS::EC2::VPC" } } } The rule and target work great when using Matched event for the Input but I noticed that launching one EC2 using the AWS wizard creates at least three OpsItems, one for each resourceType. Therefore I’d like to implement a deduplication string to cut down on the number of OpsItems generated to one if possible and I’d also like to attach a category and severity to the new OpsItem. I’m trying to use an Input Transformer as recommended by the AWS documentation but even the most simplest of Input Transformers when applied prevent any new OpsItems from being generated. When I've tested, I've also ensured that all previous OpsItems were resolved. Can anyone tell me what might be blocking the creation of any new OpsItems when using this Input Transformer configuration? Here’s what I have configured now. Input path { "awsAccountId": "$.detail.configurationItem.awsAccountId", "awsRegion": "$.detail.configurationItem.awsRegion", "configurationItemCaptureTime": "$.detail.configurationItem.configurationItemCaptureTime", "detail-type": "$.detail-type", "messageType": "$.detail.messageType", "notificationCreationTime": "$.detail.notificationCreationTime", "region": "$.region", "resourceId": "$.detail.configurationItem.resourceId", "resourceType": "$.detail.configurationItem.resourceType", "resources": "$.resources", "source": "$.source", "time": "$.time" } Input template { "awsAccountId": "<awsAccountId>", "awsRegion": "<awsRegion>", "configurationItemCaptureTime": "<configurationItemCaptureTime>", "resourceId": "<resourceId>", "resourceType": "<resourceType>", "title": "Template under ConfigDrift-EC2-Dedup4", "description": "Configuration Drift Detected.", "category": "Security", "severity": "3", "origination": "EventBridge Rule - ConfigDrift-EC2-Dedup", "detail-type": "<detail-type>", "source": "<source>", "time": "<time>", "region": "<region>", "resources": "<resources>", "messageType": "<messageType>", "notificationCreationTime": "<notificationCreationTime>", "operationalData": { "/aws/dedup": { "type": "SearchableString", "value": "{\"dedupString\":\"ConfigurationItemChangeNotification\"}" } } } Output when using the AWS supplied Sample event called “Config Configuration Item Change” { "awsAccountId": "123456789012", "awsRegion": "us-east-1", "configurationItemCaptureTime": "2022-03-16T01:10:50.837Z", "resourceId": "fs-01f0d526165b57f95", "resourceType": "AWS::EFS::FileSystem", "title": "Template under ConfigDrift-EC2-Dedup4", "description": "Configuration Drift Detected.", "category": "Security", "severity": "3", "origination": "EventBridge Rule - ConfigDrift-EC2-Dedup", "detail-type": "Config Configuration Item Change", "source": "aws.config", "time": "2022-03-16T01:10:51Z", "region": "us-east-1", "resources": "arn:aws:elasticfilesystem:us-east-1:123456789012:file-system/fs-01f0d526165b57f95", "messageType": "ConfigurationItemChangeNotification", "notificationCreationTime": "2022-03-16T01:10:51.976Z", "operationalData": { "/aws/dedup": { "type": "SearchableString", "value": "{"dedupString":"ConfigurationItemChangeNotification"}" } } }

I’m able to generate a new OpsItem for any EC2, SecurityGroup, or VPC configuration change using an EventBridge rule with the following event pattern. { "source": ["aws.config"], "detail-type": ["Config Configuration Item Change"], "detail": { "messageType": ["ConfigurationItemChangeNotification"], "configurationItem": { "resourceType": ["AWS::EC2::Instance", "AWS::EC2::SecurityGroup", "AWS::EC2::VPC"] } } } The rule and target work great when using Matched event for the Input but I noticed that launching one EC2 using the AWS wizard creates at least three OpsItems, one for each resourceType. Therefore I’d like to implement a deduplication string to cut down on the number of OpsItems generated to one if possible and I’d also like to attach a category and severity to the new OpsItem. I’m trying to use an Input Transformer as recommended by the AWS documentation but even the most simplest of Input Transformers when applied prevent any new OpsItems from being generated. When I've tested, I've also ensured that all previous OpsItems were resolved. Can anyone tell me what might be blocking the creation of any new OpsItems when using this Input Transformer configuration? Here’s what I have configured now. Input path { "awsAccountId": "$.detail.configurationItem.awsAccountId", "awsRegion": "$.detail.configurationItem.awsRegion", "configurationItemCaptureTime": "$.detail.configurationItem.configurationItemCaptureTime", "detail-type": "$.detail-type", "messageType": "$.detail.messageType", "notificationCreationTime": "$.detail.notificationCreationTime", "region": "$.region", "resourceId": "$.detail.configurationItem.resourceId", "resourceType": "$.detail.configurationItem.resourceType", "resources": "$.resources", "source": "$.source", "time": "$.time" } Input template { "awsAccountId": "<awsAccountId>", "awsRegion": "<awsRegion>", "configurationItemCaptureTime": "<configurationItemCaptureTime>", "resourceId": "<resourceId>", "resourceType": "<resourceType>", "title": "Template under ConfigDrift-EC2-Dedup4", "description": "Configuration Drift Detected.", "category": "Security", "severity": "3", "origination": "EventBridge Rule - ConfigDrift-EC2-Dedup", "detail-type": "<detail-type>", "source": "<source>", "time": "<time>", "region": "<region>", "resources": "<resources>", "messageType": "<messageType>", "notificationCreationTime": "<notificationCreationTime>", "operationalData": { "/aws/dedup": { "type": "SearchableString", "value": "{\"dedupString\":\"ConfigurationItemChangeNotification\"}" } } } Output when using the AWS supplied Sample event called “Config Configuration Item Change” { "awsAccountId": "123456789012", "awsRegion": "us-east-1", "configurationItemCaptureTime": "2022-03-16T01:10:50.837Z", "resourceId": "fs-01f0d526165b57f95", "resourceType": "AWS::EFS::FileSystem", "title": "Template under ConfigDrift-EC2-Dedup4", "description": "Configuration Drift Detected.", "category": "Security", "severity": "3", "origination": "EventBridge Rule - ConfigDrift-EC2-Dedup", "detail-type": "Config Configuration Item Change", "source": "aws.config", "time": "2022-03-16T01:10:51Z", "region": "us-east-1", "resources": "arn:aws:elasticfilesystem:us-east-1:123456789012:file-system/fs-01f0d526165b57f95", "messageType": "ConfigurationItemChangeNotification", "notificationCreationTime": "2022-03-16T01:10:51.976Z", "operationalData": { "/aws/dedup": { "type": "SearchableString", "value": "{"dedupString":"ConfigurationItemChangeNotification"}" } } }

Hi, I tried to migrate my EC2-classic to VPC Before using wizard "systems-manager/automation", i have only create an new Security Group in my standalone VPC. My VPC and my EC2 server are all in Ireland, why this error ? I just use "Test" and get security group id created just before How could i resolve it ? Regards > Step fails when it is Poll action status for completion. Traceback (most recent call last): File "/tmp/6c97ba4a-da4f-4e64-9657-1340b5e9d1e1-2022-05-02-16-03-47/customer_script.py", line 186, in verifysecurityGroup raise Exception('[ERROR] Security Group and Subnet are not in same VPC') Exception: [ERROR] Security Group and Subnet are not in same VPC Exception - [ERROR] Security Group and Subnet are not in same VPC. Please refer to Automation Service Troubleshooting Guide for more diagnosis details.

I am logged in as root user but I don't see where to terminate these services.

I'm wondering to know if there is a way to distinguish stale DNS records in Route53 using SSM, trusted advisor or any other AWS tool.

Hello, I have finished the setup of Systems Manager Change Manager, I created the SNS Topic for Template Reviewers, Change Request Approvers and also another one for Notifications of the change request progress. Every time I run a Change Request it says on the Timeline: ``` Failed to publish notifications to SNS. ``` I have tried to use the same Topic of Approvers and Template Reviewers (where I know the emails are being sent correctly) but the error is the same, I would like to know what am I missing ? Does the Topic need a different Access Policy ? Do I need to add a SNS Policy to another role ? Thanks for your insights.

how can i point a new instance with a domain which is already exist with another ip address in hosting zone? Is that even possible?

[After resolving my first issue](https://repost.aws/questions/QUXOInFRr1QrKfR0Bh9wVglA/aws-glue-not-properly-crawling-s-3-bucket-populated-by-resource-data-sync-specifically-aws-instance-information-is-not-made-into-a-table) with getting a resource data sync set up, I've now run into another issue with the same folder. When a resource data sync is created, it creates a folder structure with 13 folders following a folder structure like: `s3://resource-data-sync-bucket/AWS:*/accountid=*/regions=*/resourcetype=*/instance.json}` When running the glue crawler over this, a schema is created where partitions are made for each subpath with an `=` in it. This works fine for most of the data, except for the path starting with `AWS:InstanceInformation`. The instance information json files ALSO contain a "resourcetype" field as can be seen here. ``` {"PlatformName":"Microsoft Windows Server 2019 Datacenter","PlatformVersion":"10.0.17763","AgentType":"amazon-ssm-agent","AgentVersion":"3.1.1260.0","InstanceId":"i","InstanceStatus":"Active","ComputerName":"computer.name","IpAddress":"10.0.0.0","ResourceType":"EC2Instance","PlatformType":"Windows","resourceId":"i-0a6dfb4f042d465b2","captureTime":"2022-04-22T19:27:27Z","schemaVersion":"1.0"} ``` As a result, there are now two "resourcetype" columns in the "aws_instanceinformation" table schema. Attempts to query that table result in the error `HIVE_INVALID_METADATA: Hive metadata for table is invalid: Table descriptor contains duplicate columns` I've worked around this issue by removing the offending field and setting the crawler to ignore schema updates, but this doesn't seem like a great long term solution since any changes made by AWS to the schema will be ignored. Is this a known issue with using this solution? Are there any plans to change how the AWS:InstanceInformation documents are so duplicate columns aren't created.

I set up an s3 bucket that collects inventory data from multiple AWS accounts using the Systems Manager "Resource Data Sync". I was able to set up the Data Syncs to feed into the single bucket without issue and the Glue crawler was created automatically. Now that I'm trying to query the data in Athena, I noticed there is an issue with how the Crawler is parsing the data in the bucket. The folder "AWS:InstanceInformation" is not being turned into a table. Instead, it is turning all of the "region=us-east-1/" and "test.json" sub-items into tables which are, obviously, not queryable. To illustrate further, each of the following paths is being turned into it's own table. * s3://resource-data-sync-bucket/AWS:InstanceInformation/accountid=12345679012/region=us-east-1 * s3://resource-data-sync-bucket/AWS:InstanceInformation/accountid=12345679012/test.json * s3://resource-data-sync-bucket/AWS:InstanceInformation/accountid=23456790123/region=us-east-1 * s3://resource-data-sync-bucket/AWS:InstanceInformation/accountid=23456790123/test.json * s3://resource-data-sync-bucket/AWS:InstanceInformation/accountid=34567901234/region=us-east-1 * s3://resource-data-sync-bucket/AWS:InstanceInformation/accountid=34567901234/test.json This is ONLY happening with the "AWS:InstanceInformation" folder. All of the other folders (e.g. "AWS:DetailedInstanceInformation") are being properly turned into tables. Since all of this data was populated automatically, I'm assuming that we are dealing with a bug? Is there anything I can do to fix this?

I am trying to use ssm to access and manage ec2 in a private network. I searched for the cost of the service, but only other services such as paramete center came out, so I couldn't measure the cost of the service I wanted to do. How is the cost of using AWS ssm service to connect to ssm on a private network and use commands?

Hello there, I have a Lambda that is trying to move a file from S3 to a Windows EC2 instance. I am using `ssm` to do it. When I get granular with the perms I get the following error: ``` 2022-04-19T20:32:15.502Z 5737b35c-6d81-471f-b29c-3fd23f1a5123 INFO AccessDeniedException: User: arn:aws:sts::xxx:assumed-role/userxyz is not authorized to perform: ssm:SendCommand on resource: arn:aws:ssm:us-east-1::document/AWS-RunPowerShellScript because no identity-based policy allows the ssm:SendCommand action ``` If I attached the `AmazonSSMFullAccess` policy to the IAM Role, it works. Which other perm do I need to add so that I do not grant the very permissible managed policy? _Edit:_ Forgot to attach the policy ``` { "Effect": "Allow", "Action": "ssm:SendCommand", "Resource": [ "arn:aws:ssm:*:xxx:document/*", "arn:aws:ec2:*:xxx:instance/*" ] } ```

Scheduled SSM run command to find updates on all servers with AWS-RunPatchBaseline, Version:$DEFAULT and Operation: Scan but its filing, please suggest what I am missing here. Error Message: The find operation did not complete successfully. More information : HResult: -2145123271. I am running a SSM scheduled run command to SCAN for updates of servers, the command is AWS-RunPatchBaseline, Version:$DEFAULT and Operation: Scan but it failed with error, please suggest what I am missing here. error message: Invoke-PatchBaselineOperation, Error Message: The find operation did not complete successfully. More information : HResult: -2145123271. Tried to clean updates logs, updated registry edit to remove WUserver and WUserverupdate from HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate updated ssm and powershells tried many steps but it failed

Hello AWS Experts! First and foremost, if you have stumbled upon my inquiry and are ready to aid me in resolving the issue, I appreciate you. My company has established an AWS Workspaces environment in the Virginia region. The Workspaces service is located in two private subnets within our Virginia VPC. When a U.S.-based user connects to the Workspaces environment, they do not deal with latency issues. Additionally, from Workspaces, the user will connect to a private EC2 instance using AWS System Manager (SSM) port forwarding. The user deals with slight latency when port forwarding/RDP into the instance. If there is a way to increase network speeds for SSM port forwarding, please inform me. Furthermore, this is where your phenomenal expertise is most needed; our users in Bulgaria experience high latency when attempting to connect to their Workspaces and even higher when trying to use SSM to connect to their EC2 instance. When a Bulgarian user uses Workspaces, their round trip time averages around 150ms. I had them test their local network bandwidth, which most averaged around 84 to 120Mbps download speeds and 140 to 160Mbps upload speeds. Additionally, they checked their "connection status..." from the following website: https://clients.amazonworkspaces.com/Health The site stated that Frankfurt provided the best connection; however, we do not want to create an entirely new VPC in the Frankfurt region for Workspaces. And have our customers question why a European IP is trying to connect to their assets. A VPN may be a possibility, but we're hoping for a better solution. I have tried the following, which non proved to improve network speeds for U.S. or Bulgarian members. - Established an Internal Load Balancer - Established VPC Endpoints - Established Global Accelerator So, how could one increase network speeds to Workspaces and SSM port forwarding? Thank you!

HI, I tried to use AWS SSM Patch manager for a Windows 2019 instance. I' used S3 VPC Endpoint with a private route53 hosted Zone. The SG of the S3 endpoint allow https and http. In the SSM logs it seams I have a SSL certificate issue but I do not know how to solve this issue. Preparing to download PatchBaselineOperations PowerShell module from S3. Downloading PatchBaselineOperations PowerShell module from https://s3-eu-west-3.amazonaws.com/aws-ssm-eu-west-3/patchbaselineoperations/Amazon.PatchBaselineOperations-1.35.zip to C:\ProgramData\Amazon\SSM\InstanceData\i-0e71b155c8a6cbe29\document\orchestration\84e87dc6-d218-45de-bb2a-e3d855cabe3b\PatchWindows\Amazon.PatchBaselineOperations-1.35.zip. C:\ProgramData\Amazon\SSM\InstanceData\i-0e71b155c8a6cbe29\document\orchestration\84e87dc6-d218-45de-bb2a-e3d855cabe3b\ PatchWindows\_script.ps1 : An error occurred when executing PatchBaselineOperations: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,_script.ps1 failed to run commands: exit status 0xffffffff The instance profile is set to AdministratorRole for the test. Have you ever encountered this issue ? Thanks

I have a windows instance running Windows Server 2019 and after stopping and starting the instance, I cannot connect to it SSM or RDP. When I run aws ec2 get-console-output returns the instance id and timestamp only: { "InstanceId": "i-07dd6exxxxxxxxd1", "Timestamp": "2022-04-11T18:25:04+00:00" } If I try to execute ssm start session, it times out and returns: Command '['session-manager-plugin', '{"SessionId": "oscars@blank.com-0473707d8049c843c", "TokenValue": "***", "StreamUrl": "wss://ssmmessages.eu-west-1.amazonaws.com/v1/data-channel/oscars@blank.com-0473707d8049c843c?role=publish_subscribe&cell-number=***", "ResponseMetadata": {"RequestId": "f44033cc-12da-4617-81ad-195e17ef2e69", "HTTPStatusCode": 200, "HTTPHeaders": {"server": "Server", "date": "Mon, 11 Apr 2022 18:06:58 GMT", "content-type": "application/x-amz-json-1.1", "content-length": "839", "connection": "keep-alive", "x-amzn-requestid": "f44033cc-12da-4617-81ad-195e17ef2e69"}, "RetryAttempts": 0}}', 'eu-west-1', 'StartSession', 'prod', '{"Target": "i-07dd6exxxxxxxxd1"}', 'https://ssm.eu-west-1.amazonaws.com']' returned non-zero exit status 3221225786. Starting an RDP session returns a timeout even though Reachability Analyzer states that the instance is reachable via TCP on port 3389 from another instance. Basically the instance has become unreachable and I can only stop and start it in the console.

I am getting the below error while creating the SSM Parameter via CF passign version. Property validation failure: [Encountered unsupported properties in {/}: [Version]] **My Securitygroup.Yaml** AWSTemplateFormatVersion: 2010-09-09 Description: Basic SSH Security Group. Resources: MySSHSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: my new SSH SG SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: 0.0.0.0/0 MYSSHSecuritygroupParameter: Type: AWS::SSM::Parameter Properties: Name: Security Type: String **Value: !Ref MySSHSecurityGroup** Tier: Standard **Version: 1** Description: My SSH Securitygroup on SSM Parameter Please help on this how to pass with Version number in CF, i need to create difference type of versions needs to create on the Same Parameter using CF.

I am trying to figure out how to download file(s) from S3, using an SSM Automation document. Note, this is not a "Command" document type, as I need to use Assume Role. The instances themselves shouldn't have access to the bucket by default, which is why I need the Assume Role bit. DownloadContent with a "Command" document type requires the instance to have the IAM policies/roles attached that can read the bucket. Is there a way to do this without having the iam policy on each instance being modified/have access to the bucket?

Hi, I would like to get the value of ApproverDecisions.Comment from aws:approve to use as input to the next step. Is there any specific syntax on this? ``` ApproverDecisions { "Approver": "xxx", "Decision": "Approve", "ApprovalDecisionTime": "2022-04-06T07:22:51.034463Z", "Comment": "Approve because it is valid" } ``` I have tried the following but not working ``` mainSteps: - name: ReviewImage action: 'aws:approve' inputs: NotificationArn: 'XXX' Message: 'Notification Hello' MinRequiredApprovals: 1 Approvers: - 'XXXX' outputs: - Name: CommentVal Selector: $.ApprovalDecisions[0].Comment Type: String - Name: CommentVal2 Selector: $.ApprovalDecisions.Comment Type: String - name: TestLogging action: 'aws:runCommand' inputs: DocumentName: AWS-RunPowerShellScript InstanceIds: - XXXXX Parameters: commands: - ' $comment = {{ ReviewImage.CommentVal }} ' - ' Write-Host "$comment" ' - ' $comment2 = {{ ReviewImage.CommentVal2 }} ' - ' Write-Host "$comment2" ' ```

I'm new to SSM, and just wanted to set up a test agent on a windows VM. I followed the [hybrid environment tutorial](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-managedinstances.html). It resulted in the fowwlowing error, when running the agent installation script (in the $env:ProgramData\Amazon\SSM\Logs\erros.log file): `ERROR [processRegistration @ agent_parser.go.177] Registration failed due to error registering the instance with AWS SSM. InvalidActivation: ` As you can see, the details about the InvalidActivation are missing... What can I do to track the error ?

I have question around AWS SSM Patch manger custom Patch baseline. I create a custom patch baseline for Windows servers and add to Patch group, so far good. I tried to use this custom patch baseline in Maintenance Window task, Couldn't find anything. Only option for Run_Command is AWS-RunPatchBaseline which is default, not the custom that I create.

I have used the automation-updatessm document successfully but now it keeps getting me the error below: FunctionName "Automation-UpdateSsmParam" Payload "{\"parameterName\":\"/ami/latest/us-east-1/win2k16ah-cb\", \"parameterValue\":\"ami-078c47fc7714f0cb3\"}" OutputPayload {"Payload":"SSM parameter not found.","StatusCode":200} Payload SSM parameter not found. StatusCode 200 I have verified that the parameterName does exist. I wish there was a way to attach screenshots here.

I have a script that runs across our estate. It includes running a SQL script, and ideally I need to capture the output into a text file on that instance. As the Powershell script uses Invoke-SQLCMD, i need to redirect the verbose output to capture it. This works as intended when running the script\line in a PS window direct on the instance and the verbose output is captured in my file $logfile: `(Invoke-SQLCMD -ServerInstance ".\$InstanceName" -Query $OptimizeIndexesSQL -Verbose -ErrorAction Stop) 4>&1 | Tee-Object -FilePath $LogFile -Append` When the script runs via SSM, it fails to push the verbose output to the file. Why?

Hello, I've cloned the AWS-UpdateWindowsAMI document and added two steps for invoking Lamda function to update the parameter store with the latest AMI ID and creating EC2 tags. I'm getting the error below when I run into the aws:invokelambdafunction block. Everything else works. > Step fails when it is validating and resolving the step inputs. Failed to resolve input: createImage.ImageId to type String. createImage is not defined in Automation Document MainSteps.. Please refer to Automation Service Troubleshooting Guide for more diagnosis details. VerificationErrorMessage Failed to resolve input: createImage.ImageId to type String. createImage is not defined in Automation Document MainSteps. I've copied the LamdaFunction code straight of [this document](https://docs.aws.amazon.com/systems-manager/latest/userguide/automation-walk-patch-windows-ami-simplify.html#automation-pet1). This is the updatessmparam code in the document. I've only replaced the parameterName in the code. { "name": "CreateImage", "action": "aws:createImage", "maxAttempts": 3, "onFailure": "Abort", "inputs": { "InstanceId": "{{ LaunchInstance.InstanceIds }}", "ImageName": "{{ TargetAmiName }}", "NoReboot": true, "ImageDescription": "{{ TargetImageDescription }}" } }, { "name": "TerminateInstance", "action": "aws:changeInstanceState", "maxAttempts": 3, "onFailure": "Abort", "inputs": { "InstanceIds": [ "{{ LaunchInstance.InstanceIds }}" ], "DesiredState": "terminated" } }, { "name": "updateSsmParam", "action": "aws:invokeLambdaFunction", "timeoutSeconds": 1200, "maxAttempts": 1, "onFailure": "Continue", "inputs": { "FunctionName": "Automation-UpdateSsmParam", "Payload": "{\"parameterName\":\"/ami/latest/us-east-1/win2k19sf-cb\", \"parameterValue\":\"{{createImage.ImageId}}\"}" } }, Any help would be appreciated. Thanks. Justin

I'm trying to execute this command on an on-premises Amazon Linux 2 instance: ``` amazon-ssm-agent -register -code "<removed_code>" -id "<removed_id>" -region "us-east-1" ``` It fails with : ``` Error occurred fetching the seelog config file path: open /etc/amazon/ssm/seelog.xml: no such file or directory Initializing new seelog logger New Seelog Logger Creation Complete 2022-03-22 23:22:51 ERROR Registration failed due to error registering the instance with AWS SSM. InvalidParameter: 1 validation error(s) found. - minimum field size of 36, RegisterManagedInstanceInput.ActivationId. ``` /var/log/amazon/ssm/errors.log contains: ``` 2022-03-22 23:24:07 ERROR [newAgentIdentityInner @ identity_selector.go.96] Agent failed to assume any identity 2022-03-22 23:24:07 ERROR [NewAgentIdentity @ identity_selector.go.109] failed to find identity, retrying: failed to find agent identity 2022-03-22 23:24:13 ERROR [newAgentIdentityInner @ identity_selector.go.96] Agent failed to assume any identity 2022-03-22 23:24:13 ERROR [NewAgentIdentity @ identity_selector.go.109] failed to find identity, retrying: failed to find agent identity 2022-03-22 23:24:19 ERROR [newAgentIdentityInner @ identity_selector.go.96] Agent failed to assume any identity 2022-03-22 23:24:19 ERROR [Init @ bootstrap.go.75] failed to get identity: failed to find agent identity 2022-03-22 23:24:19 ERROR [run @ agent.go.130] error occurred when starting amazon-ssm-agent: failed to get identity: failed to find agent identity 2022-03-22 23:25:04 ERROR [processRegistration @ agent_parser.go.177] Registration failed due to error registering the instance with AWS SSM. InvalidParameter: 1 validation error(s) found. - minimum field size of 36, RegisterManagedInstanceInput.ActivationId. ```

I want to connect the instance with windows-server 2019 datacenter as OS. But the SSM agent service cannot be started either by Power Shell or windows service manage panel. In Powershell, it prompt like below. ``` Start-Service : Failed to start service 'Amazon SSM Agent (AmazonSSMAgent)'. At line:1 char:1 + Start-Service AmazonSSMAgent + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException + FullyQualifiedErrorId : StartServiceFailed,Microsoft.PowerShell.Commands.StartServiceCommand ``` Meanwhile, I have created ssm-user and added it to Admin group. But it is not helpful towards issue. I believe it is SSM configuration issue. But having no idea on how to troubleshoot. This issue only appears in Tokyo region, while the SSM Quick Start website layout is different from what we see in HongKong region.

I'm going to share parameter stored in SSM Parameter Store to all accounts in my AWS Organization. I would like track when then parameter was read (to gather statistics for parameter usage across my accounts) I noticed that EventBridge is only catching SSM Parameter Store changes like: Update, Delete, Create Is there an option to track reads?

I'm trying to follow this post https://aws.amazon.com/blogs/security/how-to-enable-secure-seamless-single-sign-on-to-amazon-ec2-windows-instances-with-aws-sso/ But I have an error message related with SSO **"An error occurred while calling the ListDirectoryAssociations API operation. SSO features are disabled. AccessDeniedException: SSO is not shared in org: xxxxx"** I have configured SSO with AzureAD as external provider. Can somebody help me with this issue? Regards

> Systems Manager advanced instances are priced on a pay-as-you-go basis. You are charged based on the number of advanced instances activated as Systems Manager managed instances and the hours those instances are running. [[System Manager Pricing](https://aws.amazon.com/systems-manager/pricing/#On-Premises_Instance_Management)] So does this only include nodes registered on Fleet Manager? If, for example, the outgoing System Manager traffic was blocked from an instance would it cease to be charged as an advanced tier instance?

I am unable to take RDP of ec2 instance since it is gone to safe mode with network, is it possible to enable SAC by using AWS CloudShell

is there is any way, where we can get the data/report of ssm agent in aws instances.

I am running into something that seems happening without a trigger. I have 111 EC2 instances, a mix of various versions of Windows and Linux. Everyday at about 18:30 GMT a seemingly random combination of servers patch. Nothing that I can find is triggering it. There are 4 maintenance windows, all currently disabled. And they were not for that time anyway. Is there something else I can look at? Where\What else can trigger patching? It appears that I am missing something. Can anyone here shed any light on what's going on and\or what I'm missing\did wrong?

I want to create a folder in the shared top directory of FSx. Flow Powershell commands using ssm's AWS-RunPowerShellScript. However, the executing user is ssm-user. This user is not an active directory user and access is denied. The New-Item command does not set credentials. Is it possible to create a folder in the FSx shared top directory by running the powershell command from ssm?

I would like to run powershell commands on a specified EC2 instance using AWS-RunPowerShellScript in Systems Manager's Run Command. I want to create a folder in the shared top directory of FSx, so I carried out the following command. ``` $path = FSx network drive path New-Item -Path "$path\D$\folder-name" -ItemType directory ``` However, access is denied. I get an error message "Unable to connect to remote server" even after executing the following commands. ``` $name = FSx name Invoke-Command -Credential $domainCredential -ComputerName $name -ScriptBlock { New-Item -Path "\D$\folder-name" -ItemType directory } ``` I would like to know what I should do.

Hi All, I have an EC2 instance in a private subnet, I connect to it using session manager via AWS console. actually, the outbound rule of the security Group of the private EC2 instance is : All traffic / all/ 0.0.0.0/0 when I delete that rule I cannot anymore connect to the EC2 instance : ``` Your session has been terminated for the following reasons: ----------ERROR------- Setting up data channel with id xxxxxxxxx-04retceff7ddr5 failed: failed to create websocket for datachannel with error: CreateDataChannel failed with no output or error: createDataChannel request failed: failed to make http client call: Post "https://ssmmessages.region1.amazonaws.com/v1/data-channel/xxxxxxxxx-04fgffgffdgefbdder": context deadline exceeded (Client.Timeout exceeded while awaiting headers) ``` what is the right outbound SG rule that allows me to connect to my instance via AWS console session manager knowing that I don't have a VPC interface for SSM?

I would like to run powershell commands on a specified EC2 instance using AWS-RunPowerShellScript in Systems Manager's Run Command. In doing so, I would like to use the dsadd-user command to add a user in the Active directory. However, when I actually execute this, I get an "Access is denied" error because I don't have access rights. My thinking is that this is because when I run this command with ssm it is running as a local user named ssm-user. I would like to run the command as a user belonging to the active directory. Is that possible? Thank you very much.

Hello, I use not only the AWS firewall but also UFW on my Debian 10 instance... So, now UFW block me. - Instance Connect don't work because I'm using Debian 10 - Since I cannot connect. I cannot install Session Manager. - Of course the daemon of UFW start at launch... Serial Console ask for a login, I did a user...etc But the login is not working. I'm missing something? Help please.

The AmazonSSMManagedInstanceCore managed policy includes **Resource: *** in all of its permission clauses, including for **ssm:GetParameter[s]**. I do not wish to give all of my instances permissions to read all of our parameters, and there are likely other resources I do not want them all having access to (**PutInventory** seems like another one I might prefer to tighten). What resources, parameters in particular, does SSM actually need access to for managing instances?

Problem Statement: We are using System Manager Service for patching our servers, but we are facing one challenge , The process of upgrading the servers are same like updating and patching server via downloading packages from UBUNTU repo over https .i.e. on PORT 80, but we can't open the port 80 as a security compliance on our servers. Pls help and guide us, then how SMS will upgrade the servers and patch them. if PORT 80 is closed then "defaultbasepatchline" fails over the servers. Thanks.

Hello, Inspector2 is reporting that a freshly-built EC2 instance is vulnerable to [CVE-2021-44731](https://nvd.nist.gov/vuln/detail/CVE-2021-44731) (affecting `snap` v2.54.2), but our installed version is 2.54.3: ``` $ snap --version snap 2.54.3+18.04.2ubuntu0.1 snapd 2.54.3+18.04.2ubuntu0.1 series 16 ubuntu 18.04 kernel 5.4.0-1066-aws ``` I haven't found any other versions of `snap` on the instance. Any idea what would cause this? Thanks in advance, Eugene

Is there a way to specify, on Windows, that the SSM Agent runs a powershell script using PowerShell 7 vs. PowerShell 5?

I've tried to configure my MWAA (wih 1.10.12, 2.0.2) with ssm backend with below configuration options: ``` secrets.backend: airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend secrets.backend_kwargs: {"connections_prefix":<conn_prefix>, "variables_prefix":"<var_prefix>"} ``` It starts up the environment and status becomes 'Available' for use, yet while accessing the airflow UI, I'm getting gateway errors. I've also added permission for SSM. Am I missing something or there is a known bug which I'm unaware of?

Hello , We patch our systems monthly but we needed a patch status report once every patch cycle is completed .Suppose we have around 100 servers out of which 90 got patched and 10 got failed .If an automated report can give the patching status then it is going to be so easy to fins those machines where not patched . Thanks Sujith

Can an existing SG be deployed using SSM Powershell to all EC2 instances? If it is possible, then would like to then remove the same group after done using it. I found little information on how to accomplish this.

Context: - we use `aws:executeAutomation` inside another SSM automation - we use `Targets` and `TargetParameterName` in combination with `MaxConcurrency` to execute on one instance at a time - the automation erroneously executes the first instance twice and then exits with: `Overall status: Failed` even though **none** of the steps returned any failures In other words, instead of executing: Instance A (Step#1) -> Instance B (Step#2), it executes Instance A (Step#1) -> Instance B (Step#2) -> Instance A (Step#1) If the child automation **is not executed from a parent** (meaning that it is executed from the cli directly), then the behavior is as expected: just 2 steps are executed, one for each of the 2 instances and the result is successful Also, a thing of note is that the cli command `aws ssm get-automation-execution --automation-execution-id` shows only the first two steps of the erroneous execution. So, the management console shows 3 steps in total, but the CLI only the first 2. If you query the third step (the one who **should not exist**) using the cli, it shows the parent correctly, but the parent does not see it Please fix this bug :D

Have a Control Tower Setup and in main account have set ABAC - SSMSessionRunAs = ${user:name} in AWS SSO. In one of the Workload accounts, I have configured Systems Manager Preferences with "Run As" but with empty user. The expected behaviour is that sessions in System Manager will be created with the AWS user account (not ssm-user). However error "Invalid RunAs username. Set default username in Session Manager Preferences page." is displayed. Of course, if I set the Run As in Systems Manager Preferences to ssm-user the Systems Manager session connects as ssm-user (not the AWS user account). A matching user account has been added to the Linux Amazon OS. It appears the ABAC variable isn't passed through to Systems Manager? The strange thing is this worked yesterday? I have also tried ABAC ${path:userName}.

Hello, I have the following VPC created with CDK: ``` this.vpc = new Vpc(this, 'vpc', { cidr: '10.0.0.0/21', natGateways: 0, subnetConfiguration: [ { subnetType: SubnetType.PUBLIC, cidrMask: 24, }, { subnetType: SubnetType.PRIVATE_ISOLATED, cidrMask: 28, }, ], }); ``` The RDS instance is created in the private isolated subnet. When I create a Bastion to access RDS as follows: ``` const bastionSecurityGroup = new SecurityGroup(this, 'bastion-host-security-group', { vpc: props.vpc, allowAllOutbound: true, }); new BastionHostLinux(this, 'bastion-host', { vpc: props.vpc, subnetSelection: props.vpc.selectSubnets({ subnetType: SubnetType.PUBLIC }), securityGroup: bastionSecurityGroup, }); this.dbSecurityGroup.addIngressRule(bastionSecurityGroup, Port.tcp(5432), 'Allow Access from Bastion', true); ``` I'm able to access it via SSM normally from my machine. However, if I omit the `subnetSelection` property and the Bastion is placed in the private isolated network, it is no longer accessible. I'm unable to get my head around what I need to do be able to access it without placing it in the public subnet. I understand that I may do so by adding a VPC Interface Endpoint, but I don't see how to do that in CDK above.

Hi, I created a basic custom inventory for Disk usage by creating a Systems manager association. I am running a powershell script to get the disk information but unable to see the inventory in the console. Script runs fine and was written to the intended location in the target machine however the inventory doesn't show up in the console. Here's my script. function Get-EbsVolumes {[CmdletBinding()] $Map = @{"0" = '/dev/sda1'}; for($x = 1; $x -le 26; $x++) {$Map.add($x.ToString(), [String]::Format("xvd{0}",[char](97 + $x)))}; for($x = 78; $x -le 102; $x++) {$Map.add($x.ToString(), [String]::Format("xvdc{0}",[char](19 + $x)))} return Get-WmiObject -Class Win32_DiskDrive | % { $Drive = $_; $SN = $Drive.SerialNumber -replace '^(vol)([^ _]+)(?: *_.+)?$', '$1-$2'; Get-WmiObject -Class Win32_DiskDriveToDiskPartition | Where-Object {$_.Antecedent -eq $Drive.Path.Path} | %{ $D2P = $_; $Partition = Get-WmiObject -Class Win32_DiskPartition | Where-Object {$_.Path.Path -eq $D2P.Dependent}; $Disk = Get-WmiObject -Class Win32_LogicalDisk`ToPartition | Where-Object {$_.Antecedent -in $D2P.Dependent} | %{ $L2P = $_; Get-WmiObject -Class Win32_LogicalDisk | Where-Object {$_.Path.Path -in $L2P.Dependent} | %{ $L2V = $_; Get-WmiObject -Class Win32_Volume | Where-Object {$_.SerialNumber -eq ([Convert]::ToInt64($L2V.VolumeSerialNumber, 16))}; } } New-Object PSObject -Property @{ Device = $Map[$Drive.SCSITargetId.ToString()]; Disk = [Int]::Parse($Partition.Name.Split(",")[0].Replace("Disk #","")); Boot = $Partition.BootPartition; Partition = [Int]::Parse($Partition.Name.Split(",")[1].Replace(" Partition #","")); DriveLetter = If($Disk -eq $NULL) {"NA"} else {$Disk.DriveLetter}; VolumeDeviceID = If($Disk -eq $NULL) {"NA"} else {$Disk.DeviceID}; DriveType = If($Disk -eq $NULL) {"NA"} else {$Disk.DriveType}; IsSystemVolume = If($Disk -eq $NULL) {"NA"} else {$Disk.SystemVolume}; IsBootVolume = If($Disk -eq $NULL) {"NA"} else {$Disk.BootVolume.ToString()}; DriveLabel = If($Disk -eq $NULL) {"NA"} else {$Disk.Label}; BlockSize = $Disk.BlockSize; Capacity = ([math]::Round($Disk.Capacity /1GB,0)).ToString(); DiskType = $Partition.Type; FreeSpace = [math]::Round($Disk.FreeSpace /1GB,0); SerialNumber = $SN } } } | Where-Object { $_.DriveType -eq 3 -and $_.IsSystemVolume -eq $false} } $data = Get-EbsVolumes | Select-Object @{n="Device";e={$_.Device}},@{n="Disk";e={$_.Disk.ToString()}},@{n="Boot";e={$_.Boot.ToString()}},@{n="DriveLetter";e={$_.DriveLetter.ToString()}},@{n="VolumeDeviceID";e={$_.VolumeDeviceID}},@{n="Partition";e={$_.Partition}},@{n="DriveType";e={$_.DriveType.ToString()}},@{n="IsSystemVolume";e={$_.IsSystemVolume.ToString()}},@{n="IsBootVolume";e={$_.IsBootVolume.ToString()}},@{n="DriveLabel";e={$_.DriveLabel.ToString()}},@{n="BlockSize";e={$_.BlockSize.ToString()}},@{n="Capacity";e={$_.Capacity.ToString()}},@{n="DiskType";e={$_.DiskType.ToString()}},@{n="FreeSpace";e={$_.FreeSpace.ToString()}},@{n="SerialNumber";e={$_.SerialNumber.ToString()}} | ConvertTo-Json -Compress $content = "{`"SchemaVersion`" : `"1.0`", `"TypeName`": `"Custom:EbsVolumesInventory`", `"Content`": $data}" $instanceId = Invoke-RestMethod -uri http://169.254.169.254/latest/meta-data/instance-id $filepath = "C:\ProgramData\Amazon\SSM\InstanceData\" + $instanceId + "\inventory\custom\EbsVolumesInventory.json" if (-NOT (Test-Path $filepath)) { New-Item $filepath -ItemType file} Set-Content -Path $filepath -Value $content

I'm trying to use EventBridge to listen for EC2 autoscaling termination events, and send a shell command to the instance to do some work before the instance terminates. I followed this guide: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-ec2-run-command.html However, it doesn't explain how Target Key and Target Value input boxes are used. I don't understand why it uses tag:environment in this case. I assume I'm supposed to input the instance id of the EC2 instance, and I know that the instance id is in the json body of the termination event. Is there a way I can pass variables to the Target Key and Target Value?

When I run the df command on my servers the results displayed show my EFS mounts. when I run the command using System Manager and doc aws:runShellScript the EFS mounts are not show but all non EFS mounts are shown. Why is this ? Am I missing a parameter ? These are RHEL 7.9.

My scenario: I have my users in Azure AD. This is connected to AWS Single Account SSO into an AWS Account using IAM SAML IDP (PS: we are not using AWS SSO Service). We are using AWS Secrets Manager and want to store per user secret using a secret name path (eg /usersecrets/<azure_ad_username>/<secret_name> When the users login using Azure AD auth, they automatically assume the IAM Role attached. I would like to do the following: Requirement1: 1. Allow users to list secrets, create secrets and get secret value for any secret which has a name /usersecrets/<azure_ad_username>/* (here the azure_ad_username is what AWS session sees when the assume role to login) 2. Deny access to any secret unless the request is coming from Federated user (i.e local IAM users in AWS account should not be able to see any secret in path /usersecrets/<azure_ad_username>/* Requirement2: In addition to the federates Azure AD users, I also want to allow a EC2 Instance Role to be able to Get/List/Describe any secret. This EC2 role is in same AWS account where secrets are and is attached to all Windows Servers. This IAM role is to allow SSM Run commands to execute on these Windows machines and fetch the secrets values (eg, to get the secret of a user and create a local windows user with same name and password as it is in secret manager using powershell. Questions: Can you help with some sample IAM Policy for the role or the secret manager resource policy I can use to meet both the requirements?

Hey all, hoping someone can help me out here. I'm running SSM on a few instances in a hybrid setup, ie it is installed on actual PCs. All the PCs are Ubuntu. SSM works wonderfully on there. However, today I did an OS upgrade from Ubuntu 18 to 20 and the SSM agent went mental. The main error is this ``` Jan 25 18:46:16 p0xxxxx amazon-ssm-agent\[3863\]: 2022-01-25 18:46:15 INFO \[OnPremCredsProvider\] Calculated hardware difference, regenerating fingerprint... Jan 25 18:46:16 p0xxxxxx amazon-ssm-agent\[3863\]: 2022-01-25 18:46:15 ERROR \[CredentialRefresher\] Retrieve credentials produced unrecoverable aws error: MachineFingerprintDoesNotMatch: Fingerprint does not match 9366ec5a-de7a-4aeb-a6db-998a0f1747d7 ``` It is reporting that everything changed, ip, disk, memory, cpu etc etc so it is generating a completely new fingerprint for the box. I have two questions on this 1. Can I somehow change the fingerprint in SSM so that this can connect again, or do I have to do a new activation? 2. How can I prevent this? I know in the AWS docs there is a command you're supposed to be able to run where you umount /etc/machine-id and then do something else. On my boxes the machine-id is not mounted so the command seems to have no effect.

How do you publish an AWS Distributor Package third-party agent and make it appear in the Third-Party tab? Is it possible to monetize an AWS Distributor Package third-party agent?

Hi all. We're just starting with SSM and hoping to use this quite extensively moving forward. I have a small handful of devices online using it but I just noticed that one of my devices has gone offline. Luckily it is not in the field yet so I was able to access it locally. What appears to have happened was SSM Agent was attempting to update but failed and never came back. I'm not so concerned that it didn't update and more concerned that it didn't start again. Going through the ssm agent logs I came across this line ``` "standardError": "E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\nWARNING: Could not install the python3-apt, this may cause the patching operation to fail.\nfailed to run commands: exit status 1 ``` The SSM update log itself ends like this ``` 2022-01-20 21:47:09 DEBUG UpdateInstanceInformation Response{ } 2022-01-20 21:47:09 INFO initiating cleanup of other versions in amazon-ssm-agent and amazon-ssm-agent-updater folder 2022-01-20 21:47:09 INFO removing artifacts in the folder: /var/lib/amazon/ssm/update/amazon-ssm-agent 2022-01-20 21:47:09 INFO removed files and folders: 3.1.821.0 2022-01-20 21:47:09 INFO removing artifacts in the folder: /var/lib/amazon/ssm/update/amazon-ssm-agent-updater 2022-01-20 21:47:09 INFO removed files and folders: 3.1.715.0 2022-01-20 21:47:09 INFO initiating cleanup of files in update download folder 2022-01-20 21:47:09 INFO Successfully downloaded manifest Successfully downloaded updater version 3.1.821.0 Updating amazon-ssm-agent from 3.1.715.0 to 3.1.821.0 Successfully downloaded https://s3.us-east-2.amazonaws.com/amazon-ssm-us-east-2/amazon-ssm-agent/3.1.715.0/amazon-ssm-agent-ubuntu-amd64.tar.gz Successfully downloaded https://s3.us-east-2.amazonaws.com/amazon-ssm-us-east-2/amazon-ssm-agent/3.1.821.0/amazon-ssm-agent-ubuntu-amd64.tar.gz Initiating amazon-ssm-agent update to 3.1.821.0 failed to install amazon-ssm-agent 3.1.821.0, ErrorMessage=The execution of command returned Exit Status: 125 exit status 125 Initiating rollback amazon-ssm-agent to 3.1.715.0 failed to uninstall amazon-ssm-agent 3.1.821.0, ErrorMessage=The execution of command returned Exit Status: 2 exit status 2 Failed to update amazon-ssm-agent to 3.1.821.0 ``` Then the error log gives me this ``` 2022-01-20 16:29:54 ERROR [Submit @ processor.go.140] [ssm-agent-worker] [MessagingDeliveryService] [Association] [associationId=5752f0d0-1f57-492e-83f7-740484b81d73] Document Submission failed: Job with id 5752f0d0-1f57-492e-83f7-740484b81d73 already exists 2022-01-20 21:47:02 ERROR [AppendError @ context.go.129] failed to install amazon-ssm-agent 3.1.821.0, ErrorMessage=The execution of command returned Exit Status: 125 exit status 125 ``` Could anyone help me out here? I cannot have these fail like this when they go out into the wild. The OS is Ubuntu Server 18 on this box, 20 on others we have.

I am using an Automation Runbook to install and configure CloudWatch Agent on managed instances. When I execute my Runbook on a tag-based Resource group, it attempts to run on instances that do not have "running" status. When it ran on stopped instances, it would eventually time out. How can I only target running instances in my Resource Group for execution? Things I have tried or looked into: * Adding a step to the start of my Runbook to assertAwsResourceProperty, DescribeInstanceStatus, and "running". This prevents the timeout, but returns nothing when run against a stopped instance, therefore Aborting and being marked as a failure. This is undesirable to me, because I don't see the Runbook as having failed, rather skipped execution for a legitimate reason. Furthermore, if run on a large batch of machines, this leads to even one stopped instance ending in the entire parent execution being marked as a failed. * Filtering my Resource Group to only contain running instances. I saw no way of doing this. * Adding a Property to my Runbook so that it can only be run on a running instance. Similar to how I use the TargetType property to limit it to /AWS::EC2::Instance. This made the most sense to me, because as the developer I know my Runbook can only be successful if the instance is running. I wanted something akin to being able to set my TargetType as service-provider::service-name::data-type-name::status. I haven't found any way of doing this. * Applying a filter when picking my Targets for execution in the Systems Manager console. This is another place where it made sense that I might find it, but I didn't . If, when I've chosen the Resource Group, the Interactive Instance picker were to update to display only the instances from that group, that would be unwieldy but at least allow me to manually deselect instances that are not running. None of this appears to be the case. Is there a simple way to accomplish this that I have overlooked?

hi.. is there a possibility to run an automation document starting from step 5 for example all the way to the end? and invoke that as a parameter?

In the SSM Fleet Manger, a list of Native and Hybrid Cloud Managed instances are displayed. Is it possible to include, in the columns on the list, and/or on the report that can be downloaded using the Download Report button on this page, any tags and their values, similar as to what can be displayed in the EC2 Instances list view?

please can you advice how can we automatic Secret,password store in parameter store to make process fully automatic and use cli to retrieve

Simple doubts : - if my private linix instances have no internet and i use ssm endpoints for patching, who will download the patches, SSM ?

Hi all, I'm using SSM on some hybrid linux nodes. I was going through the documentation and there is a mention of being able to use SSM to check open network ports on the nodes but there isn't any example of how to do it. I'm trying to setup a proof of concept right now and if there is a tutorial on how to do that that would be awesome, and would help my case. Are there any available resources?

Hey all, I have a question about using SSM to verify a config. I'm setting up a proof of concept and the last thing I'd like to figure out is if it's possible to check a specific file on a node and see what its contents is. I have hybrid ubuntu nodes and there's a specific file which is used in our application that I'd love to be able to check. Can I do something like that and if it's not too much trouble could someone point me to some resource which would help me figure out how? Ultimately if we go this way I see SSM checking a bunch of things and possibly even managing it automatically but we'll get there. Thank you so much.

I have a CDK Pipeline (aws-cdk v2.3.0) where I would like to share parameters across stages. When I try to pass the resource across stages, I get `dependency cannot cross stage boundaries`. Right now, I'm resorting to putting the variables in ssm string parameters. Is there a better way to share parameters across stages?

I have multiple machines running hybrid SSM Agent. Those machines in one network suffered a multi-day network outage. When the network issue was restored SSM Agent wouldn't 'reconnect'. I cannot start sessions to access these machines. Here are the relevant log lines from `/var/log/amazon/ssm/amazon-ssm-agent.log`: ``` 2021-12-23 13:42:22 INFO [ssm-agent-worker] [MessagingDeliveryService] increasing error count by 1 2021-12-23 13:42:24 ERROR [ssm-agent-worker] [MessagingDeliveryService] error when calling AWS APIs. error details - GetMessages Error: shared credentials are already expired, they were queried at 2021-12-21T11:30:10-06:00 and expired at 2021-12-21T18:30:10Z 2021-12-23 13:42:24 INFO [ssm-agent-worker] [MessagingDeliveryService] increasing error count by 1 2021-12-23 13:42:26 ERROR [ssm-agent-worker] [MessagingDeliveryService] error when calling AWS APIs. error details - GetMessages Error: shared credentials are already expired, they were queried at 2021-12-21T11:30:10-06:00 and expired at 2021-12-21T18:30:10Z 2021-12-23 13:42:26 INFO [ssm-agent-worker] [MessagingDeliveryService] increasing error count by 1 2021-12-23 13:42:29 ERROR [ssm-agent-worker] [MessagingDeliveryService] error when calling AWS APIs. error details - GetMessages Error: shared credentials are already expired, they were queried at 2021-12-21T11:30:10-06:00 and expired at 2021-12-21T18:30:10Z 2021-12-23 13:42:29 INFO [ssm-agent-worker] [MessagingDeliveryService] increasing error count by 1 2021-12-23 13:42:31 ERROR [ssm-agent-worker] [MessagingDeliveryService] error when calling AWS APIs. error details - GetMessages Error: shared credentials are already expired, they were queried at 2021-12-21T11:30:10-06:00 and expired at 2021-12-21T18:30:10Z 2021-12-23 13:42:31 INFO [ssm-agent-worker] [MessagingDeliveryService] increasing error count by 1 2021-12-23 13:42:33 ERROR [ssm-agent-worker] [MessagingDeliveryService] MessagingDeliveryService stopped temporarily due to internal failure. We will retry automatically after 15 minutes ``` That seems to repeat round and around. The credentials are now a couple of days old as can be seen by the timestamps. I am assuming the "internal failure" is trying to refresh the tokens. I restarted the agent on one machine (through `systemctl restart`) and it came back fine. So it's definitely some state in the running agents that is the problem. I have left the others in their failed state in case someone responds with something for me to test this further.

Hi, Whenever I type the character `a` in the search bar on the AWS Systems Manager Session Manager console, the whole console goes white and everything gets disappeared. I have to refresh the page to make everything appear again. URL: https://eu-central-1.console.aws.amazon.com/systems-manager/session-manager/sessions?region=eu-central-1 I think it is a bug and needs to be fixed or am I the only one who experiences this? Best Regards, Abdullah Khawer

Hello All, I am looking for a way to interactively run shell commands in multiple instances simultaneously. I know I can run (shell) commands interactively with a **single** instance through SSM console, and **non-interactive** scripts, document, with multiple instances through Run Command. I am using the SSH client tool with 'multi-execution mode' to perform this kind of interactive work. However, I have to open the port to do the work. If Systems Manager can add 'multi-execution mode' to SSM or interactivity to Run Command, or just a new tool. Then I do not have to poke a hole in my VPC to do the work. Best regards, Tiger

I use `aws-sdk` gem for my rails project to retrieve data from Parameter Store. Aws documentation says that `There is no charge from Parameter Store to create a SecureString parameter, but charges for use of AWS KMS encryption do apply` - which I understand To retrieve any parameters(string or secure string) I use the following code ``` Aws::SSM::Client.new( region: region ).get_parameter( name: parameter_id, with_decryption: true ).to_h ``` As you can see I pass `with_decryption: true` to get the parameter, regardless it is a secure string or not. My doubt is does it charge me if I use `with_decryption: true` while retrieving string parameter? Should I omit `with_decryption: true` when I retrieve a string parameter so that the AWS system manager does not use unnecessary decryption? Or does the AWS system manager just skip `with_decryption: true` if the parameter is not a secure string?

Hi all, on a Cloud9 instance that is deployed in a Private subnet w/ a NAT gateway with SSM access, AWS CLI works except `aws sts get-caller-identity` . How is this even possible? I was checking AWS CLI connectivity using get-caller-identity but now it fails while other AWS CLI APIs work. ``` admin:~/environment $ aws s3 ls s3://aws-cloudtrail-logs-***** PRE AWSLogs/ admin:~/environment $ aws sts get-caller-identity An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation: The security token included in the request is invalid ``` **How to reproduce:** - Create a private subnet in default VPC - Create a NAT gateway in a public subnet in default VPC - Create a route table allows engress connections to internet - Assign the route table to the private subnet that just created - Navigate to Cloud9 console, create a Cloud9 instance on the Private subnet with SSM access **(not direct access or SSH access)** - On the Cloud9 console, try aws cli with various commands.

I just starting using SMS to manage Windows 2019 Server EC2 instance patching (security updates). I noticed that by default, AWS prevents Windows OS to automatically run Windows Update. I followed the instructions for SMS Quick Setup and the Patching of my servers are failing with the following error message: (I have been searching ALL day for a resolution to this. Modifying registry settings, running DSIM commands, etc. Nothing helps. Seems like some type of certificate issue but I can't resolve it). Has anyone else had issues with getting SMS to patch AWS Windows Server 2019 EC2 instances? **Invoke-PatchBaselineOperation : Exception Details: An error occurred when attempting to search Windows Update. Exception Level 1: Error Message: A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider. (Exception from HRESULT: 0x800B0109)** Stack Trace: at WUApiLib.IUpdateSearcher.Search(String criteria) at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.WindowsUpdateAgent.SearchForUpdates(String searchCriteria) At C:\ProgramData\Amazon\SSM\InstanceData\i-03638bdca902ef8fd\document\orchestration\86ed2eda-065a-49d3-b084-69bfc89c14 3d\PatchWindows\_script.ps1:233 char:13 + $response = Invoke-PatchBaselineOperation -Operation Scan -SnapshotId ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : OperationStopped: (Amazon.Patch.Ba...UpdateOperation:FindWindowsUpdateOperation) [Invoke -PatchBaselineOperation], Exception + FullyQualifiedErrorId : Exception Level 1: Error Message: Exception Details: An error occurred when attempting to search Windows Update. Exception Level 1: Error Message: A certificate chain processed, but terminated in a root certificate which is not trusted by the t rust provider. (Exception from HRESULT: 0x800B0109) Stack Trace: at WUApiLib.IUpdateSearcher.Search(String criteria) at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.WindowsUpdateAgent.SearchForUpdates(String searc hCriteria) Stack Trace: at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.WindowsUpdateAgent.SearchForUpdates( String searchCriteria) at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.WindowsUpdateOperation.SearchAndProcessResult(Lis t`1 kbGuids) at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.WindowsUpdateOperation.SearchByGuidsPaginated(Lis t`1 kbGuids, Int32 maxPageSize) at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.WindowsUpdateOperation.FilterWindowsUpdateSearch( List`1 filteringMethods) at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.FindWindowsUpdateOperation.DoWindowsUpdateOperati on() at Amazon.Patch.Baseline.Operations.PatchNow.Implementations.WindowsUpdateOperation.DoBeginProcessing() ,Amazon.Patch.Baseline.Operations.PowerShellCmdlets.InvokePatchBaselineOperation failed to run commands: exit status 4294967295

I am backing up about 45 Windows Server EC2 instances with AWS Backup. One of the AWS Backup jobs, for about 35 of those instances does a VSS snapshot as part of the backup. I get a lot of VSS failure messages. Some of them are VSS timeouts, which I understand is a Windows issue that occurs because of an unconfigurable 10 second max time for the snapshot to complete. Some are related to the AWS VSS provider. In AWS Backup the error is "Windows VSS Backup Job Error encountered, trying for regular backup". The job then completes, but without a VSS snapshot. In SSM, the Run Command error for this task is: Encountered unexpected error. Please see error details below Message : The process cannot access the file 'C:\ProgramFiles\Amazon\AwsVssComponents\vsserr.log' because it is being used by another process. I tried to rename this file (just as a test, to see if it was in use) and says it is in use by the ec2-vss-agent.exe. So I stopped the EC2 VSS Windows service but that did not stop the ec2-vss-agent.exe process and the error remained. I did an 'end task' on the ec2-vss-agent.exe process and I then manually ran the VSS Run Command from SSM. It re-started the process, and it ran for awhile before timing out, which is the other (unrelated?) issue we see too. I can not find anything online about this issue or error and I'm at a loss as far as where to look from here. I need VSS snapshots of these servers. If anyone has any ideas about how to troubleshoot this or what else to look for, please let me know!

I have a private VPC with private subnets a private jumpbox in 1 private subnet and my private RDS aurora MySql serverless instance in another private subnet. I did those commands on my local laptop to try to connect to my RDS via port forwarding: ``` aws ssm start-session --target i-0d5470040e7541ab9 --document-name AWS-StartPortForwardingSession --parameters "portNumber"=["5901"],"localPortNumber"=["9000"] --profile myProfile aws ssm start-session --target i-0d5470040e7541ab9 --document-name AWS-StartPortForwardingSession --parameters "portNumber"=["22"],"localPortNumber"=["9999"] --profile myProfile aws ssm start-session --target i-0d5470040e7541ab9 --document-name AWS-StartPortForwardingSession --parameters "portNumber"=["3306"],"localPortNumber"=["3306"] --profile myProfile ``` The connection to the server hangs. I had this error on my local laptop: ``` Starting session with SessionId: myuser-09e5cd0206cc89542 Port 3306 opened for sessionId myuser-09e5cd0206cc89542. Waiting for connections... Connection accepted for session [myuser-09e5cd0206cc89542] Connection to destination port failed, check SSM Agent logs. ``` and those errors in `/var/log/amazon/ssm/errors.log`: ``` 2021-11-29 00:50:35 ERROR [handleServerConnections @ port_mux.go.278] [ssm-session-worker] [myuser-017cfa9edxxxx] [DataBackend] [pluginName=Port] Unable to dial connection to server: dial tcp :3306: connect: connection refused 2021-11-29 14:13:07 ERROR [transferDataToMgs @ port_mux.go.230] [ssm-session-worker] [myuser-09e5cdxxxxxx] [DataBackend] [pluginName=Port] Unable to read from connection: read unix @->/var/lib/amazon/ssm/session/3366606757_mux.sock: use of closed network connection ``` and I try to connect to RDS like this : [![enter image description here][1]][1] I even tried to put the RDS Endpoint using ssh Tunnel, but it doesn't work: [![enter image description here][2]][2] Are there any additional steps to do on the remote server ec2-instance? It seems the connection is accepted but the connection to the destination port doesn't work. or is there any best other way to connect to private rds in private vpc when de don't have site-to site-vpn or Direct connect ? [1]: https://i.stack.imgur.com/RwiZ8.png [2]: https://i.stack.imgur.com/53GIh.png

The RUN CMD is not working in EU-CENTRAL-1 region for Mac Instance. Session Managers works fine There is already a AWS Enterprise Support which I am receiving any updates. Anyone facing the same issue?

We use AWS SSO to provide permissions for Session Manager access to systems. When trying to use Session Manager in conjunction with SCP one of our users is getting the following error: _$ scp -r -i ~/.ssh/example-key-singapore system1/startsystem.sh legerity@i-06a0c25qb665a08eb.ap-southeast-1:_ _An error occurred (AccessDeniedException) when calling the TerminateSession operation: User: arn:aws:sts::001292317441:a_ _ssumed-role/AWSReservedSSO_Example_739d002d2774bna6/john.doe@companyname.com is not authorized t_ _o perform: ssm:TerminateSession on resource: arn:aws:ssm:ap-southeast-1:001292317441:session/john.doe@companyname.com-08fce585f53bab614 because no identity-based policy allows the ssm:TerminateSession action_ _kex_exchange_identification: Connection closed by remote host_ _Connection closed by UNKNOWN port 65535_ _lost connection_ The session that it says can't be terminated is actually one that is already terminated so I can't figure out how it is erroring or why. I cannot replicate this error when giving myself the same permissions. This same user can access the same system via SSM (SSH equivalent) fine. The permissions assigned to this user are: { "Effect": "Allow", "Action": \[ "ssm:StartSession" ], "Resource": \[ "arn:aws:ec2:**:001292317441:instance/\**", "arn:aws:ec2:**:053586226857:instance/\**", "arn:aws:ssm:**:**:document/AWS-StartSSHSession" ] }, { "Effect": "Allow", "Action": \[ "ssm:TerminateSession", "ssm:ResumeSession" ], "Resource": \[ "arn:aws:ssm:\**:\**:session/${aws:username}-*" ] } This same command using the same permissions works fine for me. The command should work according the config in .ssh which is: host i-**.** mi-**.** ProxyCommand bash -c "aws ssm start-session --target $(echo %h|cut -d'.' -f1) --region $(echo %h|/usr/bin/cut -d'.' -f2) --document-name AWS-StartSSHSession --parameters 'portNumber=%p'" --profile $(echo %h|cut -d '.' -f3) Does anyone have any idea what might be happening? Edited by: jonzen on Oct 29, 2021 3:38 AM Edited by: jonzen on Oct 29, 2021 3:39 AM

Hi all, I need to automatise run command to install and configure CW and get inventory for creating JSON files with the installed Applications. Splunk will pull these files from S3. I have created some scripst for this using the AWS CLI but I'm facing an issue. When I query ssm inventory like bellow, terminated instances are still showing up on the CLI eventhough some have been terminated days ago. Example: aws ssm get-inventory --filters '\[{"Key":"AWS:Tag.Key","Values":\["OS"],"Type":"Equal"},{"Key":"AWS:Tag.Value","Values":\["windows"],"Type":"Equal"}]'| grep -i instanceid | awk '{print $2}' | sed 's/.**"\(.**\)"\[^"]*$/\1/' | wc -l 5 I have only four running instances with those particular tags. Even ec2 describe says that it has been terminated. Bellow the terminated instance id which is still showing up in the inventory list. On the GUI I cannot see the terminated instance. aws ec2 describe-instances --instance-ids i-06122dfe502f15968 { "Reservations": \[ { "Instances": \[ { "Monitoring": { "State": "disabled" }, "StateReason": { "Message": "Client.UserInitiatedShutdown: User initiated shutdown", "Code": "Client.UserInitiatedShutdown" }, "PublicDnsName": "", "Platform": "windows", "State": { "Code": 48, "Name": "terminated" ... Can you guys help me with this. How can I remove them or when will AWS remove? Thanks Norbert Edited by: NorbertS on Jun 17, 2021 11:51 AM

Hello, all! First post, but long time reader. We're trying to enable SSH tunneling using Session Manager. The bastion host is a STIGed Windows 2019 box. We're following these instructions: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager.html When we try and start the SSM agent on the bastion host, we get the following error: ERROR Agent failed to assume any identity ERROR failed to find identity, retrying: failed to find agent identity ERROR Failed to start agent. failed to get identity: failed to find agent identity My assumption is that this is a role/policy issue. I've revisited the instructions (https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-instance-profile.html), recreated the role, and reattached the policy to the EC2 instance. Am I misreading the error and this isn't role policy? Is it a role/policy related to something other than the EC2? Any suggestions gratefully accepted!

In an AWS test environment, our testers frequently need to change the date/time on instances as part of their test suites. Unfortunately, this appears to break Session Manager authentication. The SSM agent generates errors such as: 2021-05-27 18:58:54 ERROR {HandleAwsError @ awserr.go.49} {ssm-agent-worker} {HealthCheck} error when calling AWS APIs. error details - InvalidSignatureException: Signature not yet current: 20210527T175854Z is still later than 20210430T180355Z (20210430T175855Z + 5 min.) status code: 400, request id: caaa36f2-c644-4c91-b461-0dbd8a51774e In the above case, the (Windows 10) instance's clock was set to a future date. Is there any configuration option that would allow us to bypass date-based checks during authentication? Presumably this is an SSL certificate issue. I note from the AWS SDK that certain authentication options allow for a maximum clock drift of five minutes. Does this apply here? In short: is there any way to keep Session Manager available despite changes to the system clock?

Customer is wondering how to patch their servers across their AWS organization. They saw the following blog that explains how to do this with security hub, but they were wondering if there is a simpler way to define patch groups across accounts and regions? https://aws.amazon.com/blogs/mt/multi-account-patch-compliance-with-patch-manager-and-security-hub/

Hi, I recently followed all the guidance to enable ECS ExecuteCommand access for my containers (excellent feature). I followed https://aws.amazon.com/blogs/containers/new-using-amazon-ecs-exec-access-your-containers-fargate-ec2/, deployed the new infrastructure, and I was able to connect to my ECS Fargate task. Success! Very exciting. I go back today to try to troubleshoot a problem in our beta environment, and I keep getting... "An error occurred (InvalidParameterException) when calling the ExecuteCommand operation: The execute command failed because execute command was not enabled when the task was run or the execute command agent isn’t running. Wait and try again or run a new task with execute command enabled and try again." My IAM permissions haven't changed and the cluster/service/task configuration hasn't changed. I double-checked by re-running Terraform to ensure the plans were the same (they are - no diff between what's in the account and what's in the templates). I spun up a new task and described the state of the task: $ aws --region us-west-2 --profile <redacted> ecs describe-tasks --cluster <redacted> --tasks 1fe5bd64db4d428794fa0b956c1efda6 | jq '.tasks\[0] | {"managedAgents": .containers\[0].managedAgents, "enableExecuteCommand": .enableExecuteCommand}' { "managedAgents": \[ { "lastStartedAt": "2021-04-07T09:10:01.885000-04:00", "name": "ExecuteCommandAgent", "lastStatus": "RUNNING" } ], "enableExecuteCommand": true } I then tried to connect: aws --region us-west-2 --profile <redacted> ecs execute-command --cluster <redacted> \ --task 1fe5bd64db4d428794fa0b956c1efda6 \ --container <redacted> \ --interactive \ --command "/bin/bash" Got the error I pasted above. Then I checked the status again. $ aws --region us-west-2 --profile <redacted> ecs describe-tasks --cluster <redacted> --tasks 1fe5bd64db4d428794fa0b956c1efda6 | jq '.tasks\[0] | {"managedAgents": .containers\[0].managedAgents, "enableExecuteCommand": .enableExecuteCommand}' { "managedAgents": \[ { "name": "ExecuteCommandAgent", "lastStatus": "STOPPED" } ], "enableExecuteCommand": true } There's ample RAM available. I'm not sure why it isn't working after nothing has changed. I don't have any additional tools to troubleshoot this. Worse, it's also happening in our production environment too. Any suggestions?

Hi all, I have 6 hosts in a patch group that I need to patch in a certain sequence, what I would like to do is, e.g. , first do instance1, then 2, then 3, then 6 then 5 then 4, is this possible and if so, how must it be configured?

I am attempting to upgrade a Windows Server 2012R2 EC2 Instance to Windows Server 2019 using the AWS Systems Manager automation: AWSEC2-CloneInstanceAndUpgradeWindows I have followed the instructions outlined here to perform the upgrade: https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/automated-upgrades.html I have updated the SSM agent on the server that I am upgrading to the newest version. I have also updated the EC2Config service to the latest version and rebooted the server. I have created a new IAM role to use for the upgrade and I have attached the AmazonSSMManagedInstanceCore policy to the role. I have set this role as the IAM role on the instance that I am attempting to upgrade. I am running the upgrade with a subnet that is in the same default VPC as the instance that I am trying to upgrade. The subnet is a different previously unused default subnet within that default VPC as the instructions suggest. The subnet does have Auto-assign public IPv4 address set to yes. The automation fails on step 7: Automation step7: runUpgradeFrom2012R2Or2016 Failed to run automation with executionId: theId Failed : {Status=\[Failed], Output=\[No output available yet because the step is not successfully executed, No output available yet because the step is not successfully executed], ExecutionId=theId} I am having trouble narrowing down the problem or finding a cause for the failure based on the error above. Any pointers or help will be greatly appreciated! Thank you, Brian

I've down this before for other raspberry pi's without a problem, but know I get this error: ERROR Registration failed due to error registering the instance with AWS SSM. MissingEndpoint: 'Endpoint' configuration is required for this service Using this as the guide: https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-install-managed-linux.html Any ideas?

I just created a EC2 inside a fully private VPC (without IGW, without NAT, no internet access at all), and follow the instructuion to create endpoints. As a test result, what I found is I can't connect to this EC2 using session manager via browser Here's the link to the instructions I've followed, https://aws.amazon.com/premiumsupport/knowledge-center/ec2-systems-manager-vpc-endpoints/ To verify and compare, then I just created 2 cloudformationized environment to make sure I am using same EC2 AMI, Same IAM Profile, Same endpoints and endpoint SG policy (allow all traffics), same VPC enableDNS settings, same ACL (allow all traffic). The only difference is one of the EC2 have outbound internet access. The test result is: **Only EC2 have outbount internet access can be connected using session manager via browser.** Is that correct ?

I'm using the aws:downloadContent Command Document plugin to download a file from an S3 bucket. It works fine when the S3 bucket is in the same account as the instance, but when the bucket is in a different account, the step fails with "Access Denied". The bucket is public, and I've given the instance full S3 permissions to access it. Is this a build-in restriction of SSM?

Hi, is there a way to find out when was the SSM agent latest version was made available for update via AWS SDK?

Hi, I'm trying to run a script every time a Session Manager session is terminated after a user exits the terminal. I have a script that connects to Session Manager using AWS CLI. I was trying to use EventBridge to create a rule based on the TerminateSession API call but apparently the endpoint is never called unless a specific TerminateSession command is triggered either through the console, API or CLI. Or at least I haven't seen it in the CloudTrail events history anyway. Is there any other way to react to terminals being closed? Thanks.

Hello, Can someone please show some light on me with the difference between State Manager vs Maintenance Window ? As I see both oh them require an SSM Document, Targets and a Schedule; so why would you go with one or the other? Thank you.