- Newest
- Most votes
- Most comments
Update 1
I found the AWS SSM agent in those instances that I can't add to a new application use up my CPU. The CPU utilization is nearly 100%. The problem that I can't implement the application insights is the bug caused by the AWS SSM agent because the SSM agent is a Prerequisites of the application insights. It seems the SSM agent in the instance repeat the restarting process because my terminal shows several warning SSM agent is starting and the restarting use up the CPU.
I check the error logs in my instance, and I foun the main error is
2021-02-26 15:51:35 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [d2c14f85-0078-4f80-8990-1df22d983f1e] file watcher error: CloseHandle: The handle is invalid.
2021-02-26 15:51:36 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [057ce019-2095-456a-a874-1be23f075ed9] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:29:23 ERROR [IsProcessExists @ process.go.86] [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [7b5a3ded-1341-4707-8436-036beeec501f] encountered error when finding process: open process error: %!(EXTRA syscall.Errno=The parameter is incorrect.)
2021-02-26 16:29:23 ERROR [GetDocumentState @ docmanager.go.121] [MessagingDeliveryService] [EngineProcessor] encountered error with message unexpected end of JSON input while reading Interim state of command from file - 42cb437f-1a9f-490a-9b3c-cce608bac353
2021-02-26 16:29:23 ERROR [IsProcessExists @ process.go.86] [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [057ce019-2095-456a-a874-1be23f075ed9] encountered error when finding process: open process error: %!(EXTRA syscall.Errno=The parameter is incorrect.)
2021-02-26 16:29:23 ERROR [IsProcessExists @ process.go.86] [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [86713365-8052-4891-920a-fba7c98d4f73] encountered error when finding process: open process error: %!(EXTRA syscall.Errno=The parameter is incorrect.)
2021-02-26 16:29:23 ERROR [IsProcessExists @ process.go.86] [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [cdd9b8a7-4a66-4e0b-8b4a-05963f6eabef] encountered error when finding process: open process error: %!(EXTRA syscall.Errno=The parameter is incorrect.)
2021-02-26 16:29:23 ERROR [IsProcessExists @ process.go.86] [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [d2c14f85-0078-4f80-8990-1df22d983f1e] encountered error when finding process: open process error: %!(EXTRA syscall.Errno=The parameter is incorrect.)
2021-02-26 16:29:37 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [8587c8ee-1aa5-411a-838f-ff4ec3f1218d] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:29:43 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [f2a63484-2081-4225-8b14-a60f4ddb9a16] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:29:48 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [31cc4d1a-bfb1-4796-9868-1f0ea47268a0] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:29:51 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [099010e5-5d71-42a1-b060-b05bea906b31] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:29:51 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [099010e5-5d71-42a1-b060-b05bea906b31] file watcher error: CloseHandle: The handle is invalid.
2021-02-26 16:30:05 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [770f959c-1728-4a27-aff0-39e1b8615103] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:30:18 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [583d06d6-a6fe-48f9-ad35-9fcc85dcb9c4] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:30:23 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [a83c88e0-c649-4a2f-8817-7d7f01424784] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:30:35 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [e5d6741a-620d-4511-81bf-b9a6cb25086e] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:30:36 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [814b5855-de13-4cb7-ae32-cc764224392a] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:30:58 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [6f933b47-bac8-490c-819e-c285a0b9b848] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:03 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [0d9987a0-a371-4e68-8a40-f61b8407e0d1] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:03 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [16217593-ddf4-4a05-8f1c-d9ef8986b7f3] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:03 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [f17261a5-1fe5-466a-a4c2-503bd2775b7e] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:03 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [f17261a5-1fe5-466a-a4c2-503bd2775b7e] file watcher error: CloseHandle: The handle is invalid.
2021-02-26 16:31:04 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [d6187eb2-043b-4b74-9ce8-0127dd25cbdc] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:27 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [cd04a726-6212-424c-ab30-0f6cee6f9699] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:29 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [8ee12023-3377-4bf4-aa16-ab2eb0d9b905] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:46 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [2acc077e-6636-4fb2-8633-f04c58924e29] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:49 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [4fdd806a-4984-4d18-82ec-707f32e8631e] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:31:54 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [03666b9b-aa35-4734-8a70-65929b9cb369] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:32:01 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [ed5152b4-6615-47bf-bdb9-2be157908cb9] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:32:01 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [0bf201dc-1fa9-4c14-a0b0-d9eeedc858ff] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:32:01 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [0bf201dc-1fa9-4c14-a0b0-d9eeedc858ff] file watcher error: CloseHandle: The handle is invalid.
2021-02-26 16:32:09 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [a1d115fa-8794-403a-ae19-329a93a1ed15] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:32:17 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [be7d9932-cd57-41f1-8311-472329e73546] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:32:17 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [9b75b6e3-0ddf-4938-874c-e05ccd31de00] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:32:34 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [f1ad7167-ec59-4fd7-a3a3-0b739dd2b62d] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:07 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [c5977b62-88ee-48b1-a91d-67f85ff1e26b] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:11 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [a7afa3e9-6511-4aaa-a5be-5aee66272164] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:11 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [a7afa3e9-6511-4aaa-a5be-5aee66272164] file watcher error: CloseHandle: The handle is invalid.
2021-02-26 16:33:17 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [6afb9237-c0ff-4ad4-91b7-1bdb8d3d1362] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:20 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [e7cbfc98-e25d-468f-bd58-a05b1ae83a14] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:23 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [7e47d6fd-b5d3-43b3-9b93-ffd818f8930a] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:28 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [ef7a2f46-2654-4f9f-b8c2-b9a6b65ab791] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:38 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [ce478058-ec9b-4772-9ea7-92dcc87d3ad4] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:45 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [72b7fbfe-a853-4fc7-abe2-3f2cef7c069b] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:33:53 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [96526ae3-9b39-4862-a801-6648ff917457] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:07 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [22d750bd-7e21-4f88-ac19-72be6e7ef65b] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:09 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [7b4ba7f7-34ea-4af9-abb8-8a891c86274c] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:27 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [02f0409b-dda0-43ae-8e5a-21250be97b50] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:27 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [02f0409b-dda0-43ae-8e5a-21250be97b50] file watcher error: CloseHandle: The handle is invalid.
2021-02-26 16:34:28 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [61d8be89-d3d0-41fd-ae37-120683526f5c] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:32 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [86877979-1d2d-4e30-99a7-d6fd90dd103e] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:59 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [36736af2-36c4-4ddf-8cdc-c9698309eb82] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:35:00 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [e821f103-4995-4b6b-820e-b52041a0084c] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:48 ERROR [watch @ filechannel.go.300] [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1d40238c-3aaa-4a54-aa73-fd3b36cdf6dc] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:34:48 ERROR [watch @ filechannel.go.300] [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1d40238c-3aaa-4a54-aa73-fd3b36cdf6dc] file watcher error: CloseHandle: The handle is invalid.
2021-02-26 16:35:07 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [355577bd-a6f6-405f-9330-f7aca22aa076] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:35:17 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [5bb31d3b-581b-440a-8192-6dc41899dc99] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:35:21 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [e4cd4a75-eead-455d-9cb3-d3a5c1b9ae13] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:35:49 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [c00febdb-3aa1-4077-a6ee-858557c39086] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:36:08 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [a4482fdb-0427-441a-ae01-aafebef48120] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:36:14 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [4952fe07-d4a3-42a0-8cc5-a71849735c90] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:36:29 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [77cb368e-428a-4d89-b3db-4c2bd6e870e7] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:36:39 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [61da6876-5187-4898-9d70-da5606b1cebe] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:36:42 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [990da9cb-6302-460b-8eda-37cba3a248fe] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:36:45 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [0eeae86c-72f4-4932-b08d-dc3e1495fa26] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:36:50 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [32ef26c7-2983-46ce-9a9b-795ad8191997] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:37:16 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [0951b433-e18b-45cc-94f5-261946486444] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:37:31 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [61bd5b4f-688e-4684-ad65-cb5b0938b647] file watcher error: CancelIo: The handle is invalid.
2021-02-26 16:37:33 ERROR [watch @ filechannel.go.300] [ssm-document-worker] [d3f43705-a64b-4004-a21c-23d712991a6c] file watcher error: CancelIo: The handle is invalid.
Hi everyone,
I finally figure out why the CPU utilization goes to 100%, there are two reasons.
- The version of the SSM agent is 2.X in my instances, which may cause a conflict between the application insights service and SSM service. The solution is to manually update the version of the SSM agent. The process does not require a server restart.
- I didn't attach the role AmazonSSMManagedInstanceCore to the instance I use the application insight service. To be mentioned, the service requires two roles, AmazonSSMManagedInstanceCore and CloudWatchAgentServerPolicy. The system runs abnormally if we don't attach the two roles restrictedly (Without some permissions).
After that, everything runs very well. hope it help you when you meet the same error.
Relevant content
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 8 months ago
- AWS OFFICIALUpdated 5 months ago