跳至內容

為什麼統一 CloudWatch 代理程式沒有將指標或日誌事件推送至 CloudWatch?

4 分的閱讀內容
0

我已在 Amazon Elastic Compute Cloud (Amazon EC2) 執行個體上設定統一 Amazon CloudWatch 代理程式,將指標和日誌發佈至 Amazon CloudWatch。不過,我在 CloudWatch 主控台中看不到指標或日誌事件。

解決方法

查看代理程式日誌以找出問題

開始疑難排解前,請查看代理程式日誌檔案,以協助找出統一 CloudWatch 代理程式套件的問題。

如果代理程式無法連線至必要的 AWS 服務端點或 Amazon Virtual Private Cloud (Amazon VPC) 端點,您可能會收到「Agent Log Error: No Connectivity to Endpoint」錯誤。

如果您不具備呼叫 CloudWatch API 所需的必要權限,便會收到「Agent Log Error: Insufficient Permissions」錯誤。

「No connectivity」錯誤範例:

2021-08-30T04:07:46Z E! cloudwatch: code: RequestError, message: send request failed, original error: Post "https://monitoring.us-east-1.amazonaws.com/": dial tcp 172.31.11.121:443: i/o timeout
2021-08-30T04:07:46Z W! 210 retries, going to sleep 1m0s before retrying.
2021-08-30T04:07:46Z E! cloudwatch: code: RequestError, message: send request failed, original error: Post "https://monitoring.us-east-1.amazonaws.com/": dial tcp 172.31.11.121:443: i/o timeout
2021-08-30T04:07:46Z W! 211 retries, going to sleep 1m0s before retrying.

「Insufficient permissions」錯誤範例:

2021-08-30T02:15:45Z E! cloudwatch: code: AccessDenied, message: User: arn:aws:sts::123456789012:assumed-role/cwagent/i-0744de7c842d2c2ba is not authorized to perform: cloudwatch:PutMetricData, original error:
2021-08-30T02:15:45Z W! 1 retries, going to sleep 400ms before retrying.
2021-08-30T02:15:46Z E! WriteToCloudWatch failure, err:  AccessDenied: User: arn:aws:sts::123456789012:assumed-role/cwagent/i-0744de7c842d2c2ba is not authorized to perform: cloudwatch:PutMetricData
    status code: 403, request id: f1171fd0-05b6-4f7d-bac2-629c8594c46e

確認可連線至 CloudWatch 端點

如果前往 CloudWatch 的流量會通過 Amazon VPC 端點,而非公共網際網路,請檢查以下參數:

  • 如果您使用私有名稱伺服器,請確認 DNS 解析會傳回正確的回應。
  • 確認 CloudWatch 端點會解析為私有 IP 位址。
  • 確認與 Amazon VPC 端點關聯的安全群組允許來自主機的傳入流量。

接著,請完成以下步驟,確認可連線至 CloudWatch 端點:

  1. 執行以下 telnet 命令,檢查是否可連線至指標端點:

    telnet monitoring.us-east-1.amazonaws.com 443
    Trying 52.46.138.115...
    Connected to monitoring.amazonaws.com.
    Escape character is '^]'.
    ^]
    telnet> quit
    Connection closed.
    
  2. 執行以下 telnet 命令,檢查是否可連線至日誌端點:

    telnet logs.us-east-1.amazonaws.com 443
    Trying 3.236.94.218...
    Connected to logs.us-east-1.amazonaws.com.
    Escape character is '^]'.
    ^]
    telnet> quit
    Connection closed
    
  3. 執行以下 dig 命令,確認 Amazon VPC 端點會解析為私有 IP 位址:

    dig monitoring.us-east-1.amazonaws.com
    +short172.31.11.121
    172.31.0.13
    

檢閱統一 CloudWatch 代理程式組態

CloudWatch 代理程式組態檔案會指定代理程式要發佈至 CloudWatch 的指標和日誌。檢閱代理程式組態檔案,確認其中包含您要發佈的日誌和指標。

確認主機具備發佈指標和日誌的權限

注意: 如果您在執行 AWS Command Line Interface (AWS CLI) 命令時收到錯誤訊息,請參閱對 AWS CLI 錯誤進行疑難排解。此外,請確定您使用的是最新的 AWS CLI 版本。

AWS 受管政策 CloudWatchAgentServerPolicy 和 CloudWatchAgentAdminPolicy 定義部署統一 CloudWatch 代理程式所需的權限。以這些政策為參考,確認主機具備正確的權限。

以下 AWS CLI config 命令顯示 Amazon EC2 執行個體缺少 AWS Identity and Access Management (IAM) 角色:

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:CWT-Web-Server -s
****** processing amazon-cloudwatch-agent ******
/opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source ssm:CWT-Web-Server --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default
Region: us-east-1
credsConfig: map[]
Error in retrieving parameter store content: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors
Fail to fetch/remove json config: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Fail to fetch the config!

以下 AWS CLI config 命令顯示 EC2 執行個體使用了不正確的 IAM 角色:

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:CWT-Web-Server -s
****** processing amazon-cloudwatch-agent ******
/opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source ssm:CWT-Web-Server --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default
Region: us-east-1
credsConfig: map[]
Error in retrieving parameter store content: AccessDeniedException: User: arn:aws:sts::123456789012:assumed-role/cwagent/i-0744de7c842d2c2ba is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:us-east-1:123456789012:parameter/CWT-Web-Server
    status code: 400, request id: b85b0a7a-0fb1-47b4-924f-be8cf43a3b4d
Fail to fetch/remove json config: AccessDeniedException: User: arn:aws:sts::123456789012:assumed-role/cwagent/i-0744de7c842d2c2ba is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:us-east-1:123456789012:parameter/CWT-Web-Server
    status code: 400, request id: b85b0a7a-0fb1-47b4-924f-be8cf43a3b4d

Fail to fetch the config!

以下 get-caller-identity 命令會傳回與執行個體關聯的 IAM 使用者或角色:

aws sts get-caller-identity
{
    "UserId": "AROA123456789012ABCDE:i-0744de7c842d2c2ba",
    "Account": "123456789012",
    "Arn": "arn:aws:sts::123456789012:assumed-role/CloudWatchAgentServerRole/i-0744de7c842d2c2ba"
}

確認代理程式可正常啟動

若要啟動代理程式,請執行以下有效的啟動命令。

Linux 命令

如果組態檔案儲存在執行個體本機,請執行以下命令:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:configuration-file-path

如果組態檔案儲存在 AWS Systems Manager 的一項功能參數存放區中,請執行以下命令:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:configuration-parameter-store-name

Windows 命令

如果組態檔案儲存在執行個體本機,請執行以下命令:

& "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -s -c file:"C:\Program Files\Amazon\AmazonCloudWatchAgent\config.json

如果組態檔案儲存在參數存放區中,請執行以下命令:

& "C:\Program Files\Amazon\AmazonCloudWatchAgent\amazon-cloudwatch-agent-ctl.ps1" -a fetch-config -m ec2 -s -c ssm:configuration-parameter-store-name

重要: 請勿從 Windows 控制台啟動代理程式。

確認代理程式正在執行

代理程式必須處於作用中狀態,才能發佈指標和日誌。若要確認 CloudWatch 代理程式處於作用中狀態,請執行以下命令:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
{
    "status": "running",
    "starttime": "2021-08-30T02:13:44+00:00",
    "configstatus": "configured",
    "cwoc_status": "stopped",
    "cwoc_starttime": "",
    "cwoc_configstatus": "not configured",
    "version": "1.247349.0b251399"
}

更新代理程式組態後重新啟動代理程式

代理程式不會自動套用組態檔案的變更。如果您更新代理程式組態以加入新的或不同的指標和日誌,請執行以下命令重新啟動代理程式:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a stop
****** processing cwagent-otel-collector ******
cwagent-otel-collector has already been stopped

****** processing amazon-cloudwatch-agent ******
Redirecting to /bin/systemctl stop amazon-cloudwatch-agent.service

執行以下命令,提取更新後的組態檔案並重新啟動代理程式:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
****** processing amazon-cloudwatch-agent ******
/opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source file:config.json --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default
Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json.tmp
Start configuration validation...
/opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default
2021/08/31 02:45:37 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json.tmp ...
Valid Json input schema.
I! Detecting run_as_user...
Configuration validation first phase succeeded
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml
Configuration validation second phase succeeded
Configuration validation succeeded
amazon-cloudwatch-agent has already been stopped
Redirecting to /bin/systemctl restart amazon-cloudwatch-agent.service

注意: 將 config.json 替換為您的組態檔案路徑。

執行以下命令,確認代理程式正在執行:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
{
  "status": "running",
  "starttime": "2021-08-31T02:45:37+0000",
  "configstatus": "configured",
  "cwoc_status": "stopped",
  "cwoc_starttime": "",
  "cwoc_configstatus": "not configured",
  "version": "1.247349.0b251399"
}

在輸出中,確認狀態顯示為執行中。當狀態為執行中時,即表示 CloudWatch 代理程式處於執行狀態。

相關資訊

如何安裝及設定統一 CloudWatch 代理程式,將 Amazon EC2 執行個體的指標和日誌傳送至 CloudWatch?

AWS 官方已更新 7 個月前