当我尝试将慢日志发布到 CloudWatch Logs 时，如何解决 CloudFormation 中收到的错误？

简短描述

要解决此错误，请在日志组级别使用单独的策略，以允许 Amazon Elasticsearch Service (Amazon ES) 将日志推送到 CloudWatch Logs。然后，使用 AWS::Elasticsearch::Domain 资源中的 AccessPolicies 为 Amazon ES 域设置权限。

以下步骤将向您介绍如何通过 Python 3.6 中创建的 AWS Lambda 支持的自定义资源，使用 CloudFormation 将慢日志发布到 CloudWatch 中。自定义资源将触发 Lambda 函数，该函数将触发 PutResourcePolicy API 以发布慢日志。

**注意：**当 CloudFormation 启用向 CloudWatch 发布日志时，AWS::Logs::LogGroup 资源没有用于分配资源访问策略的属性。为 CloudWatch Logs 日志组指定的资源访问策略应为 Amazon ES 授予足够的权限来发布日志流。您无法直接使用 CloudFormation 资源创建访问策略权限。这是因为 AWS::Logs::LogGroup 资源的 PutResourcePolicy API 调用不受 CloudFormation 支持。

解决方法

注意：如果您在运行 AWS Command Line Interface (AWS CLI) 命令时收到错误讯息，请确保您运行的是最新版本的 AWS CLI。

以下 CloudFormation 模板使用自定义资源以获取日志组的名称。然后，模板应用允许 Amazon ES 服务对日志组进行 API 调用的策略。

1.    创建名为 ESlogsPermission.yaml 的 CloudFormation 模板：

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

AWSTemplateFormatVersion: 2010-09-09
Description: AWS cloudFormation template to publish slow logs to Amazon CloudWatch Logs.
Parameters:
  LogGroupName:
    Type: String
    Description: Please don't change the log group name while updating
  ESDomainName:
    Description: A name for the Amazon Elastic Search domain
    Type: String
  LambdaFunctionName:
    Description: Lambda Function Name
    Type: String
Resources:
  AwsLogGroup:
    Type: 'AWS::Logs::LogGroup'
    Properties:
      LogGroupName: !Ref LogGroupName
  LambdaLogGroup:
    Type: 'AWS::Logs::LogGroup'
    Properties:
      LogGroupName: !Sub '/aws/lambda/${LambdaFunctionName}'
  LambdaExecutionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      Policies:
        - PolicyName: root1
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: !Sub >-
                  arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${LambdaFunctionName}:log-stream:*
        - PolicyName: root2
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                Resource:
                  - !Sub >-
                    arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${LambdaFunctionName}
                  - !Sub >-
                    arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/${LogGroupName}
              - Effect: Allow
                Action:
                  - 'logs:PutResourcePolicy'
                  - 'logs:DeleteResourcePolicy'
                Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*'
  logGroupPolicyFunction:
    DependsOn: LambdaLogGroup
    Type: 'AWS::Lambda::Function'
    Properties:
      FunctionName: !Ref LambdaFunctionName
      Code:
        ZipFile: >
          import urllib3

          import json

          import boto3

          http = urllib3.PoolManager()

          SUCCESS = "SUCCESS"

          FAILED = "FAILED"

          def send(event, context, responseStatus, responseData,
          physicalResourceId=None, noEcho=False):
              responseUrl = event['ResponseURL']
              print(responseUrl)
              responseBody = {}
              responseBody['Status'] = responseStatus
              responseBody['Reason'] = 'See the details in CloudWatch Log Stream: ' + context.log_stream_name
              responseBody['PhysicalResourceId'] = physicalResourceId or context.log_stream_name
              responseBody['StackId'] = event['StackId']
              responseBody['RequestId'] = event['RequestId']
              responseBody['LogicalResourceId'] = event['LogicalResourceId']
              responseBody['NoEcho'] = noEcho
              responseBody['Data'] = responseData
              json_responseBody = json.dumps(responseBody)
              print("Response body:\n" + json_responseBody)
              headers = {
                  'content-type' : '',
                  'content-length' : str(len(json_responseBody))
              }
              try:
                  response = http.request('PUT',responseUrl,body=json_responseBody.encode('utf-8'),headers=headers)
                  print("Status code: " + response.reason)
              except Exception as e:
                  print("send(..) failed executing requests.put(..): " + str(e))
          def handler(event, context):
              logsgroup_policy_name=event['ResourceProperties']['CWLOGS_NAME']
              cw_log_group_arn=event['ResourceProperties']['CWLOG_ARN']
              cwlogs = boto3.client('logs')
              loggroup_policy={
                  "Version": "2012-10-17",
                  "Statement": [{
                  "Sid": "",
                  "Effect": "Allow",
                  "Principal": { "Service": "es.amazonaws.com"},
                  "Action":[
                  "logs:PutLogEvents",
                  " logs:PutLogEventsBatch",
                  "logs:CreateLogStream"
                      ],
                  'Resource': f'{cw_log_group_arn}'
                  }]
                  }
              loggroup_policy = json.dumps(loggroup_policy)
              if(event['RequestType'] == 'Delete'):
                  print("Request Type:",event['RequestType'])
                  cwlogs.delete_resource_policy(
                  policyName=logsgroup_policy_name
                 )
                  responseData={}
                  send(event, context, SUCCESS, responseData)
              elif(event['RequestType'] == 'Create'):
                  try:
                      cwlogs.put_resource_policy(
                      policyName = logsgroup_policy_name,
                      policyDocument = loggroup_policy
                       )
                      responseData={}
                      print("Sending response to custom resource")
                      send(event, context, SUCCESS, responseData)
                  except Exception as  e:
                      print('Failed to process:', e)
                      send(event, context, FAILED, responseData)
              elif(event['RequestType'] == 'Update'):
                  try:
                      responseData={}
                      print("Update is not supported on this resource")
                      send(event, context, SUCCESS, responseData)
                  except Exception as  e:
                      print('Failed to process:', e)
                      send(event, context, FAILED, responseData)
      Handler: index.handler
      Role: !GetAtt 
        - LambdaExecutionRole
        - Arn
      Runtime: python3.6
  logGroupPolicycustomresource:
    Type: 'Custom::LogGroupPolicy'
    Properties:
      ServiceToken: !GetAtt 
        - logGroupPolicyFunction
        - Arn
      CWLOGS_NAME: !Ref LogGroupName
      CWLOG_ARN: !GetAtt 
        - AwsLogGroup
        - Arn
  ElasticsearchDomain:
    Type: 'AWS::Elasticsearch::Domain'
    DependsOn: logGroupPolicycustomresource
    Properties:
      DomainName: !Ref ESDomainName
      ElasticsearchVersion: '6.2'
      EBSOptions:
        EBSEnabled: true
        VolumeSize: 10
        VolumeType: gp2
      LogPublishingOptions:
        SEARCH_SLOW_LOGS:
          CloudWatchLogsLogGroupArn: !GetAtt 
            - AwsLogGroup
            - Arn
          Enabled: true

2.    要使用 ESlogsPermission.yaml 文件启动 CloudFormation 堆栈，请使用 CloudFormation 控制台或以下 AWS CLI 命令：

aws cloudformation create-stack --stack-name yourStackName --template-body file://yourTemplateName --parameters ParameterKey=LogGroupName,ParameterValue=Your-LogGroup-Name, ParameterKey=ESDomainName,ParameterValue=Your-ES-Name --capabilities CAPABILITY_NAMED_IAM --region yourRegion

**注意：**请将 yourStackName、yourTemplateName、Your-LogGroup-Name、Your-ES-Name 和 yourRegion 替换为您的值。

CloudFormation 模板为您执行以下操作：

1.    创建日志组。

2.    创建 Lambda 函数。Lambda 函数通过使用自定义资源从 CloudFormation 模板的参数部分中获取日志组名称。Lambda 函数为日志组名称调用 PutResourcePolicy API。日志组必须具有允许 Amazon ES 域放置日志的策略。

3.    创建由 Lambda 支持的自定义资源以调用在步骤 2 中创建的 Lambda 函数。自定义资源有助于将 PutResourcePolicy 应用于日志组 Amazon 资源名称 (ARN)，以便 Amazon ES 可以流式传输日志。在模板中，CloudFormation 使用自定义资源通过 LogPublishingOption 创建 Amazon ES 域。

---