Skip to content

How to use PutLogEvent in a step function?

0

We have a step in our (Express) step function where we want to be able to log a specifically formatted message to Cloudwatch. For security reasons we cannot log execution data so we cannot simply extract from there. So the idea would be to add a PutLogEvents step into the step function where we can insert the message. BUT: the PutLogEvents API requires that the request payload contain a timestamp "expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC." ref: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_InputLogEvent.html There no way in Amazon States Language to generate such a timestamp, and the time started in the ExecutionData of the Contextt object is in ISO fomat. Neither is there a way to convert ISO to milliseconds in ASL.

So how then is it possible to use the PutLogEvent within a Step Function? Its listed as an available option in the console editor so presumably it should be usable. Please provide an example.

StepFunction Editor

2 Answers
4
Accepted Answer

Hey,

As you've identified, PutLogEvent requires timestamp in Unix epoch and there is no intrinsic function in ASL to do this so you'll have to handle it outside ASL.

You can introduce an "Invoke Lambda" step before the "PutLogEvent" to get you the timestamp in the Unix epoch format.

  • I was able to do this by creating this Python Lambda function:
import json
import datetime
def lambda_handler(event, context):    
    epoch_timestamp = iso_to_epoch(event['currentTime'])    
    return {
        'statusCode': 200,
        'epoch':epoch_timestamp
    }
def iso_to_epoch(iso_time):    
    # Parse the ISO 8601 time string into a datetime object
    dt = datetime.datetime.fromisoformat(iso_time)    
    # Convert the datetime object to an epoch timestamp
    epoch = int(dt.timestamp())    
    return epoch
  • And invoking it with this State Machine:
{
  "StartAt": "Lambda Invoke",
  "States": {
    "Lambda Invoke": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "OutputPath": "$.Payload",
      "Parameters": {
        "Payload.$": "$",
        "FunctionName": "arn:aws:lambda:ap-southeast-1:012345678901:function:iso-to-unix-converter:$LATEST"
      },
      "Next": "PutLogEvents"
    },
    "PutLogEvents": {
      "Type": "Task",
      "Parameters": {
        "LogEvents": [
          {
            "Message": "MyData",
            "Timestamp.$": "$.epoch"
          }
        ],
        "LogGroupName": "MyData",
        "LogStreamName": "MyData"
      },
      "Resource": "arn:aws:states:::aws-sdk:cloudwatchlogs:putLogEvents",
      "End": true
    }
  }
}
  • executed with this input :
{
  "currentTime": "2024-01-11T14:11:03+00:00"
}

I could see the PutLogEvent call being made.

Hope this helps.

AWS
SUPPORT ENGINEER
answered 2 years ago
EXPERT
reviewed 2 years ago
AWS
EXPERT
reviewed 2 years ago
  • Thanks for your comment, I can see that this would work but it seems like a gap in the step function capability set. I was hoping there was something that I was missing. If I have to setup a lambda just to get a timestamp then I may as well setup that same lambda to do the logging directly - while eliminating the gap in duration between lambda creating the timestamp and next step using it. Case in point, if viewing logs for the step function in timestamp order (cloudwatch default) , the 'logged message' will actually occur before the 'PutLogEvents' step starts. its a trivial difference but can be meaningful.

0

I was able to use this JSONata as my PutlogEvents argument:

{
  "LogEvents": [
    {
      "Message": "MyData",
      "Timestamp": "{% $millis() %}"
    }
  ],
  "LogGroupName": "MyData",
  "LogStreamName": "MyData"
}
answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.