Local step-function does not apply success task request until Lambda will not finishes

0

Hi, I am trying to send a task-success request for my 'waitForTaskToken' step, but I see that the local step-machine server is waiting for the lambda to finish, instead of going to the next step how it works now on the AWS. At this moment task-success request from AWS CLI froze.

Local step-machine server does not return a response for 'send-task-success' request when it has been sent and waiting for the lambda to finish.

Step Functions Local Version: 1.8.1 (docker image) Step-function definition:

{
  "Comment": "",
  "StartAt": "waitForTaskToken",
  "States": {
    "waitForTaskToken": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
      "HeartbeatSeconds": 900,
      "TimeoutSeconds": 900,
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:123456789012:function:delay-function",
        "Payload": {
          "Token.$": "$$.Task.Token"
        }
      },
      "End": true
    }
  }
}

lambda 'delay-function' in this case - simple NodeJS function which prints the task token and waits 130 seconds.

As we can see in the logs below:

  1. 2022-02-09 10:16:25.078 - Executed local step-function

  2. 2022-02-09T10:16:25.092Z - Received task token 'f68313c6-89bc-420f-b335-b8369dc5064e' and waiting 130000 ms

  3. 2022-02-09 10:16:50.673 - I sent a task-success request from aws cli 'aws stepfunctions --endpoint-url http://localhost:8083 send-task-success --task-output '{"test": 123}' --task-token f68313c6-89bc-420f-b335-b8369dc5064e' after which this CLI command freezed, the server does not return a response.

  4. 2022-02-09 10:16:50.673: - local step-function server received this request but does not return the result. [ISSUE THERE] does not stop the current step with the result from 'send-task-success' request until Lambda will not finishes.

  5. 2022-02-09 10:18:35.100 - Lambda return data, (Duration: 130006.11 ms).

  6. 2022-02-09 10:18:35.102 - Only now the local step-function server has returned the result for 'send-task-success' request. [ISSUE THERE] for some reason only now server returns a response for 'send-task-success' request instead of returning it at the same time it was sent (as it works now on AWS)

Logs:

stepfunctions  | 2022-02-09 10:16:25.078: [200] StartExecution <= {"sdkResponseMetadata":null,"sdkHttpMetadata":null,"executionArn":"arn:aws:states:us-east-1:123456789012:execution:HelloWorld:e2e402fa-6bbd-4b56-87f1-4cf1a8fc1273","startDate":1644401785077}
stepfunctions  | 2022-02-09 10:16:25.080: arn:aws:states:us-east-1:123456789012:execution:HelloWorld:e2e402fa-6bbd-4b56-87f1-4cf1a8fc1273 : {"Type":"TaskScheduled","PreviousEventId":2,"TaskScheduledEventDetails":{"ResourceType":"lambda","Resource":"invoke.waitForTaskToken","Region":"us-east-1","Parameters":"{\"FunctionName\":\"arn:aws:lambda:us-east-1:123456789012:function:function\",\"Payload\":{\"Token\":\"f68313c6-89bc-420f-b335-b8369dc5064e\"}}","TimeoutInSeconds":900,"HeartbeatInSeconds":900}}
stepfunctions  | 2022-02-09 10:16:25.081: arn:aws:states:us-east-1:123456789012:execution:HelloWorld:e2e402fa-6bbd-4b56-87f1-4cf1a8fc1273 : {"Type":"TaskStarted","PreviousEventId":3,"TaskStartedEventDetails":{"ResourceType":"lambda","Resource":"invoke.waitForTaskToken"}}
lambda         | START RequestId: b7e0b919-6519-4f84-bb36-70c540c8d550 Version: $LATEST
lambda         | 2022-02-09T10:16:25.092Z       b7e0b919-6519-4f84-bb36-70c540c8d550    INFO    >>> f68313c6-89bc-420f-b335-b8369dc5064e
stepfunctions  | 2022-02-09 10:16:50.673: SendTaskSuccess => {"requestClientOptions":{"readLimit":131073,"skipAppendUriPath":false},"requestMetricCollector":null,"customRequestHeaders":null,"customQueryParameters":null,"cloneSource":null,"sdkRequestTimeout":null,"sdkClientExecutionTimeout":null,"taskToken":"f68313c6-89bc-420f-b335-b8369dc5064e","output":"{\"test\": 123}","generalProgressListener":{"syncCallSafe":true},"readLimit":131073,"cloneRoot":null,"requestCredentials":null,"requestCredentialsProvider":null}
stepfunctions  | 2022-02-09 10:17:51.380: SendTaskSuccess => {"requestClientOptions":{"readLimit":131073,"skipAppendUriPath":false},"requestMetricCollector":null,"customRequestHeaders":null,"customQueryParameters":null,"cloneSource":null,"sdkRequestTimeout":null,"sdkClientExecutionTimeout":null,"taskToken":"f68313c6-89bc-420f-b335-b8369dc5064e","output":"{\"test\": 123}","generalProgressListener":{"syncCallSafe":true},"readLimit":131073,"cloneRoot":null,"requestCredentials":null,"requestCredentialsProvider":null}
} "Token": "f68313c6-89bc-420f-b335-b8369dc5064e"7e0b919-6519-4f84-bb36-70c540c8d550    INFO    EVENT: 
lambda         | END RequestId: b7e0b919-6519-4f84-bb36-70c540c8d550
lambda         | REPORT RequestId: b7e0b919-6519-4f84-bb36-70c540c8d550 Duration: 130006.11 ms  Billed Duration: 130007 ms                                                                                    Memory Size: 3008 MB     Max Memory Used: 3008 MB
stepfunctions  | 2022-02-09 10:18:35.100: arn:aws:states:us-east-1:123456789012:execution:HelloWorld:e2e402fa-6bbd-4b56-87f1-4cf1a8fc1273 : {"Type":"TaskSubmitted","PreviousEventId":4,"TaskSubmittedEventDetails":{"ResourceType":"lambda","Resource":"invoke.waitForTaskToken","Output":"{\"StatusCode\":200,\"FunctionError\":null,\"LogResult\":null,\"Payload\":\"$LATEST\",\"ExecutedVersion\":null}"}}
stepfunctions  | 2022-02-09 10:18:35.101: arn:aws:states:us-east-1:123456789012:execution:HelloWorld:e2e402fa-6bbd-4b56-87f1-4cf1a8fc1273 : {"Type":"TaskSucceeded","PreviousEventId":5,"TaskSucceededEventDetails":{"ResourceType":"lambda","Resource":"invoke.waitForTaskToken","Output":"{\"test\": 123}"}}
stepfunctions  | 2022-02-09 10:18:35.102: [200] SendTaskSuccess <= {"sdkResponseMetadata":null,"sdkHttpMetadata":null}
stepfunctions  | 2022-02-09 10:18:35.102: [200] SendTaskSuccess <= {"sdkResponseMetadata":null,"sdkHttpMetadata":null}
stepfunctions  | 2022-02-09 10:18:35.103: arn:aws:states:us-east-1:123456789012:execution:HelloWorld:e2e402fa-6bbd-4b56-87f1-4cf1a8fc1273 : {"Type":"TaskStateExited","PreviousEventId":6,"StateExitedEventDetails":{"Name":"waitForTaskToken","Output":"{\"test\": 123}"}}
stepfunctions  | 2022-02-09 10:18:35.124: arn:aws:states:us-east-1:123456789012:execution:HelloWorld:e2e402fa-6bbd-4b56-87f1-4cf1a8fc1273 : {"Type":"ExecutionSucceeded","PreviousEventId":7,"ExecutionSucceededEventDetails":{"Output":"{\"test\": 123}"}}
1 Answer
0

Hi AWS-User-0409856,

I'm sorry you've encountered this issue. I was able to reproduce it and will look into what the problem is.

AWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions