State Machine times-out on Choice

Question

I updated my state machine to use a task token and now it's timing out when it gets to the choice. This was previously working, and the cloudwatch logs for the first supportticketparse lambda function aren't indicating that anything is wrong. My guess is that I have and error in the payload definition, but I'm not sure. Is anything obviously wrong with the definition? If not, where else can I look for a potential problems?

```
{
  "Comment": "A description of my state machine",
  "StartAt": "zen sort",
  "TimeoutSeconds": 60,
  "States": {
    "zen sort": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
      "OutputPath": "$.Payload",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:363540650061:function:support-ticket-parse:$LATEST",
        "Payload": {
          "ExecutionContext.$": "$$",
          "APIGatewayEndpoint": "https://9084s1lza7.execute-api.us-east-1.amazonaws.com/test/",
          "Payload.$": "$"
        }
      },
      "Next": "Choice"
    },
    "Choice": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.name",
          "StringMatches": "New User Account ",
          "Next": "send email"
        },
        {
          "Not": {
            "Variable": "$.name",
            "StringMatches": "New User Account "
          },
          "Next": "Fail"
        }
      ],
      "Default": "Fail"
    },
    "send email": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
      "OutputPath": "$.Payload",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:363540650061:function:send_email:$LATEST",
        "Payload": {
          "ExecutionContext.$": "$$",
          "APIGatewayEndpoint": "https://9084s1lza7.execute-api.us-east-1.amazonaws.com/test/",
          "Payload.$": "$"
        }
      },
      "Retry": [
        {
          "ErrorEquals": [
            "Lambda.ServiceException",
            "Lambda.AWSLambdaException",
            "Lambda.SdkClientException"
          ],
          "IntervalSeconds": 2,
          "MaxAttempts": 6,
          "BackoffRate": 2
        }
      ],
      "End": true
    },
    "Fail": {
      "Type": "Fail"
    }
  }
}
```

Accepted Answer

When you added .waitForTaskToken to your Lambda task, did you also add logic elsewhere to receive the task token (in the Lambda function or in something downstream that you passed the token to) and to call back to Step Functions to close the loop? If not, then I believe you are just seeing the expected behavior here. With the callback pattern / .waitForTaskToken, step functions will call the target specified in your task then pause that part of the workflow until you call back or there's a timeout.
https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html#connect-wait-token

This seems the most likely case, however you stated that it "times-out on Choice" which would not align. I wonder if it just seems to look that way because you can see that the Lambda function was called but don't see the Choice state having executed.

As a side, it seems that you are calling Lambda functions that are then calling API-Gateway. It's possible you need to have these proxy Lambda functions, but if not, note that you can invoke API-Gateway directly from Step Functions.
https://docs.aws.amazon.com/step-functions/latest/dg/connect-api-gateway.html

Answer

Have you configured logging on your step function? Refer to this if you haven't - https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html

Logs should provide you a better idea of why the Step Function is getting timed-out?

Also I looked at your step function. I don't see a token in the callback to the lambda getting passed

Take a look at the example on this https://docs.aws.amazon.com/step-functions/latest/dg/connect-lambda.html

{  
   "StartAt":"GetManualReview",
   "States":{  
      "GetManualReview":{  
         "Type":"Task",
         "Resource":"arn:aws:states:::lambda:invoke.waitForTaskToken",
         "Parameters":{  
            "FunctionName":"get-model-review-decision",
            "Payload":{  
               "model.$":"$.new_model",
               "token.$":"$$.Task.Token" --> This is missing in your step function
            },
            "Qualifier":"prod-v1"
         },
         "End":true
      }
   }
}
This has more information on the role of the token for callback functions

State Machine times-out on Choice

Relevant content