malformed proxy response with middy, but works with serverless invoke local

0

Hi, since yesterday we are observing our HTTP API started to fail due to a malformed proxy response. In API GW logs we have:

"httpMethod": "POST",
"routeKey": "POST /login",
"status": "500",
"protocol": "HTTP/1.1",
"responseLength": "35",
"error": "Internal Server Error",
"errorResponseType": "MALFORMED_PROXY_RESPONSE",
"integrationError": "The response from the Lambda function doesn't match the format that API Gateway expects. Invalid status code",
"integrationStatus": "200",
"authorizerError": "-"

No code changes. Does anyone know what may happen?

We have noticed that the issue occurs only when using middy middleware. There is no issue when running lambda locally with serverless invoke local

1 Answer
0

If there were no code changes on your side then I'd suggest creating a support case with the team to troubleshoot - from re:Post we have no visibility of the services you're using so debugging is pretty difficult.

That said: If you're only seeing errors with that particular middleware my thought would be to turn on additional debugging in the Lambda function; log the output before it is sent to API Gateway and try to determine what responses are causing that error.

profile pictureAWS
EXPERT
answered a year ago
  • Hi Thanks. Currently no support plan :/. We found 2 workarounds for the issue: either stop using middy or wrap middy with a function expression:

    const handler = middy(targetHandler).use(someMiddleware());
    const wrappedHandler = async (event, context) => {
          return handler(event, context);
     };
    
    export { wrappedHandler };
    

    From the experiments, we found that the same corrupted behavior happens if middy is wrapped with the following function expression:

    const wrappedHandler = async (event, context, callback) => {
          return handler(event, context, callback);
     };
    

    I think adding callback in the function expression made Lambda expect us to use it to return output from lambda. Middy instance as well expects callback as 3rd param. I do not know what happened Lambda started to interpret the handler in a different way. Or maybe I miss something here? I will be grateful for more suggestions.

  • Just a guess: If you are using async function calls within a Lambda function you must make sure that they complete before the Lambda function exits. The Lambda service doesn't know that an async call hasn't completed so it can prematurely terminate your function and in that case the returned data from the Lambda function will be incomplete. It's possible that there is a timing issue here where it used to work but under load (maybe?) the timing slips a bit and then your Lambda errors out.

  • I would say everything is properly awaited. What you say does not explain lambda behavior when middy wrapper is applied. Other thing is what you mean under load? many of our lambdas were affected at the same moment. Middy contributors share suspicions about how "callback" in function expression might be interpreted by Lambda https://github.com/middyjs/middy/issues/1005

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions