Lambda error: Calling the invoke API action failed with this message: Network Failure

Question

I have a lambda function that writes ~6000 retained messages to IoT Core. I am able to run this function locally without problem.
In Lambda after about half of the data has been processed, it stops running with the error:
`Calling the invoke API action failed with this message: Network Failure`

This error does not get piped to CloudWatch, but I do see it if I manually execute a Test in the Lambda console.

It seems that it might be exceeding the IoT quota for 'Retained message inbound publish requests per second per topic', which is fixed at 1 per second. But since I am able to run it locally and successfully publish the retained messages, that seems like it might not be the problem.

Answer

The function is running today with no errors. No code changes. It takes about 500s to complete, so it is on the longer side for a lambda function.

Since nothing is logged to CloudWatch upon failure, it seems like some lambda supervisor process was killing the function.

The lambda function is triggered by EventBridge. Is there any way to know that the lambda function failed like that to be able to retry?

Answer

There can be many reasons for things to fail - it's unfortunate in this case that the error message isn't more detailed. Is it possible to enable additional debugging to see what the underlying problem is?

I agree that it's highly probably that there is some sort of throttling happen - and the debugging might throw some light on that.

That said, in the bigger picture there are always going to be cases where things fail and the code needs to retry. Worth reading: https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/

Lambda error: Calling the invoke API action failed with this message: Network Failure

Relevant content