Greengrass Lambda failing to start due to log pipe blocked

0

One of my lambda components is failing to start on an IOT device. Another IOT device received the lambda and it is triggering as expected. The first deployment worked (new IOT device) then subsequent ones failed with the same issues listed below. All the lambda does is log a message when it receives an event on a PUB/SUB topic (testing something out) using Python logger.

The errors I see in the logs (amongst others) are:

  • Refusing to execute because the active thread is interrupted
  • Error while waiting for LogHandler to fully shutdown.
  • Lambda log pipe /opt/greengrass/v2/work/xxxxx/logs/2/warn is blocked, attempting to unblock

Things I've tried:

  • I've tried a "reset" on the configuration and deployed a new version of the component, but this doesn't fix the issue.
  • I've rebooted the nucleus (running as a service), which also does nothing to the lambda

Any clues as to what's going on and what caused this?

Thanks!

  • I agree with the recommendation in the answer below to use native components, however if you still want help with the lambda, please provide the full logs from Greengrass and your component's log that show the error.

    Log pipe being blocked is not an error that should impact your lambda's execution in any way.

  • Interesting I see both comments stating we should use native components. I'm not against it for reasons posted below but inheriting a previous design.

    Is it not recommended because lambda support is not as mature and generally causes more problems? Or just specifically to subscribing to a topic and responding to events on the topic per the use case I'm having trouble with.

    We do use lambdas in several other use cases and I'm concerned that perhaps, this part of the GreenGrass eco-system is not as robust and should be avoided for more traditional approaches.

  • I recommend native components because there's less "magic" involved. Lambda tries to do a lot of things for you, but then if anything goes poorly you can't do anything about it. Native components on the other hand can have much better performance, debuggability, and less actual complexity (though it may seem more complex initially).

nwind21
asked 9 months ago228 views
1 Answer
0
Accepted Answer

If you are starting a new deployment on Greengrass V2 I would advice to use Generic components and not Lambda Functions components.

A generic component is simpler to write and debug and can more easily access devices resources if needed (such as cameras, etc).

All you need to do to trigger some logic on the reception of a message is to create a subscription to the pubsub topic and write the corresponding handler. You can refer to this sample code (from https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-publish-subscribe.html#ipc-publish-subscribe-examples)

import time
import traceback

from awsiot.greengrasscoreipc.clientv2 import GreengrassCoreIPCClientV2
from awsiot.greengrasscoreipc.model import (
    SubscriptionResponseMessage,
    UnauthorizedError
)


def main():
    topic = 'my/topic';

    try:
        ipc_client = GreengrassCoreIPCClientV2()
        # Subscription operations return a tuple with the response and the operation.
        _, operation = ipc_client.subscribe_to_topic(topic=topic, on_stream_event=on_stream_event,
                                                     on_stream_error=on_stream_error)
        print('Successfully subscribed to topic: ' + topic)

        # Keep the main thread alive, or the process will exit.
        try:
            while True:
                time.sleep(10)
        except InterruptedError:
            print('Subscribe interrupted.')

        # To stop subscribing, close the stream.
        operation.close()
    except UnauthorizedError:
        print('Unauthorized error while subscribing to topic: ' +
              topic, file=sys.stderr)
        traceback.print_exc()
        exit(1)
    except Exception:
        print('Exception occurred', file=sys.stderr)
        traceback.print_exc()
        exit(1)


def on_stream_event(event: SubscriptionResponseMessage) -> None:
    try:
        message = str(event.binary_message.message, 'utf-8')
        topic = event.binary_message.context.topic
        print('Received new message on topic %s: %s' % (topic, message))
    except:
        traceback.print_exc()


def on_stream_error(error: Exception) -> bool:
    print('Received a stream error.', file=sys.stderr)
    traceback.print_exc()
    return False  # Return True to close stream, False to keep stream open.

if __name__ == '__main__':
    main()
AWS
EXPERT
answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions