Proper way to interact with resources/services outside of GG v2

Question

Hi,  
We have a MQTT events flow generated outside of Greengrass Core on IoT device and we want to forward those events to AWS IoT Core using custom forwarder deployed as Greengrass component. It seems a few options are available.  
  
1. Use 3rd-party MQTT broker running outside GG core environment. It aggregates all events and provides some topics to subscribe to. GG component subscribes to these topics and once a new event appears, it pulls it, transforms it if needed and publishes to IoT core topic through IPC. GG component is a generic or lambda-based component written in Python so it is possible to use 'paho-mqtt' package to connect to external MQTT broker and 'awsiotsdk' to communicate to Iot Core via IPC. Looks fine in general. Custom authentication and authorization mechanizms can be implemented in MQTT broker and it's independend from GG.  
  
2. The same custom MQTT broker as in #1 but deployed as GG component. External datasources publish their events the same way but lifecycle of this broker is managed by GG because it is its component. Not bad, but it looks a bit redundant because GG v2 has it's own internal topics to be used to connect different components to each other.  
  
3. Use IPC and internal GG topics directly. External datasources publish events directly to these topics. This idea appeared base on this discussion - https://github.com/aws/aws-iot-device-sdk-python-v2/issues/145. Looks promising because in this case we use native GG mechanisms without any additional resources. The problem is authentication and authorization. In order to connect to internal topic we need to have AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT and SVCUID. AWS_GG_NUCLEUS_DOMAIN_SOCKET_FILEPATH_FOR_COMPONENT is static but I guess SVCUID is generated for each component individually and updated periodically. It raises some security concerns how to get this info from external processes, where to store it and how to change. But at least this approach exists and can be potentially used.  
  
So the question is which approach could be considered as the most proper, well-architectured and secured? I think GG developers team thought about this scenario and maybe they have some recommendations. Is #3 just a dirty hack or is it one of the 'official' ways to pass events to GG/IoT from outside? Maybe other options which I missed?  
  
Thanks in advance.  
  
Edited by: lacteolus on Feb 18, 2021 1:58 AM

Answer

Hi,  
  
3 should not be used as your are basically breaking the Greengrass authentication and authorization system's intent. It certainly can be done that way, but I would really caution against it. Either #1 or #2 makes the most sense to me; so much sense in fact that we've already implemented an example of this solution https://github.com/aws-greengrass/aws-greengrass-component-examples/tree/main/mqtt-bridge.  
This example code must be treated exactly as that, an example. It does not implement any security above username+password and has very minimal error handling. You can have a look at it to give you a head start, but I would tell you to not deploy it as-is into a production environment as it has not been tested to those standards.  
  
Hope that helps,  
Michael Dombrowski

Answer

@lacteolus I've done #1 using the exact method you described, and it works reliably. Two things to consider:  
  
1) Spooling from GG to IOT Core is only memory-backed at the moment, there's no persistent message storage if your device goes offline. May/may not be a deal breaker  
2) I think there's a tiny leak in the python IPC code : https://github.com/aws/aws-iot-device-sdk-python-v2/issues/149  
  
There's a another way which i'm starting to consider, and that's using StreamManager ( https://forums.aws.amazon.com/thread.jspa?threadID=335746&tstart=0  )- which does persistent message storage, but you'd need to run a lambda in AWS to push data from a kinesis data stream to IoT Core.

Answer

As I know AWS has plans to add "world-facing" broker as part of greengrass core. I think it would help to communicate components and other services running on device.

Proper way to interact with resources/services outside of GG v2

Relevant content