Greengrass StreamManager dependency problem

Question

Hello!

I created a Greengrass core device through the [online tutorials](https://docs.aws.amazon.com/greengrass/v2/developerguide/getting-started.html). On step **5b** I followed the **recommended option**, so I think it uses *sts:AssumeRole* for authentication.
Everything is working fine and I tried to install and use the AWS StreamManager Component. I updated my deployment and Stream Manager was successfully installed on my core device. But when I created a stream and appended data to that stream, I got errors in `/greengrass/v2/logs/aws.greengrass.StreamManager.log` about Stream Manager not being able to connect to IoT Analytics.

The full debug logs of Stream Manager can be found in [this gist](https://gist.github.com/hacor/364face5e03d1065ceccc04e88ed29cb)

I found following issues in this logs:
- Line #140: `2022-03-04T08:30:35.708Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. 2022 Mar 04 08:30:35,707 [DEBUG] (pool-7-thread-1) com.amazonaws.auth.AWSCredentialsProviderChain: Unable to load credentials from WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path.. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=STARTING}
`
- Line #167: `2022-03-04T08:30:36.085Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. 2022 Mar 04 08:30:36,074 [ERROR] (pool-7-thread-1) com.amazonaws.iot.greengrass.streammanager.export.destination.IOTAnalyticsDestination: Encountered error while exporting data to AWS IoT Analytics. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=STARTING}
`

When I created the file `/home/ggc_user/.aws/credentials` using basic credentials

```bash
[default]
aws_access_key_id = ******
aws_secret_access_key = *******
```

Stream manager works perfectly. 
As I read in the logs, the Stream Manager package should have the `aws-java-sdk-sts` module as a dependency in order to work with this method. Because the Stream manager component is not open source, I'm unable to create a PR. So I hope this message will reach the Stream Manager developers.

Here are the component versions used in my deployment:
```bash
Component Name: aws.greengrass.StreamManager
    Version: 2.0.14
    State: RUNNING
    Configuration: {"JVM_ARGS":"","LOG_LEVEL":"DEBUG","port":"8088","runWithDefault":{"posixUser":"ggc_user:ggc_group"},"STREAM_MANAGER_AUTHENTICATE_CLIENT":false,"STREAM_MANAGER_ENABLE_LOCK_ON_METADATA_STORE":"false","STREAM_MANAGER_EXPORTER_MAX_BANDWIDTH":"2147483647","STREAM_MANAGER_EXPORTER_S3_DESTINATION_MULTIPART_UPLOAD_MIN_PART_SIZE_BYTES":"5242880","STREAM_MANAGER_EXPORTER_THREAD_POOL_SIZE":"5","STREAM_MANAGER_SERVER_PORT":"8088","STREAM_MANAGER_STORE_ROOT_DIR":"."}
Component Name: aws.greengrass.TokenExchangeService
    Version: 2.0.3
    State: RUNNING
    Configuration: {"port":0.0}
Component Name: aws.greengrass.Nucleus
    Version: 2.5.3
    State: FINISHED
    Configuration: {"awsRegion":"eu-central-1","componentStoreMaxSizeBytes":"10000000000","deploymentPollingFrequencySeconds":"15","envStage":"prod","fleetStatus":{"periodicStatusPublishIntervalSeconds":86400.0},"greengrassDataPlanePort":"8443","httpClient":{},"iotCredEndpoint":"c18zy2f9lrcrq0.credentials.iot.eu-central-1.amazonaws.com","iotDataEndpoint":"a3sh1t0ug8wcry-ats.iot.eu-central-1.amazonaws.com","iotRoleAlias":"GreengrassV2TokenExchangeRoleAlias","jvmOptions":"-Dlog.store=FILE","logging":{},"mqtt":{"spooler":{}},"networkProxy":{"proxy":{}},"platformOverride":{},"runWithDefault":{"posixShell":"sh","posixUser":"ggc_user:ggc_group"},"telemetry":{}}
```

Best regards,

Hacor

Answer

Hacor,

You should delete the credentials file that you created, that's not how Greengrass should be getting credentials. Instead, Greengrass provides credentials to components including Stream manager using the Token Exchange Service component. https://docs.aws.amazon.com/greengrass/v2/developerguide/device-service-role.html

Greengrass uses the IoT Credentials Provider service which retrieves credentials from an IoT Role Alias which you configured. This role alias must point to an IAM role which has the necessary permissions for Stream manager such as IoT Analytics.

Answer

Hi all,

After spending a bit more time on this, we did confirm that `.aws/credentials` file is **not** required.

Our app is streaming to Kinesis via `StreamManager` using only the device cert and Token Exchange Service.

The confusion was mainly due to `StreamManager` performing batching, so it takes a few minutes for data to start showing up in AWS...even though the logs on the device show data being appended to the stream much earlier.

Answer

Hello Michael!

Thanks for the answer. I know I should delete the credentials file, I just created it in order to test whether I was capable of getting Stream Manager to work or not.
I provisioned the Greengrass core device with the installer option `provision=true` so it created the policies, role and the role alias automatically as a part of the installation. 
I also updated the role to use the `iotanalytics:BatchPutMessage` as mentioned in the description of the component [here](https://docs.aws.amazon.com/greengrass/v2/developerguide/stream-export-configurations.html#export-to-iot-analytics). Stream Manager should be able to access IoT Analytics.
I do want to put the attention to line #140 again of the gist mentioned above. Stream manager complains in the starting stage about being unable to assume roles. I really do think that's causing the problem.
The Token Exchange service seems up and running fine, my custom components are getting the `AWS_CONTAINER_CREDENTIALS_FULL_URI` environment variable perfectly.

I hope we can find a solution, so I can delete the credentials file.
Best regards,

Hacor

Answer

Did this ever get resolved?   I am trying to use `StreamManager 2.1.0` with a custom component and seeing similar behavior.   I too had questions about whether I needed the `.aws/credentials` file versus Token Exchange Service. Thanks

Answer

Hello All

Thanks for all the suggestions and advice. @ttnickb I assume all the AWS suggestions are probably right and it would have something todo with roles/permissions.
On my end, I never got to solve my issue without a credentials file (only allowing Stream Manager to do things of course). And now the project moved in such a way I didn't need the Stream Manager anymore.
So I kind of lost sight of this problem

So if you did find a solution, please post it here and it can certainly help others.

Thanks!
Kind Regards

Hacor

Greengrass StreamManager dependency problem

Relevant content