GGv2 StreamManager connection error

Question

Hi there,

I used the stream manager to upload video file to s3. I have 2 devices running with the same firmware version. but one of the stop sending data to s3

here is the error I can find in the log

```
2022-09-12T01:19:41.495Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. 2022 Sep 12 11:19:41,492 [ERROR] (pool-7-thread-2) com.amazonaws.iot.greengrass.streammanager.export.upload.MessageUploaderTask: Encountered Throwable when exporting messages. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.495Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. com.amazonaws.SdkClientException: Unable to execute HTTP request: Input/output error.

2022-09-12T01:19:41.496Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.496Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.497Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.497Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.498Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.498Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.499Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.500Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.500Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.501Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.501Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.502Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.502Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6508) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.503Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1856) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.503Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1816) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.503Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.iot.greengrass.streammanager.export.destination.S3TaskDestination.makeSinglePutRequest(S3TaskDestination.java:435) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.504Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.iot.greengrass.streammanager.export.destination.S3TaskDestination.sendMessage(S3TaskDestination.java:155) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.504Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at com.amazonaws.iot.greengrass.streammanager.export.upload.MessageUploaderTask.upload(MessageUploaderTask.java:77) ~[AWSGreengrassGreenlake-1.0-super.jar:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.505Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) [?:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.505Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
2022-09-12T01:19:41.506Z [INFO] (Copier) aws.greengrass.StreamManager: stdout. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]. {scriptName=services.aws.greengrass.StreamManager.lifecycle.startup.script, serviceName=aws.greengrass.StreamManager, currentState=RUNNING}
```

Thanks for your help

Answer

Hello,

The error `Unable to execute HTTP request: Input/output error`  seems to indicate some sort of lower-level issue with the availability/upload of the video file.

Looks like the actual S3 PutObject request is failing, so I am wondering if there is an issue with the video file itself. Seems it is reaching this point in the SDK: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/http/AmazonHttpClient.java#L1219

Are there any other logs, and is Stream Manager attempting to retry the upload?

How is the video file being generated and can you confirm that it is valid at the time of upload? Are you also sending any other messages on the stream apart from the video file?

Thanks!

GGv2 StreamManager connection error

Contenido relevante