Questions tagged with AWS IoT Core

Content language: English

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Setting up a raspberry pi and PreTestValidation is failing

When I run the test it fails saying it cannot make the directory /temp/idt. I am not sure where exactly it is trying to make this directory though. I am running the test from terminal on my Mac but I have the latest MacOS installed and you cannot create a folder in the root anymore. I can modify a file that will create something like a shortcut there that the system perceives to be a folder but it's just an alias. Is there a way to bypass this test or change the directory location so it won't bottleneck here? the command is using a "-p" switch so in theory if I know exactly where this is being created I can just create it manually and it should overlook after that. I just can't seem to see where it should go. The details, output, and logs are below. I appreciate any input here. So simply put, I have all the prelims done, the environment: I'm running AWS CLI on my MacBook Air M2 (2022) running MacOS Ventura and the edge device is a Raspberry Pi 4 model B running 11.8. Testing suite: devicetester_greengrass_v2_4.5.11_testsuite_2.4.1_mac when I run ./devicetester_mac_x86-64 run-suite --userdata userdata.json the output is: davidosullivan@Davids-MacBook-Air bin % ./devicetester_mac_x86-64 run-suite --userdata userdata.json [INFO] [2022-11-28 05:39:34]: Running pool=pool1 idt=4.5.11 suite=GGV2Q_2.4.1 [WARNING] [2022-11-28 05:39:34]: IDT is accessing to Device Under Test with an insecure SSH method, to make it more secure you can provide publicKeyPath to device.json or resource.json file. More details can be found on AWS IoT Device Tester documentation Configure IDT settings section. [INFO] [2022-11-28 05:39:35]: Automatic updates are supported for this release of IDT, checking for updates. [INFO] [2022-11-28 05:39:35]: executionId=a8751a98-6ed6-11ed-b7fb-f293db416e41 suiteId=GGV2Q [WARNING] [2022-11-28 05:39:35]: Failed to get version of Greengrass with error exit status 1 [WARNING] [2022-11-28 05:39:35]: Device Tester is not able to check for any latest IDT test suites [INFO] [2022-11-28 05:39:35]: Using test suite "GGV2Q_2.4.1" for execution [WARNING] [2022-11-28 05:39:35]: IDT is accessing to Device Under Test with an insecure SSH method, to make it more secure you can provide publicKeyPath to device.json or resource.json file. More details can be found on AWS IoT Device Tester documentation Configure IDT settings section. time="2022-11-28T05:39:35+01:00" level=info msg="GGV2 Suite Info" hostArch=amd64 hostName=Davids-Air.home hostOs=darwin groupId=pretestvalidation testCaseId=pretestvalidation deviceId=raspberrypi time="2022-11-28T05:39:35+01:00" level=info msg="Running test case" time="2022-11-28T05:39:35+01:00" level=info msg="Setting up device for testing" time="2022-11-28T05:39:36+01:00" level=info msg="Device architecture: armv7l" time="2022-11-28T05:39:36+01:00" level=info msg="Cleaning up test resources..." time="2022-11-28T05:39:36+01:00" level=info msg="Temporary resources on DUT are successfully removed" time="2022-11-28T05:39:36+01:00" level=info msg="Cleaning up dut resources..." time="2022-11-28T05:39:36+01:00" level=info msg="Cleaning up AWS resources... This may take a while..." [ERROR] [2022-11-28 05:39:36]: Test exited unsuccessfully executionId=a8751a98-6ed6-11ed-b7fb-f293db416e41 testCaseId=pretestvalidation error=exit status 1 [INFO] [2022-11-28 05:39:36]: All tests finished. executionId=a8751a98-6ed6-11ed-b7fb-f293db416e41 [INFO] [2022-11-28 05:39:36]: ========== Test Summary ========== Execution Time: 1s Tests Completed: 1 Tests Passed: 0 Tests Failed: 1 Tests Skipped: 0 ---------------------------------- Test Groups: pretestvalidation: FAILED ---------------------------------- Failed Tests: Group Name: pretestvalidation Test Name: pretestvalidation Reason: Command '{mkdir -p /temp/idt map[] 0s}' exited with code 1. Error output: mkdir: cannot create directory ‘/temp/idt’: Permission denied . ---------------------------------- Path to AWS IoT Device Tester Report: /Users/davidosullivan/projects/idt/devicetester_greengrass_v2_mac/results/a8751a98-6ed6-11ed-b7fb-f293db416e41/awsiotdevicetester_report.xml Path to Test Execution Logs: /Users/davidosullivan/projects/idt/devicetester_greengrass_v2_mac/results/a8751a98-6ed6-11ed-b7fb-f293db416e41/logs Path to Aggregated JUnit Report: /Users/davidosullivan/projects/idt/devicetester_greengrass_v2_mac/results/a8751a98-6ed6-11ed-b7fb-f293db416e41/GGV2Q_Report.xml ================================== Note: davidosullivan@Davids-MacBook-Air bin % The Logs <testsuites name="GGV2Q results" time="1" tests="1" failures="0" skipped="0" errors="1" disabled="0"> <testsuite name="pretestvalidation" package="" tests="1" failures="0" time="1" disabled="0" errors="1" skipped="0"> <testcase classname="GGV2Q pretestvalidation" name="pretestvalidation"> <error>Command &#39;{mkdir -p /temp/idt map[] 0s}&#39; exited with code 1. Error output: mkdir: cannot create directory ‘/temp/idt’: Permission denied&#xA;.</error> </testcase> </testsuite> </testsuites> <apnreport> <awsiotdevicetesterversion>4.5.11</awsiotdevicetesterversion> <testsuiteversion>2.4.1</testsuiteversion> <signature>283938a8376e684465e29efa2a88e61c1543dfd58dff21451248d707d73403a4bd81f766e97977ca46a986384b863a4698bdbc39c58d9f6d8edc3d26d49348a7</signature> <keyname>65d77318-3995-4993-9dd5-01b329742307</keyname> <session> <testsession>958c70de-6ed6-11ed-b139-f293db416e41</testsession> <starttime>2022-11-28T05:39:04.549453+01:00</starttime> <endtime>2022-11-28T05:39:05.584253+01:00</endtime> </session> <awsproduct> <name>AWS Greengrass</name> <version></version> <features> <feature name="aws-iot-greengrass-v2-core" value="not-supported" type="required"></feature> <feature name="aws-iot-greengrass-v2-ml-component" value="not-supported" type="optional"></feature> <feature name="aws-iot-greengrass-v2-docker-application-manager" value="not-supported" type="optional"></feature> <feature name="aws-iot-greengrass-v2-stream-management-component" value="not-supported" type="optional"></feature> <feature name="aws-iot-greengrass-v2-hsi" value="not-supported" type="optional"></feature> </features> </awsproduct> <device> <sku>hbshub</sku> <name>pool1</name> <features> <feature name="arch" value="armv7l"></feature> <feature name="ml" value="no"></feature> <feature name="docker" value="no"></feature> <feature name="streamManagement" value="no"></feature> <feature name="hsi" value="hsm | no"></feature> <feature name="platform.operatingSystem" value="Linux"></feature> </features> <executionMethod>ssh</executionMethod> </device> <devenvironment> <os name="darwin"></os> </devenvironment> <qualificationsuite> <checksumkeyname>GGV2Q_mac_2.4.1</checksumkeyname> <checksum>ca8c0331d16637835af8dc2f403ffcf217a70022e6c89e119f9be53d6ec2bb7f4ca30f21984d49a3a10634b0077a2a36</checksum> </qualificationsuite> <report> <testsuites name="GGV2Q results" time="1" tests="1" failures="0" skipped="0" errors="1" disabled="0"> <testsuite name="pretestvalidation" package="" tests="1" failures="0" time="1" disabled="0" errors="1" skipped="0"> <testcase classname="GGV2Q pretestvalidation" name="pretestvalidation"> <error>Command &#39;{mkdir -p /temp/idt map[] 0s}&#39; exited with code 1. Error output: mkdir: cannot create directory ‘/temp/idt’: Permission denied&#xA;.</error> </testcase> </testsuite> </testsuites> </report> </apnreport>
1
answers
0
votes
19
views
asked 7 days ago

AWS Greengrass doesn't send data to AWS Kinesis

The main purpose of my program is to connect to an incoming MQTT channel, and send the data received to my AWS Kinesis Stream called "MyKinesisStream". Here is my code: ``` import argparse import logging import random from paho.mqtt import client as mqtt_client from stream_manager import ( ExportDefinition, KinesisConfig, MessageStreamDefinition, ResourceNotFoundException, StrategyOnFull, StreamManagerClient, ReadMessagesOptions, ) broker = 'localhost' port = 1883 topic = "clients/test/hello/world" client_id = f'python-mqtt-{random.randint(0, 100)}' username = '...' password = '...' logging.basicConfig(level=logging.INFO) logger = logging.getLogger() args = "" def connect_mqtt() -> mqtt_client: def on_connect(client, userdata, flags, rc): if rc == 0: print("Connected to MQTT Broker!") else: print("Failed to connect, return code %d\n", rc) client = mqtt_client.Client(client_id) client.username_pw_set(username, password) client.on_connect = on_connect client.connect(broker, port) return client def sendDataToKinesis( stream_name: str, kinesis_stream_name: str, payload, batch_size: int = None, ): try: print("Debug: sendDataToKinesis with params:", stream_name + " | ", kinesis_stream_name, " | ", batch_size) print("payload:", payload) print("type payload:", type(payload)) except Exception as e: print("Error while printing out the parameters", str(e)) logger.exception(e) try: # Create a client for the StreamManager kinesis_client = StreamManagerClient() # Try deleting the stream (if it exists) so that we have a fresh start try: kinesis_client.delete_message_stream(stream_name=stream_name) except ResourceNotFoundException: pass exports = ExportDefinition( kinesis=[KinesisConfig( identifier="KinesisExport" + stream_name, kinesis_stream_name=kinesis_stream_name, batch_size=batch_size, )] ) kinesis_client.create_message_stream( MessageStreamDefinition( name=stream_name, strategy_on_full=StrategyOnFull.OverwriteOldestData, export_definition=exports ) ) sequence_no = kinesis_client.append_message(stream_name=stream_name, data=payload) print( "Successfully appended message to stream with sequence number ", sequence_no ) readValue = kinesis_client.read_messages(stream_name, ReadMessagesOptions(min_message_count=1, read_timeout_millis=1000)) print("DEBUG read test: ", readValue) except Exception as e: print("Exception while running: " + str(e)) logger.exception(e) finally: # Always close the client to avoid resource leaks print("closing connection") if kinesis_client: kinesis_client.close() def subscribe(client: mqtt_client, args): def on_message(client, userdata, msg): print(f"Received `{msg.payload.decode()}` from `{msg.topic}` topic") sendDataToKinesis(args.greengrass_stream, args.kinesis_stream, msg.payload, args.batch_size) client.subscribe(topic) client.on_message = on_message def run(args): mqtt_client_instance = connect_mqtt() subscribe(mqtt_client_instance, args) mqtt_client_instance.loop_forever() def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser() parser.add_argument('--greengrass-stream', required=False, default='...') parser.add_argument('--kinesis-stream', required=False, default='MyKinesisStream') parser.add_argument('--batch-size', required=False, type=int, default=500) return parser.parse_args() if __name__ == '__main__': args = parse_args() run(args) ``` (the dotted parts ... are commented out as they are sensitive information, but they are correct values.) The problem is that it just won't send any data to our kinesis stream. I get the following STDOUT from the run: ``` 2022-11-25T12:13:47.640Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. Connected to MQTT Broker!. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} 2022-11-25T12:13:47.641Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. Received `{"machineId":2, .... "timestamp":"2022-10-24T12:21:34.8777249Z","value":true}` from `clients/test/hello/world` topic. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} 2022-11-25T12:13:47.641Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. Debug: sendDataToKinesis with params: test | MyKinesisStream | 100. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} 2022-11-25T12:13:47.641Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. payload: b'{"machineId":2,... ,"timestamp":"2022-10-24T12:21:34.8777249Z","value":true}'. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} 2022-11-25T12:13:47.641Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. type payload: <class 'bytes'>. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} 2022-11-25T12:13:47.641Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. Successfully appended message to stream with sequence number 0. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} 2022-11-25T12:13:47.641Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. DEBUG read test: [<Class Message. stream_name: 'test', sequence_number: 0, ingest_time: 1669376980985, payload: b'{"machineId":2,"mach'>]. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} 2022-11-25T12:13:47.641Z [INFO] (Copier) jp.co.xyz.StreamManagerKinesis: stdout. closing connection. {scriptName=services.jp.co.xyz.StreamManagerKinesis.lifecycle.Run, serviceName=jp.co.xyz.StreamManagerKinesis, currentState=RUNNING} ``` So we can see that the data arrives from MQTT, the python code executes the append message, and it seems that my kinesis streams have the information as it can read it in the next step... then closes the connection without any error. But the problem is, that from AWS side, we cannot see the data arriving on the stream: [![screnshot of the aws console][1]][1] What can be the problem here? Our greengrass core is configured properly, can be accessed from the AWS, and the Component is running and healthy also: [![Screenshot of IoT Core status][2]][2] [![Screenshot of the state if the StreamManager component][3]][3] [1]: https://i.stack.imgur.com/wN5I4.png [2]: https://i.stack.imgur.com/JMJmn.png [3]: https://i.stack.imgur.com/2qnfn.png
1
answers
0
votes
19
views
ForestG
asked 9 days ago