GreenGrass Nucleus Fails to Start on ArmV7 Architecture

0

Hello,

When I attempt to run nucleus in an embedded Linux OS on Armv7 I get the error below.

  • Installation and provisioning appears to succeed.
  • /tmp is mounted exec
  • I'm running it without systemd as root with command /bin/sh /greengrass/v2/alts/current/distro/bin/loader
  • openjdk 11.0.22-internal 2024-01-16
  • AWS Greengrass v2.12.4

This is on a custom linux OS. I can add whatever packages might be missing.

  • Kernel v 5.4
  • glib 2.29
  • busybox used for most basic Linux functionality but we have a few things broken out
  • shell is ash (busybox)

Thanks in advance for the help!

Launching Nucleus...
Error while trying to setup Greengrass Nucleus
java.lang.UnsatisfiedLinkError: /tmp/AWSCRT_1574803778343577265libaws-crt-jni.so: /tmp/AWSCRT_1574803778343577265libaws-crt-jni.so: internal error
	at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
	at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2450)
	at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2506)
	at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2705)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2635)
	at java.base/java.lang.Runtime.load0(Runtime.java:768)
	at java.base/java.lang.System.load(System.java:1854)
	at software.amazon.awssdk.crt.CRT.extractAndLoadLibrary(CRT.java:324)
	at software.amazon.awssdk.crt.CRT.loadLibraryFromJar(CRT.java:358)
	at software.amazon.awssdk.crt.CRT.<clinit>(CRT.java:41)
	at software.amazon.awssdk.crt.CrtResource.<clinit>(CrtResource.java:104)
	at com.aws.greengrass.ipc.IPCEventStreamService.startup(IPCEventStreamService.java:97)
	at com.aws.greengrass.lifecyclemanager.KernelLifecycle.launch(KernelLifecycle.java:150)
	at com.aws.greengrass.lifecyclemanager.Kernel.launch(Kernel.java:271)
	at com.aws.greengrass.easysetup.GreengrassSetup.performSetup(GreengrassSetup.java:347)
	at com.aws.greengrass.easysetup.GreengrassSetup.main(GreengrassSetup.java:274)

contents of /greengrass/v2/logs/greengrass.log

2024-07-23T02:42:55.991Z [WARN] (main) oshi.util.FileUtil: File not found or not readable: /etc/lsb-release. {}
2024-07-23T02:42:56.040Z [INFO] (main) com.aws.greengrass.util.platforms.Platform: Getting platform instance com.aws.greengrass.util.platforms.unix.linux.LinuxPlatform.. {}
2024-07-23T02:42:56.353Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: No ongoing deployment detected. Proceed as default. {}
2024-07-23T02:42:57.413Z [INFO] (main) com.aws.greengrass.config.Configuration: config-loading. Read configuration from a file path. {path=/greengrass/v2/config/config.tlog}
2024-07-23T02:42:58.043Z [INFO] (main) com.aws.greengrass.lifecyclemanager.Kernel: effective-config-dump-complete. {file=/greengrass/v2/config/effectiveConfig.yaml}
2024-07-23T02:42:59.502Z [INFO] (main) com.aws.greengrass.lifecyclemanager.KernelLifecycle: system-start. Launch Nucleus. {configPath=/greengrass/v2/config, rootPath=/greengrass/v2, version=2.12.4}
2024-07-23T02:42:59.982Z [ERROR] (main) com.aws.greengrass.easysetup.GreengrassSetup: Error while trying to setup Greengrass Nucleus. {}
java.lang.UnsatisfiedLinkError: /tmp/AWSCRT_3605056115494501950libaws-crt-jni.so: /tmp/AWSCRT_3605056115494501950libaws-crt-jni.so: internal error
	at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
	at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2450)
	at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2506)
	at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2705)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2635)
	at java.base/java.lang.Runtime.load0(Runtime.java:768)
	at java.base/java.lang.System.load(System.java:1854)
	at software.amazon.awssdk.crt.CRT.extractAndLoadLibrary(CRT.java:324)
	at software.amazon.awssdk.crt.CRT.loadLibraryFromJar(CRT.java:358)
	at software.amazon.awssdk.crt.CRT.<clinit>(CRT.java:41)
	at software.amazon.awssdk.crt.CrtResource.<clinit>(CrtResource.java:104)
	at com.aws.greengrass.ipc.IPCEventStreamService.startup(IPCEventStreamService.java:97)
	at com.aws.greengrass.lifecyclemanager.KernelLifecycle.launch(KernelLifecycle.java:150)
	at com.aws.greengrass.lifecyclemanager.Kernel.launch(Kernel.java:271)
	at com.aws.greengrass.easysetup.GreengrassSetup.performSetup(GreengrassSetup.java:347)
	at com.aws.greengrass.easysetup.GreengrassSetup.main(GreengrassSetup.java:274)

2024-07-23T02:43:00.012Z [WARN] (Thread-1) com.aws.greengrass.lifecyclemanager.Kernel: Shutting down Nucleus due to external signal. {}
2024-07-23T02:43:00.020Z [INFO] (Thread-1) com.aws.greengrass.lifecyclemanager.KernelLifecycle: system-shutdown. {main=null}
2024-07-23T02:43:00.029Z [INFO] (Thread-1) com.aws.greengrass.lifecyclemanager.KernelLifecycle: Waiting for services to shutdown. {}
2024-07-23T02:43:00.090Z [INFO] (Thread-1) com.aws.greengrass.lifecyclemanager.Kernel: effective-config-dump-complete. {file=/greengrass/v2/config/effectiveConfig.yaml}
2024-07-23T02:43:00.098Z [INFO] (Serialized listener processor) com.aws.greengrass.lifecyclemanager.KernelLifecycle: executor-service-shutdown-initiated. {}
2024-07-23T02:43:00.100Z [INFO] (Thread-1) com.aws.greengrass.lifecyclemanager.KernelLifecycle: Waiting for executors to shutdown. {}
2024-07-23T02:43:00.102Z [INFO] (Thread-1) com.aws.greengrass.lifecyclemanager.KernelLifecycle: executor-service-shutdown-complete. {executor-terminated=true, scheduled-executor-terminated=true}
2024-07-23T02:43:00.105Z [INFO] (Thread-1) com.aws.greengrass.lifecyclemanager.KernelLifecycle: context-shutdown-initiated. {}
2024-07-23T02:43:00.120Z [INFO] (Thread-1) com.aws.greengrass.lifecyclemanager.KernelLifecycle: context-shutdown-complete. {}
  • Can you provide more details on the Embedded Linux OS (e.g., version, utils installed, shell, etc)?

  • Added some information to the question. We use a custom Linux OS.

  • See my comment below Greg_B's answer. It appears to be failing to select the correct libaws-crt-jni.so to extract.

  • I see you are getting closer to resolution (and maybe a defect in nucleus or the CRT!). When using busybox/ash, ensure that sudo is available too. Here's an example issue regarding it. And other busybox issues you might run into.

  • Thanks Gavin. I appreciate your help. Yes, I've already added sudo , and created the default user/group so that useradd isn't needed. I will debug the .so selection process based on our distro. I'm leaving on vacation for a week and will pick this back up after.

mtanner
posta 2 mesi fa163 visualizzazioni
2 Risposte
1
Risposta accettata

I know the cause of the linking issue now. Nucleus/AWS-CRT provides a hard float version of ./linux/armv7/glibc/libaws-crt-jni.so, which is incompatible with our processor.

we've cross compiled libaws-crt-jni.so for software floating point and installed that in /lib which solves the issue.

mtanner
con risposta 2 mesi fa
  • Glad you got to the root cause and great diving deep on that one. Also appreciate you providing the details here in case others run into it.

0

Hi. The permissions for /tmp look fine, but when other people have hit this problem, it's often because /tmp is mounted as noexec when it should be exec: https://docs.aws.amazon.com/greengrass/v2/developerguide/setting-up.html#greengrass-v2-requirements

profile pictureAWS
ESPERTO
Greg_B
con risposta 2 mesi fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande