Building awscrt for RISC V

Question

Hi all,

just trying to use AWS IoT Core on my Mango Pi board (riscv64 architecture). To do so, I need awscrt, among others.
There is no precompiled binary for this architecture and the build fails with 
```
~/crt/aws-lc/generated-src/linux-riscv64/crypto: No such file or directory
```
So I wonder whether someone tried this platform before and may share some thoughts/ideas.

Thanks, Jaroslav

full listing follows, just in case I am doing something stupid:
```
pi@nezha:~$ python3 -m pip install awscrt
Defaulting to user installation because normal site-packages is not writeable
Collecting awscrt
  Using cached awscrt-0.16.1.tar.gz (21.8 MB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: awscrt
  Building wheel for awscrt (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [161 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-riscv64-3.10
      creating build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/websocket.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/s3.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/mqtt5.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/mqtt.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/io.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/http.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/exceptions.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/crypto.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/common.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/checksums.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/auth.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/_test.py -> build/lib.linux-riscv64-3.10/awscrt
      copying awscrt/__init__.py -> build/lib.linux-riscv64-3.10/awscrt
      creating build/lib.linux-riscv64-3.10/awscrt/eventstream
      copying awscrt/eventstream/rpc.py -> build/lib.linux-riscv64-3.10/awscrt/eventstream
      copying awscrt/eventstream/__init__.py -> build/lib.linux-riscv64-3.10/awscrt/eventstream
      running build_ext
      > /usr/bin/cmake -H/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/crt -Bbuild/temp.linux-riscv64-3.10/deps -DCMAKE_INSTALL_PREFIX=build/temp.linux-riscv64-3.10/deps/install -DCMAKE_BUILD_TYPE=RelWithDebInfo
      -- The C compiler identification is GNU 11.3.0
      -- The CXX compiler identification is GNU 11.3.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      pkgconfig not found. Disabling unwind tests.
      -- Go not found. Disabling some code generation and using pre-generated code in generated-src/
      -- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
      -- Run check_run file_to_test 'memcmp_invalid_stripped_check.c', flag_to_set 'MEMCMP_INVALID_STRIPPED', and compile_flags '-O2 -g -DNDEBUG'.
      -- Test stdalign_check.c passed, enabling AWS_LC_STDALIGN_AVAILABLE
      -- Test builtin_swap_check.c passed, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
      -- The ASM compiler identification is GNU
      -- Found assembler: /usr/bin/cc
      Copying platform assembly files from /tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/crt/aws-lc/generated-src/linux-riscv64/crypto/ to /tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/build/temp.linux-riscv64-3.10/deps/aws-lc/crypto
      CMake Error at aws-lc/crypto/CMakeLists.txt:135 (file):
        file COPY cannot find
        "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/crt/aws-lc/generated-src/linux-riscv64/crypto/":
        No such file or directory.

-- Detected CMAKE_SYSTEM_PROCESSOR as riscv64
      -- Detected 64-Bit system
      -- Looking for pthread.h
      -- Looking for pthread.h - found
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- madvise() support detected
      -- clone() support detected
      -- S2N found target: crypto
      -- CMake 3.22.1
      CMake Deprecation Warning at aws-c-common/CMakeLists.txt:17 (cmake_policy):
        The OLD behavior for policy CMP0077 will be removed from a future version
        of CMake.
      
        The cmake-policies(7) manual explains that the OLD behaviors of all
        policies are deprecated and that a policy should be set to OLD only under
        specific short-term circumstances.  Projects should be ported to the NEW
        behavior and not rely on setting a policy to OLD.

-- Performing Test AWS_HAVE_GCC_OVERFLOW_MATH_EXTENSIONS
      -- Performing Test AWS_HAVE_GCC_OVERFLOW_MATH_EXTENSIONS - Success
      -- Performing Test AWS_HAVE_MSVC_MULX
      -- Performing Test AWS_HAVE_MSVC_MULX - Failed
      -- Performing Test AWS_HAVE_WINAPI_DESKTOP
      -- Performing Test AWS_HAVE_WINAPI_DESKTOP - Failed
      -- Performing Test AWS_ARCH_INTEL
      -- Performing Test AWS_ARCH_INTEL - Failed
      -- Performing Test AWS_ARCH_ARM64
      -- Performing Test AWS_ARCH_ARM64 - Failed
      -- Performing Test AWS_ARCH_ARM32
      -- Performing Test AWS_ARCH_ARM32 - Failed
      -- Performing Test AWS_HAVE_GCC_INLINE_ASM
      -- Performing Test AWS_HAVE_GCC_INLINE_ASM - Success
      -- Performing Test AWS_HAVE_AUXV
      -- Performing Test AWS_HAVE_AUXV - Success
      -- Performing Test AWS_HAVE_EXECINFO
      -- Performing Test AWS_HAVE_EXECINFO - Failed
      -- Performing Test AWS_HAVE_LINUX_IF_LINK_H
      -- Performing Test AWS_HAVE_LINUX_IF_LINK_H - Success
      -- Performing Test HAVE_M_AVX2_FLAG
      -- Performing Test HAVE_M_AVX2_FLAG - Failed
      -- Performing Test HAVE_AVX2_INTRINSICS
      -- Performing Test HAVE_AVX2_INTRINSICS - Failed
      -- Performing Test HAVE_MM256_EXTRACT_EPI64
      -- Performing Test HAVE_MM256_EXTRACT_EPI64 - Failed
      -- Looking for pthread_mutexattr_init
      -- Looking for pthread_mutexattr_init - not found
      -- Performing Test HAS_FPIC_FLAG
      -- Performing Test HAS_FPIC_FLAG - Success
      -- Performing Test HAS_64BIT_FILE_OFFSET_BY_DEFAULT
      -- Performing Test HAS_64BIT_FILE_OFFSET_BY_DEFAULT - Success
      -- Looking for fseeko
      -- Looking for fseeko - found
      -- Looking for stdint.h
      -- Looking for stdint.h - found
      -- Looking for stdbool.h
      -- Looking for stdbool.h - found
      -- Performing Test HAS_WGNU
      -- Performing Test HAS_WGNU - Failed
      -- Performing Test HAVE_SYSCONF
      -- Performing Test HAVE_SYSCONF - Success
      -- Looking for pthread_attr_setaffinity_np
      -- Looking for pthread_attr_setaffinity_np - found
      -- Performing Test PTHREAD_SETNAME_TAKES_2ARGS
      -- Performing Test PTHREAD_SETNAME_TAKES_2ARGS - Success
      -- Performing Test HAS_SANITIZERS
      -- Performing Test HAS_SANITIZERS - Failed
      -- Packaging currently only supported on Fedora.
      -- Using libcrypto from AWS-LC
      -- Configuring incomplete, errors occurred!
      See also "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/build/temp.linux-riscv64-3.10/deps/CMakeFiles/CMakeOutput.log".
      See also "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/build/temp.linux-riscv64-3.10/deps/CMakeFiles/CMakeError.log".
      Traceback (most recent call last):
        File "", line 2, in 
        File "", line 34, in 
        File "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/setup.py", line 348, in 
          setuptools.setup(
        File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/setup.py", line 255, in run
          self._build_dependencies(dep_build_dir, dep_install_path)
        File "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/setup.py", line 247, in _build_dependencies
          self._build_dependencies_impl(build_dir, install_path)
        File "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/setup.py", line 195, in _build_dependencies_impl
          run_cmd(cmake_args)
        File "/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/setup.py", line 37, in run_cmd
          subprocess.check_call(args)
        File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/usr/bin/cmake', '-H/tmp/pip-install-qsvx9kyw/awscrt_19c9b5f13b6e44febba6335902605ce9/crt', '-Bbuild/temp.linux-riscv64-3.10/deps', '-DCMAKE_INSTALL_PREFIX=build/temp.linux-riscv64-3.10/deps/install', '-DCMAKE_BUILD_TYPE=RelWithDebInfo']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for awscrt
  Running setup.py clean for awscrt
Failed to build awscrt
```

Answer

Hello Jaroslav,

This is a common issue when there are missing dependencies or when the build process expects certain files to be present which aren’t. Therefore, before building awscrt, make sure all the required dependencies are installed on your system. 
The logs mention that it’s using pre-generated code in `generated-src/` due to the absence of Go and Perl. Ensure that this pre-generated code is indeed compatible with the riscv64 architecture.
Anyway, if you have OpenSSL installed, you can try setting the environment variable `AWS_CRT_BUILD_USE_SYSTEM_LIBCRYPTO=1` before running the pip install command. This will instruct the build process to use the system’s `libcrypto `instead of building its own.

If none works, find alternative libraries...

Building awscrt for RISC V

相关内容