Installing Python Package TensorFlow on Amazon Linux 2023 for Graviton

As follow-on to this previous article discussing how to install Python package GeoPandas on Graviton, this article provides a process for compiling the HDF5 package on Graviton which is a prerequisite to install TensorFlow.

As of the time of writing in April 2024, Amazon EMR does not provide support for TensorFlow on Graviton based clusters. Over time, the software packages available for Arm continue to grow, but there are still areas where software is not yet packaged and immediately available in repositories from install. Recently, a customer approached us for help with installing the Python library TensorFlow on Amazon Linux 2023 for Graviton. On x86, the software is compiled, packaged and installs with a single command from the PyPI repo. The same attempt on Graviton fails due to prerequisite packages not being available.

The below process can be used to install the HDF5 and TensorFlow in a custom AMI or as a bootstrap action when launching your Amazon EMR cluster.

#!/bin/bash

#
# Install OS Dependencies
sudo dnf -y install gcc-c++ cpp sqlite-devel libtiff cmake python3-pip \
    python-devel python3.11-devel openssl-devel tcl libtiff-devel libcurl-devel \
    swig libpng-devel libjpeg-turbo-devel expat-devel python3-Cython

cd ~

#
# Build HDF5
wget https://hdf-wordpress-1.s3.amazonaws.com/wp-content/uploads/manual/HDF5/HDF5_1_14_3/src/hdf5-1.14.3.tar.gz
tar zxvf hdf5-1.14.3.tar.gz
cd hdf5-1.14.3
./configure -prefix=/usr
make -j -l6
sudo make install

cd ~

#
# Install H5py and TensorFlow
HDF5_DIR=/usr pip3 install --no-binary=h5py h5py tensorflow

With this procedure, our customer was able to install TensorFlow and launch EMR clusters based on Graviton instances, saving cost with their data analytics workload.

Reach out If you run into similar issues with packages that are not yet available for Arm. We are interested in helping ensure your workloads can take advantage of all that Graviton has to offer.