Spark artifacts location not found in EMR

Hello,

Please note that the Spark jars are currently not available publicly in AWS EMR maven repository at S3 unfortunately, only Apache Hadoop and Apache Hive libraries and dependencies are available. I recommend to reach out your TAM with business justification with highlighting the feature request requirement for Spark artifacts availability.

On the other hand, the main Spark project publishes Spark libraries and dependencies that are mostly compatible with EMR Spark versions. You can browse them on the Spark Packages site or Maven Central. Just search for "spark" and filter by the desired Spark version and inter-relate with Amazon's EMR release guide which lists the major components and versions included in each EMR release.

Alternatively, you can launch an EMR cluster and analyse the libraries installed there. Basically, you can download the spark jars from EMR primary node present in (/usr/lib/spark/) to build your custom application as EMR uses public Spark releases with additional customization so these versions should be obtainable. Please ensure you check licensing if reusing or distributing.