- Mais recentes
- Mais votos
- Mais comentários
Hello,
Please note that the Spark jars are currently not available publicly in AWS EMR maven repository at S3 unfortunately, only Apache Hadoop and Apache Hive libraries and dependencies are available. I recommend to reach out your TAM with business justification with highlighting the feature request requirement for Spark artifacts availability.
On the other hand, the main Spark project publishes Spark libraries and dependencies that are mostly compatible with EMR Spark versions. You can browse them on the Spark Packages site or Maven Central. Just search for "spark" and filter by the desired Spark version and inter-relate with Amazon's EMR release guide which lists the major components and versions included in each EMR release.
Alternatively, you can launch an EMR cluster and analyse the libraries installed there. Basically, you can download the spark jars from EMR primary node present in (/usr/lib/spark/) to build your custom application as EMR uses public Spark releases with additional customization so these versions should be obtainable. Please ensure you check licensing if reusing or distributing.
Conteúdo relevante
- AWS OFICIALAtualizada há 2 anos
- AWS OFICIALAtualizada há 3 anos
- AWS OFICIALAtualizada há um ano
- AWS OFICIALAtualizada há um ano