Spark with scala 2.12.x on EMR

Hi,

On EMR, Spark is built with Scala-2.11.x, which is currently the stable version. As per- https://spark.apache.org/releases/spark-release-2-4-0.html , Scala-2.12 is still under experimental support.
Our service team is already aware of this feature request, and they shall be adding Scala-2.12.0 support in coming releases, once it becomes stable. We don't have an ETA for this currently. AWS product teams regularly look into feature requests and bug reports, however, they have the final say on when to implement or fix them according to their own R&D schedules. Any new changes for EMR can be tracked at "https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-whatsnew.html ".

WORKAROUND:
For Workaround, you may build your own Spark and install it on EMR cluster as an application. EMR used Apache BigTop libraries to install applications(like Spark, hive etc.). Please refer(https://aws.amazon.com/blogs/big-data/building-and-deploying-custom-applications-with-apache-bigtop-and-amazon-emr/ ) for deploying custom applications on EMR cluster.

Hope it helps. Please let us know if you face any further issues. We would be happy to help you.

Thanks for your answer, cool