How to add additional library i.e. databricks spark xml to a running EMR cluster and access it in Notebook

0

How to add additional library i.e. databricks spark xml to a running EMR cluster and access it in Notebook

Rajeev
asked 5 months ago216 views
1 Answer
0

Hi,

You can try the below steps to install databricks spark xml on EMR cluster.

On EMR Master node:

cd /usr/lib/spark/jars
sudo wget https://repo1.maven.org/maven2/com/databricks/spark-xml_2.11/0.9.0/spark-xml_2.11-0.9.0.jar

Make sure to select the correct jar according to your Spark version and the guidelines provided in https://github.com/databricks/spark-xml.

Then, launch your Jupyter notebook and you should be able to run the following:

df = spark.read.format('com.databricks.spark.xml').options(rootTag='objects').options(rowTag='object').load("s3://bucket-name/sample.xml")

Please feel free to reach to AWS support for any further assistance needed on the same.

Thank you!

AWS
SUPPORT ENGINEER
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions