By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Can we use a conda environment on AWS Glue-ETL?

0

I have a python file that runs on a conda environment (i.e, qiime2 which is a conda env installed from "https://data.qiime2.org/distro/core/qiime2-2021.4-py38-linux-conda.yml")

Is there a way to setup a conda environment on AWS Glue-ETL before running a job using my python file? Is seems it is possible to provide pip-installable python modules or whl files in the "Python library path" field, but I cannot find how to set up the running-environment using yml file.

Topics
AnalyticsDatabase
Tags
AWS GlueExtract Transform & Load Data
Language
English
rePost-User-9256543
asked 2 years ago744 views
1 Answer
0

Hi,

AWS Glue lets you install additional Python modules and libraries for use with AWS Glue ETL. If you want to install Python modules that aren’t included out of the box, please follow the steps lined out in the documentation here:

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.html

Thanks.

profile pictureAWS
Nick Furr
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content