What is the most cost efficient and fastest way to start GLUE ETL development

Hi,

It always depend on the actual priorities you have to define the trade off between cost efficiency and developer experience.

The most cost efficient I would agree is using the docker container . If you prefer to use a traditional IDE environment instead of a notebook you should be able to use it for that as well. Performance will depend on the local machine.

If you give more weight to a flexible developer experience using notebooks and performance when running the jobs or cells then Interactive sessions would be a better choice. You can install your notebook with all extensions and tune the number of DPUs you want to use changing that between sessions.

If the developer is fine with a managed notebook, Glue Studio Notebooks would have the same cost of Interactive sessions (it just depends on the DPU you select for the session and duration of the session), same configuration based on session . Glue Studio Notebooks are only available in some regions currently.

hope this helps

Yes , I agree, I am more inclined to use the docker image , but I faces quite a lot of challenges and still facing , in setting it up. For starter , the Jupyter notebook does not have nbextensions preinstalled ,and I could not get it installed either , asked a separate question on that . Then the SPARK is very slow .

Havent tried the interative sessions , will start with it soon .