1 Risposta
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
1
I see that there are many repetitive blocks of code in your Glue jobs and you would like to extract these pieces of code and implement them as functions. For this purpose, I would suggest you to bundle up these code blocks into python modules and then get them imported into your Glue jobs whenever there is a need.
- Make a python module of your generic functions.
- Next, .zip the module and upload it into a s3 bucket.
- Add the s3 path to your module (.zip file) to the "Python library path" field present in job details section.
To know more about this, please refer this documentation.
In this way, I think you can decrease the number of lines in your Glue jobs by pushing the redundant code to s3 bucket and then fetching it whenever required.
I hope this helps you in your usecase.
Contenuto pertinente
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa