2 Risposte
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
1
For the use case of automatically running ML jobs with on-demand infrastructure, with the ability to accept input parameters, I'd recommend SageMaker Processing as a better fit than Notebook Instances + Lifecycle Configs.
With processing jobs:
- You could still use notebook files if needed, using a tool like Papermill or a more basic pattern like just loading the file and running through the code cells. For example using a FrameworkProcessor, you should be able to upload a bundle of files to S3 (including your notebooks and a plain Python entrypoint to manage running them).
- You could trigger processing jobs from events just like your current notebook start-up, but could provide many different parameters to control what gets executed.
- The history of jobs and their parameters will be automatically tracked through the SageMaker Console - with logs and metrics also available for analysis.
- You wouldn't be limited to the 5 minute time-out of a LCConfig script
If you really needed to stick with the notebook pattern though, modifying the LCConfig each time seems less than ideal... So maybe I'd suggest bringing in another external state you could use to manage some state: For example have your LCConfig script read a parameter from SSM or DynamoDB to tell it which notebook to execute on the current run?
0
How about putting two notebooks in two different notebook instances?
con risposta un anno fa
Contenuto pertinente
- AWS UFFICIALEAggiornata un anno fa
- AWS UFFICIALEAggiornata un anno fa
- AWS UFFICIALEAggiornata un anno fa
- AWS UFFICIALEAggiornata 2 anni fa