How to import python scripts in a preprocessor script

0

Like the individual who posted this question on the AWS forum, I am also looking to use create_monitoring_schedule. In a record_preprocessor_script file, I want to import various dependent Python scripts, also download and use .pkl files and CSVs from S3. Here's the link to the original question for reference: https://repost.aws/ja/questions/QUMXUCX9nPQWK0WdIg7e7nog/in-case-of-defaultmodelmonitor-create-monitoring-schedule-i-need-to-use-record-preprocessor-script-which-actually-imports-few-dependent-py-scripts. In my situation, the inference endpoint uses a custom container image. The data processed by this endpoint undergoes complex preprocessing before the actual inference takes place. As a result, the data that's captured for monitoring also reflects the post-preprocessing state. This means that I need to implement similarly complex preprocessing for model monitoring. I have explored the BYOC (Bring Your Own Container) sample codes. While I am keen on using the default model monitor code for handling data drift and model drift, apart from preprocessing, I attempted to extend the Dockerfile using the sagemaker-model-monitor-analyzer container image to see if implementation could be simplified. Unfortunately, it appears that this pre-built image is not publicly available, so I was unable to implement it. Starting from scratch to develop model and data drift monitoring in BYOC is quite challenging for me (as I have limited understanding in this area). How might I be able to fulfill my requirements?

q
gefragt vor 4 Monaten237 Aufrufe
1 Antwort
0

Hi, given the level of customization required by your use case, looking at the BYOC scenario for SageMaker Model Monitor is a valid option (https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-byoc-containers.html). I also understand that this is more complex to achieve compared to using a pre-built image, but hopefully the following resources can help.

BYOC examples for model monitor https://github.com/aws-samples/detecting-data-drift-in-nlp-using-amazon-sagemaker-custom-model-monitor https://github.com/aws-samples/sagemaker-model-monitor-bring-your-own-container

Library used in the built-in Model Monitor container https://github.com/awslabs/deequ

Hope this helps.

AWS
EXPERTE
beantwortet vor 3 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen