By using AWS re:Post, you agree to the Terms of Use
/Amazon SageMaker Deployment/

Questions tagged with Amazon SageMaker Deployment

Sort by most recent
  • 1
  • 90 / page

Browse through the questions and answers listed below or filter and sort to narrow down your results.

0
answers
0
votes
2
views
asked 9 days ago

How to create (Serverless) SageMaker Endpoint using exiting tensorflow pb (frozen model) file?

Note: I am a senior developer, but am very new to the topic of machine learning. I have two frozen TensorFlow model weight files: `weights_face_v1.0.0.pb` and `weights_plate_v1.0.0.pb`. I also have some python code using Tensorflow 2, that loads the model and handles basic inference. The models detect respectively faces and license plates, and the surrounding code converts an input image to a numpy array, and applies blurring to the images in areas that had detections. I want to get a SageMaker endpoint so that I can run inference on the model. I initially tried using a regular Lambda function (container based), but that is too slow for our use case. A SageMaker endpoint should give us GPU inference, which should be much faster. I am struggling to find out how to do this. From what I can tell reading the documentation and watching some YouTube video's, I need to create my own docker container. As a start, I can use for example `763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.8.0-gpu-py39-cu112-ubuntu20.04-sagemaker`. However, I can't find any solid documentation on how I would implement my other code. How do I send an image to SageMaker? Who tells it to convert the image to numpy array? How does it know the tensor names? How do I install additional requirements? How can I use the detections to apply blurring on the image, and how can I return the result image? Can someone here please point me in the right direction? I searched a lot but can't find any example code or blogs that explain this process. Thank you in advance! Your help is much appreciated.
1
answers
0
votes
2
views
asked 23 days ago

Sagemaker onboarding exceptions for IAM user - CreateDomain error and ValidationException

I am an IAM user. The permission policies I have from the admin are IAMFullAccess, AmazonS3FullAccess, AmazonSageMakerFullAccess, and AmazonEC2FullAccess. When trying to onboard sagemaker, I get the following two exceptions AccessDeniedException User: arn:aws:iam::123456789:user/username is not authorized to perform: sagemaker:CreateDomain on resource: arn:aws:sagemaker:region:123456789:domain/domain because no identity-based policy allows the sagemaker:CreateDomain action and ValidationException Access denied in getting/accepting the portfolio shared by SageMaker. Please call withservicecatalog:AcceptPortfolioShare & servicecatalog:ListAcceptedPortfolioShares permission. The first exception seems to indicate that I have not been given any identity-based policy that allows me to call createdomain on the sagemaker api, but as I listed at the beginning I have been given a full access policy for sagemaker and other services, and I attach the AmazonSageMakerFullAccess policy to the execution role when trying to onboard. Looking at this error online I found a suggestion to add a policy containing kms:CreateGrant and dms:DescribeKey, but it didn't help and looking at the api-permissions-reference (https://docs.aws.amazon.com/sagemaker/latest/dg/api-permissions-reference.html) I only need such things if I specified a customer managed key, which I did not. I found a question on this forum that was related (https://repost.aws/questions/QUyWQfPusnSHG6Ujfzx27o1w/sagemaker-studio-create-domain-error), but the answer seems to have listed permission policies that are needed. These are permissions I should already have in the full access policies. I created a seperate personal account and was able to successfully onboard sagemaker with no issues, so the problem is coming specifically from the IAM account and its permissions.
0
answers
0
votes
3
views
asked a month ago

Sagemaker endpoint endlessly "Creating" after Huggingface fetch error

I attempted to create an endpoint using the huggingface features of the sagemaker package for Python ```python from sagemaker.huggingface import HuggingFaceModel import sagemaker role = sagemaker.get_execution_role() # Hub Model configuration. https://huggingface.co/models hub = { 'HF_MODEL_ID':'MY_ACC/MY_MODEL_NAME', 'HF_TASK':'text-generation' } # create Hugging Face Model Class huggingface_model = HuggingFaceModel( transformers_version='4.6.1', pytorch_version='1.7.1', py_version='py36', env=hub, role=role, ) # deploy model to SageMaker Inference predictor = huggingface_model.deploy( initial_instance_count=1, # number of instances instance_type='ml.m5.xlarge' # ec2 instance type ) predictor.predict({ 'inputs': "Can you please let us know more details about your " }) ``` `MY_ACC/MY_MODEL_NAME` is a private model, so the endpoint creation kept outputting the following errors: ``` This is an experimental beta features, which allows downloading model from the Hugging Face Hub on start up. It loads the model defined in the env var `HF_MODEL_ID` ``` ``` Traceback (most recent call last): File "/usr/local/bin/dockerd-entrypoint.py", line 23, in <module> serving.main() File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/serving.py", line 34, in main _start_mms() File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 49, in wrapped_f return Retrying(*dargs, **dkw).call(f, *args, **kw) File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 206, in call return attempt.get(self._wrap_exception) File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 247, in get six.reraise(self.value[0], self.value[1], self.value[2]) File "/opt/conda/lib/python3.6/site-packages/six.py", line 719, in reraise raise value File "/opt/conda/lib/python3.6/site-packages/retrying.py", line 200, in call attempt = Attempt(fn(*args, **kwargs), attempt_number, False) File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/serving.py", line 30, in _start_mms mms_model_server.start_model_server(handler_service=HANDLER_SERVICE) File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/mms_model_server.py", line 75, in start_model_server use_auth_token=HF_API_TOKEN, File "/opt/conda/lib/python3.6/site-packages/sagemaker_huggingface_inference_toolkit/transformers_utils.py", line 154, in _load_model_from_hub model_info = _api.model_info(repo_id=model_id, revision=revision, token=use_auth_token) File "/opt/conda/lib/python3.6/site-packages/huggingface_hub/hf_api.py", line 155, in model_info r.raise_for_status() File "/opt/conda/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) ``` ``` requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/api/models/MY_ACC/MY_MODEL_NAME ``` It seems like there is a never-ending loop of checking for this resource that it cannot find. I have killed the Python process that started the endpoint creation, but it has carried on regardless. How do I fix this? I just want to delete the endpoint, but that option is greyed-out as it is still in the creation phase. Thanks
1
answers
0
votes
6
views
asked 2 months ago
  • 1
  • 90 / page