Questions tagged with Machine Learning & AI

I am exploring the Sagemaker Built-in algorithms, and I am curious to learn more about the details of the algorithms. However, I am surprised that it is hard to find any references for the research background and implementation details in the numerous documents and tutorials for particular algorithms. If such information exists somewhere, I would highly appreciate a pointer. Thanks a lot in advance!

Hi Dears, I am building ML model using DeepAR Algorithm. I faced this error while i reached to this point : Error : ClientError: An error occurred (UnknownOperationException) when calling the CreateHyperParameterTuningJob operation: The requested operation is not supported in the called region. ------------------- Code: from sagemaker.tuner import ( IntegerParameter, CategoricalParameter, ContinuousParameter, HyperparameterTuner, ) from sagemaker import image_uris container = image_uris.retrieve(region= 'af-south-1', framework="forecasting-deepar") deepar = sagemaker.estimator.Estimator( container, role, instance_count=1, instance_type="ml.m5.2xlarge", use_spot_instances=True, # use spot instances max_run=1800, # max training time in seconds max_wait=1800, # seconds to wait for spot instance output_path="s3://{}/{}".format(bucket, output_path), sagemaker_session=sess, ) freq = "D" context_length = 300 deepar.set_hyperparameters( time_freq=freq, context_length=str(context_length), prediction_length=str(prediction_length) ) Can you please help in solving the error? I have to do that in af-south-1 region. Thanks Basem hyperparameter_ranges = { "mini_batch_size": IntegerParameter(100, 400), "epochs": IntegerParameter(200, 400), "num_cells": IntegerParameter(30, 100), "likelihood": CategoricalParameter(["negative-binomial", "student-T"]), "learning_rate": ContinuousParameter(0.0001, 0.1), } objective_metric_name = "test:RMSE" tuner = HyperparameterTuner( deepar, objective_metric_name, hyperparameter_ranges, max_jobs=10, strategy="Bayesian", objective_type="Minimize", max_parallel_jobs=10, early_stopping_type="Auto", ) s3_input_train = sagemaker.inputs.TrainingInput( s3_data="s3://{}/{}/train/".format(bucket, prefix), content_type="json" ) s3_input_test = sagemaker.inputs.TrainingInput( s3_data="s3://{}/{}/test/".format(bucket, prefix), content_type="json" ) tuner.fit({"train": s3_input_train, "test": s3_input_test}, include_cls_metadata=False) tuner.wait()

I recently tried the Polly console set to Neural and SSML and keyed in some text and tried several voices with the Whisper action and it works for the Mathew voice but does not work for any other one. If its set to Normal instead of Neural the whisper does work for other voices but does not sound good. Is there a reason the whisper only works for that one voice using the Neural setting?

Hello, I tried using [Inf1 EC2 instance](https://aws.amazon.com/ec2/instance-types/inf1/) for deploying my ML model. I need to monitor the GPU usage of the ML model. I could find the CPU usage in the aws console, but not gpu usage. Already tried: 1. https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-gpu-monitoring-gpumon.html This didn't work. It threw this error ``` (python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ python3 gpumon.py Traceback (most recent call last): File "gpumon.py", line 146, in <module> nvmlInit() File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1450, in nvmlInit nvmlInitWithFlags(0) File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1440, in nvmlInitWithFlags _nvmlCheckReturn(ret) File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 765, in _nvmlCheckReturn raise NVMLError(ret) pynvml.nvml.NVMLError_DriverNotLoaded: Driver Not Loaded ``` Also, `nvidia-smi` didn't work ``` (python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ nvidia-smi NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running. ``` Kindly provide some help to monitor GPU usage.

Hi all, I've followed the guide provided in this medium blog by senior AWS ML SA: https://towardsdatascience.com/hosting-vs-code-in-sagemaker-studio-f211385e25f7 I've created the lifecycle configuration (jupyterlab app) for the following bash script: ``` #!/bin/bash mkdir -p vscode cd vscode sudo su curl -fsSL https://code-server.dev/install.sh | sh exit ``` However, when I attach the configuration to the domain or a specific user's jupyter server in the SM Studio, the JupyterServer App fails to be created with the following message displayed on the loading page of SM Studio's jupyterlab: > "The JupyterServer app default encountered a problem and was stopped.Details: ConfigurationError: LifecycleConfig execution failed with non zero exit code 1 for script arn:aws:sagemaker:eu-west-3:615740825886:studio-lifecycle-config/install-vscode-on-jupyterserver. Check https://docs.aws.amazon.com/sagemaker/latest/dg/studio-lcc-debug.html for debugging instructions." I checked the page for debugging instructions, specifically the part where I am instructed to check the cloudwatch logs for the log group: aws/sagemaker/studio The problem is this log group does not exist in CloudWatch (I have admin permission). I would like to have this set up for the entire team, Any help would be greatly appreciated! Best, Ruoy

Overview: I'm trying to run an XGboost model on a bunch of parquet files sitting in S3 using dask by setting up a fargate cluster and connecting it to a Dask cluster. Total dataframe size totals to about 140 GB of data. I scaled up a fargate cluster with properties: Workers: 40 Total threads: 160 Total memory: 1 TB So there should be enough data to hold the data tasks. Each worker has 9+ GB with 4 Threads. I do some very basic preprocessing and then I create a DaskDMatrix which does cause the task bytes per worker to get a little high, but never above the threshold where it would fail. Next I run xgb.dask.train which utilizes the xgboost package not the dask_ml.xgboost package. Very quickly, the workers die and I get the error `XGBoostError: rabit/internal/utils.h:90: Allreduce failed`. When I attempted this with a single file with only 17MB of data, I would still get this error but only a couple workers die. Does anyone know why this happens since I have double the memory of the dataframe? ``` X_train = X_train.to_dask_array() X_test = X_test.to_dask_array() y_train = y_train y_test = y_test ``` dtrain = xgb.dask.DaskDMatrix(client,X_train, y_train) output = xgb.dask.train( client, {"verbosity": 1, "tree_method": "hist", "objective": "reg:squarederror"}, dtrain, num_boost_round=100, evals=[(dtrain, "train")])`

Hello, i need some help regarding my question title. i am working on a project that needs to get a video stored on a website which i have created to work with my machine learning code which is running on pycharm it all seems to be working locally and schedulely but i want it to run it on cloud what type of instance do i need because the model i am using takes a lot of time on cpu and gpu based instance will certainly help. if you need more details regarding this let me know thanks.

> The problem is related to registering a **AWS DeepLens v1.1 Camera** I am failing to register a AWS Deeplens camera using machines running Ubuntu 20.4 (focal) and Mint 19.3 (tessa) or ubuntu bionic, with the error: **Device not detected**. This is when trying to follow the *Register an AWS DeepLens v1.1 device * steps on the AWS web console. Trying to go via other forums, I am made to believe that there is a need to have the **awscam** package installed on the linux machines. I ama failing to install the package as well, with the error below. Is the a awscam package for debian distributions, and more precisely for Ubuntu 20.4? **sudo apt-get install awscam** ``` sudo apt-get install awscam Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package awscam ``` **lsb_release -a** ``` lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.4 LTS Release: 20.04 Codename: focal ```

Any expert that is familiar with the AI topics for QA Automation and Rendering optimization?

Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning (ML) from weeks to minutes in Amazon SageMaker Studio, the first fully integrated development environment (IDE) for ML. With SageMaker Data Wrangler, you can simplify the process of data preparation and feature engineering, and complete each step of the data preparation workflow, including data selection, cleansing, exploration, and visualization, from a single visual interface. SageMaker Data Wrangler runs on ml.m5.4xlarge by default. SageMaker Data Wrangler includes built-in data transforms and analyses written in PySpark so you can process large data sets (up to hundreds of gigabytes (GB) of data) efficiently on the default instance. Starting today, you can use additional M5 or R5 instance types with more CPU or memory in SageMaker Data Wrangler to improve performance for your data preparation workloads. Amazon EC2 M5 instances offer a balance of compute, memory, and networking resources for a broad range of workloads. Amazon EC2 R5 instances are the memory optimized instances. Both M5 and R5 instance types are well suited for CPU and memory intensive applications such as running built-in transforms for very large data sets (up to terabytes (TB) of data) or applying custom transforms written in Panda on medium data sets (up to tens of GBs). To learn more about the newly supported instances with Amazon SageMaker Data Wrangler, visit the [blog ](https://aws.amazon.com/blogs/machine-learning/process-larger-and-wider-datasets-with-amazon-sagemaker-data-wrangler/) or the [AWS document](https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler-data-flow.html), and the[ pricing page](https://aws.amazon.com/sagemaker/pricing/). To get started with SageMaker Data Wrangler, visit the [AWS documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler.html).

I have a scanned PDF with handwritten text and I am trying to search through it using AWS Kendra. However, Kendra doesn't search through the handwritten text. I tried using textract and comprehend, but there seem to be limitations to PDF size and number of pages with both services. Is there a way to allow Kendra to search through this handwritten text?

How can you use the functions of SageMaker Canvas with python or outside the user interface?

We are working on Automation of CCTV camera videos (RTSP link) using image processing techniques such as back ground subtraction, contour detection, annotation using opencv-python for 80 sites.so we have to run 80 python scripts in parallel manner. I would like to know which AMI suits for this project without any drop in outputs. The outputs are images in jpg format. These code would run 24*7.kindly suggest best AMI for this so that we would suggest the same to our management for purchase.

After create model in Redshift i can load the generated model in python via Sagemaker. The features are generated automatically e.g. f0, f1 and so on. Altough i have only 10 columns i get maybe 500 features. This comes from some categories but where can i see the mapping of the features to the original category names? For example f100 is the category "XY" of column 5.

I run neural network tensorflow train on studiolab. and I got: ``` Epoch 145/4000 1941/1941 - 10s - ... - 10s/epoch - 5ms/step ``` then I try to make a train job with script_mode with `ml.c5.xlarge` ``` estimator = TensorFlow(entry_point='untitled.py', source_dir='./training/', instance_type='ml.c5.xlarge', instance_count=1, output_path="s3://sagemaker-[skip]", role=sagemaker.get_execution_role(), framework_version='2.8.0', py_version='py39', hyperparameters={...}, metric_definitions=[...], script_mode=True) ``` and its got: ``` Epoch 19/4000 1941/1941 - 49s - ... - 49s/epoch - 25ms/step ``` Why is it 5 times slower than studiolab notebook? Is it because instance type?

After creating and finishing the model training in Redshift we can predict with the new model. How can we see the target function of the model or feature importance of the single columns which were used as an input? To describe it in another way we want to see the correlations to the target column after the model training.

Hi everyone! Currently, I've accomplished detecting all the faces from a collection and then generating sub-galleries of each individual subject with all their photos associated with the ruby SDK '~> 1.65' In order to do this, I've indexed the faces of all photos within a collection, list all the faces (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#list_faces-instance_method), then grabbing each face_id recognized and search the faces related to that face_id (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#search_faces-instance_method), and delete the face id used to do the API call and all the returned ones to tell when a new detected subject starts and ends. My issue is that some indexed faces are returning a wrong bound box not fitting the face properly with a few images, sometimes not even close to the face (I've seen a brick as a bound box). This is weird because internally those "wrong faces" seem to be associated properly with other faces as if the service were just returning wrong box values but doing all the AI jobs properly. Is there a way to prevent or tweak the call params to prevent this? or is it normal for the API to return sometimes wrong values due to the multiple decimals? Thanks in advance!

Hi everyone! Currently, I've accomplished detecting all the faces from a collection and then generating sub-galleries of each subject with all their photos associated with the ruby SDK '~> 1.65' To do this, I've indexed the faces of all photos within a collection, list all the faces (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#list_faces-instance_method), then grabbing each face_id recognized and search the faces related to that face_id (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#search_faces-instance_method), and delete the face id used to do the API call and all the returned ones to tell when a new detected subject starts and ends. My issue is that the search faces API returns different results depending on which face id param you are doing the request with. For example, if there are 10 faces ids detected that belong to a person (1, 2, 3, ..., 10), the search faces call with face id = 1 param, should return the faces id (2, 3, 4, ..., 10) but if you continue to do this with the other face ids this is not always the case with some scenarios where the search faces call with face id = 3 has returned a subset of the previously mentioned like just (4, 5, 6). Is there any other way to achieve this to prevent this kind of "error"? if not, this is a real concern for us because it depends on the order in which we call the search faces with different face ids, and sometimes it seems like there is more than 1 subject detected with almost the same photos when in reality it's the same person. Thanks in advance!

I need to extract text from some .PDFs documents. The formatting of these texts varies (column where it starts and ends) but the beginning and end are always similar, it starts with the word **DECRETO** and ends with the word **CHEFE + something** In this link you can see an example of the original text: https://i.postimg.cc/Xvk0TXJ9/texto.png Is it possible to do this with AWS tools? What's the best way? **OBS: Text language is PT-BR**

I have a private GitLab and would like to connect it to Sagemaker Studio and/or Notebooks. Is that supported? What is it is through a VPC? If so, how can it be done?

Note: I am a senior developer, but am very new to the topic of machine learning. I have two frozen TensorFlow model weight files: `weights_face_v1.0.0.pb` and `weights_plate_v1.0.0.pb`. I also have some python code using Tensorflow 2, that loads the model and handles basic inference. The models detect respectively faces and license plates, and the surrounding code converts an input image to a numpy array, and applies blurring to the images in areas that had detections. I want to get a SageMaker endpoint so that I can run inference on the model. I initially tried using a regular Lambda function (container based), but that is too slow for our use case. A SageMaker endpoint should give us GPU inference, which should be much faster. I am struggling to find out how to do this. From what I can tell reading the documentation and watching some YouTube video's, I need to create my own docker container. As a start, I can use for example `763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.8.0-gpu-py39-cu112-ubuntu20.04-sagemaker`. However, I can't find any solid documentation on how I would implement my other code. How do I send an image to SageMaker? Who tells it to convert the image to numpy array? How does it know the tensor names? How do I install additional requirements? How can I use the detections to apply blurring on the image, and how can I return the result image? Can someone here please point me in the right direction? I searched a lot but can't find any example code or blogs that explain this process. Thank you in advance! Your help is much appreciated.

As per standard entity recognition from AWS Comprehend, it recognizes all currency, numbers, duration as QUANTITY - Is there anyway to get category of the quantity which AWS Comprehend is recognizing?

Hello, I have submitted an entry to Deep Racer April Open league from a temporary AWS account which has been deleted/removed. How I get the entry (or the model) moved/ported to my new/permanent AWS account. I cannot access the temporary AWS account anymore nor I have saved my model to submit again from new/permanent account. Please help. Regards, Raghavendra

We have several desktop (Win/Mac/Nix) monolithic applications that generate JSON events exposing these events via POST. Currently it is a one way deal, outbound from app to the stream for nothing but analytics. We are now looking to analyze those events to determine if there are actionable "data sequences" that start to appear. **My Question is (and opinions are welcomed...)** **What is the most efficient (#1 - performant, #2 - cost effective, #3 - resilient) way to stream those records (250-500 byte msg, at a peak of 1,000 / second) to AWS for the purpose of:** 1. Transforming the record data to a known model (i.e. std date format, simple XForms) 2. Analyzing the normalized record (ML - predictive analytics or iterative learning) 3. Persisting the record 4. Based on the analysis, sending back a payload (or pointer to a CDN stored payload) to the original calling app? In a nutshell looking to do a type of on-preises, *near real-time* application performance analytics of the applications using a single on-premises service spewing JSON out to AWS native cloud services that do all teh work, and return info as timely as possible. Thanks. <><

Hello We are using Greengrass v2 with the Sagemaker component in order to do inference on the edge. The following error is seen intermettently while running our application. Could you please advise of anything we can do to avoid it? File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 946, in __call__ return _end_unary_response_blocking(state, call, False, None) File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "Socket closed" debug_error_string = "{"created":"@1650459124.460491853","description":"Error received from peer unix:/tmp/aws.greengrass.SageMakerEdgeManager.sock","file":"src/core/lib/surface/call.cc","file_line":906,"grpc_message":"Socket closed","grpc_status":14}"

Hello, When using Greengrass v2 with Sagemaker component the intermittent error below is seen when the device powers off and on. Could you please advise what could be the possible errors? eturn self.client.ListModels(agent_pb2.ListModelsRequest()) File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 923, in call return _end_unary_response_blocking(state, call, False, None) File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 826, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses" debug_error_string = "{"created":"@1650457997.824138381","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":4143,"referenced_errors":[{"created":"@1650457988.031111729","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":398,"grpc_status":14}]}"

When I run a training job from SageMaker with the XGBoost algorithm, I get the error: Traceback (most recent call last): File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_trainer.py", line 84, in train entrypoint() File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/training.py", line 94, in main train(framework.training_env()) File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/training.py", line 90, in train run_algorithm_mode() File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/training.py", line 68, in run_algorithm_mode checkpoint_config=checkpoint_config File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/algorithm_mode/train.py", line 121, in sagemaker_train validated_train_config = hyperparameters.validate(train_config) File "/miniconda3/lib/python3.7/site-packages/sagemaker_algorithm_toolkit/hyperparameter_validation.py", line 280, in validate raise exc.UserError("Extraneous hyperparameter found: {}".format(hp)) sagemaker_algorithm_toolkit.exceptions.UserError: Extraneous hyperparameter found: prob_buffer_row Extraneous hyperparameter found: prob_buffer_row I cannot delete the prob_buffer_row parameter as it comes by default when defining the algorithm. Am I doing something wrong or is it a bug?

Since, the March 2022 release update of AWS Textract (you can find the announcement here: https://aws.amazon.com/about-aws/whats-new/2021/04/amazon-textract-announces-quality-update-table-extraction-feature/), the AnalyzeDocument API has shown an unexpected behavior. I have been trying to process a single page pdf document through textract, which contains just one table of 40 rows and 25 columns. In the output, I happen to see that textract is breaking the table at the 4th column , dividing it into 2 tables (a left table with 3 columns, and a right table with remaining columns). This break in between is causing my accuracy to disrupt in the output. Surprisingly, this behavior was not observed before the latest release. I can say this, as I hold the outputs from the last release as well, in that the output was one single table like that of the source document. Can someone from the community help me on this, are there some extra hyper parameter which I have to use while calling textract, to get past this break which is happening. Thanks in advance.

Using IoT Analytics but cannot find button to empty the contents. Can only delete the entire data store.

what other data input types can be used pipe input mode in sagemaker? an example of implementation is here https://aws.amazon.com/blogs/aws/sagemaker-nysummit2018/, and can this be used for inference as well as, similar to training?

looking at few examples, for training in sagemaker . are there some guidelines based on the model size, data to be trained , what type of instance cpu/gpu to use? also, can one use spot instances ( may be with multiple gpu cores)?

We are developing a custom model monitoring container to be used to interact with Sagemaker's model monitoring API, as we require additional custom metrics. One of our requirements is for these metrics to populate and visualize appropriately in Studio visualization tabs, e.g. Data Quality, Model Quality, and Explainability. **None of the built-in containers are an option for us, and we are trying to interop with the Monitoring APIs as seamlessly as possible.** The docs specify the container contracts for output, specifically *statistics.json*, *constraints.json*, and *constraint_violations.json*. However, the docs are not clear on what JSON files to emit for specifically *custom Model Quality metrics*. There does not appear to be a provided schema. Things I have tried: - Emitting CloudWatch metrics to a different namespace, *sagemaker/Endpoint/model-quality-metrics* - Attempting to retrofit the statistics.json file to also include Model Quality metrics. Is there a separate JSON file I should write to */opt/ml/processing/resultdata* for our custom model quality metrics to populate the Studio visualization tab? Specifically, where do the base containers look for this JSON report: https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality-metrics.html

Does Data Capture feature used for model monitor and analytics work with the multi model endpoint (one container).. we ran into an error. See error " An error occurred (ValidationException) when calling the CreateEndPointConfig operation: Data Capture Feature is not supported with MultiModel mode" Theoretically, it should work because it is calling the DataCaptureConfig: from sagemaker.model_monitor import DataCaptureConfig endpoint_name = 'your-pred-model-monitor-' + strftime("%Y-%m-%d-%H-%M-%S", gmtime()) print("EndpointName={}".format(endpoint_name)) data_capture_config=DataCaptureConfig( enable_capture = True, sampling_percentage=100, destination_s3_uri=s3_capture_upload_path)

Can the Comprehend model return a full text from a context phrase that I search for? For example, suppose this text below is an area in a newspaper (in .PDF format) that contains several subjects and that talks about Weddings in a specific area containing title and body text: *Wedding 2022* *We had a very traditional wedding and it was extremely expensive, but it was worth it. Carol and I only paid half. Her parents paid for everything else. We got married in church. Carol wore a white dress and she looked fantastic. I wore a suit and I think I looked quite good too! We had a big reception. We had 200 guests. The reception was in a wonderful hotel. We took lots of pictures. It was just great!* If I send the model “Wedding 2022” and “We got married in church” will it be able to find this text among different themes and will I be able to receive all this text below? *Wedding 2022* *We had a very traditional wedding and it was extremely expensive, but it was worth it. Carol and I only paid half. Her parents paid for everything else. We got married in church. Carol wore a white dress and she looked fantastic. I wore a suit and I think I looked quite good too! We had a big reception. We had 200 guests. The reception was in a wonderful hotel. We took lots of pictures. It was just great!* Is Comprehend the best tool to try to solve this problem?

I am testing inference in sagemaker , by using one of the container listed here -> https://github.com/aws/deep-learning-containers/blob/master/available_images.md. the model is zipped up as below and with in inference.py file , i am overwriting functions like model_fn method and predict_fn. I tested this with batch transform and it worked but for few small input files but for other larger files, i keep getting "Model server did not respond to /invocations request within 3600 seconds" . I'm trying to find out what is the cause of it? 3600 is the max we can set for "invocation timeout in seconds" parameter and the default input size for batch is 6mb , the input files i'm using are way smaller than that but i still get that error. Directory structure ``` model.tar.gz/ |- model.pth |- code/ |- inference.py |- requirements.txt ``` file : inference.py ``` import torch import os def model_fn(model_dir): model = Your_Model() with open(os.path.join(model_dir, 'model.pth'), 'rb') as f: model.load_state_dict(torch.load(f)) return model def predict_fn(): // ``` based on docs here, https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-batch-code.html#your-algorithms-batch-code-how-containers-should-respond-to-inferences, do we need to install flask and have an /invocations endpoint , that responds 200 ok , when we are using custom container?

After created a Batch inference job to get Batch Recommendation I cannot figure it out how to delivery recommendation to my application (React Native) from file stored on S3 Bucket.

We ran into the following error when trying to call SageMaker SKlearn Container using SklearnProcess run(). Also the current boto3 version is 1.20.23 ( there is a github post indicated an issue with one boto3 release 1.16 with timeout issue). Looks like a missing permission for STS service? Any other suggestions? - /opt/conda/lib/python3.7/site-packages/botocore/httpsession.py in send(self, request) 416 raise ProxyConnectionError(proxy_url=proxy_url, error=e) 417 except URLLib3ConnectTimeoutError as e: --> 418 raise ConnectTimeoutError(endpoint_url=request.url, error=e) 419 except URLLib3ReadTimeoutError as e: 420 raise ReadTimeoutError(endpoint_url=request.url, error=e) ConnectTimeoutError: Connect timeout on endpoint URL: "https://sts.us-xxxx-x.amazonaws.com/"

I'm in the process of recognizing the Textract for German invoices. The result was very bad. out of 20 invoices, none of the fields were recognized. Is there a way to training data?

AWS' feature store pages describe feature groups and how to store features and their types (https://docs.aws.amazon.com/sagemaker/latest/dg/feature-store-introduction-notebook.html) . However, it is not clear how to store feature transforms if needed. Is this possible with AWS' feature store?

example provided in the aws documentation , https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html, see sample input csv can be structured like a sample below. is there a link, to see the actual csv file that is used in an working example , instead of the sample posted in the docs. it is also mentioned that each record is one per line and no end of line character is allowed. what if the input type is other than csv, like json? do the same rules apply to json files as well, one record per line, or can one record be spread across multiple lines ,what about new line character in that case? ``` Record1-Attribute1, Record1-Attribute2, Record1-Attribute3, ..., Record1-AttributeM Record2-Attribute1, Record2-Attribute2, Record2-Attribute3, ..., Record2-AttributeM ... ... ```

based on the documentation , https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html, it states that " The ideal value for MaxConcurrentTransforms is equal to the number of compute workers in the batch transform job." how to figure out what the number of compute workers is , i assume this depends on the instance type. also what about the instance count parameter we can set , do we have to take that into account as well?

Hi, there When training model using sagemaker and calling panda.read_csv().. we are getting the above error. It appears that this is a known issue and the workaround is to upgrade aiobotocore to the latest 2.20 release. However, the missing link is where to upgrade the aiobotocore .. since we are running model against SageMaker and its fully managed M5 instance... instead of an EC2 instance. Any thoughts? Error message in full: /opt/conda/lib/python3.7/site-packages/aiobotocore/client.py in create_client(self, service_name, region_name, is_secure, endpoint_url, verify, credentials, scoped_config, api_version, client_config) 43 service_client, endpoint_url, client_config 44 ) ---> 45 self._register_lazy_block_unknown_fips_pseudo_regions(service_client) 46 return service_client 47 AttributeError: 'AioClientCreator' object has no attribute '_register_lazy_block_unknown_fips_pseudo_regions' See notes below under this link: https://github.com/boto/botocore/pull/2558/files#

Hi all, Just wondering if there is ever a need to re-index photos added to a Rekognition collection? My company is used to working with other 3rd party facial recognition vendors and any time a new algorithm version is released, we need to re-enroll/re-index the existing photos in order to use it. Obviously when you need to pay a fee for each indexing API call, this would amount to a lot of extra costs if so. Also, how does AWS Rekognition handle algorithm upgrades? Is it automatic/transparent or do you have to approve the use of new algorithm changes? Thanks!

Hi, New to Rekognition and just had some resiliency related questions that I'm struggling to find the answers for. 1. How are the underlying photos in a collection stored? Does AWS store these in an S3 bucket? If so, is there any need to store the source photos in your own private S3 bucket or is simply having them in a collection sufficient? 2. How are Rekognition collections backed up? Thanks!

does cloudformation support sagemaker batch transform? if yes, can the jobs be triggered/run automatically once the stack is created?

the order flowers blueprint runs fine on lex through console but when the lambda blueprint is attached the test fails with "Invalid Lambda Response: Received error response from Lambda: Unhandled" the log shows the error: [ERROR] KeyError: 'userId' Traceback (most recent call last): File "/var/task/lambda_function.py", line 206, in lambda_handler return dispatch(event) File "/var/task/lambda_function.py", line 182, in dispatch logger.debug('dispatch userId= {} , intentName= {} '.format(intent_request['userId'], intent_request['currentIntent']['name'])) [ERROR] KeyError: 'userId' Traceback (most recent call last): File "/var/task/lambda_function.py", line 206, in lambda_handler return dispatch(event) File "/var/task/lambda_function.py", line 182, in dispatch logger.debug('dispatch userId={}, intentName={}'.format(intent_request['userId'], intent_request['currentIntent']['name'])) the reuse of the function for fulfillment and validation is very hard to follow because the doc says: In the Editor, choose AWS Lambda function as Fulfillment, and select the Lambda function that you created in the preceding step (OrderFlowersCodeHook). Choose OK to give Amazon Lex permission to invoke the Lambda function. ..... the problem is there is no choice for the lambda function in that form nor is there one in the validation form!!! there is a reference to the function being stored in an alias but i cant find a way to tie this together. i have really tried to follow the documentation systematically but cant get through it. where am i going off the rails??? shouldnt the blueprint run without this stumbling block? this has cost me a month of hair pulling agony and is really the first step in building an index for my knowledge system already running on aws where a session could build a compound index such as "Application:Billing\Function:Encounters\Process:Create\Plan:BlueCross\Product:PPO\Coverage:Medical\Service:Physical\FromDate:01-01-2022\" it seems like a natural way to specify types and values in a modern interface. Thanks, AEH

configuring a batch job for inference in sagemaker ( sample code below) . for my use case, there are multiple input files and i'm trying to configure such that it can process files one by one or if the instance type allows process them in parallel. if value of max concurrrent transforms is greater than 1, does it processes file in parallel. as things are predefined for the job , before it starts , for exampel the instance_count is already set to 1. how does it do horizontal scaling? can it add add more instances. does the value of instance type or instance count dictate what value we can configure for max concurrent transforms parameter? ``` from sagemaker.transformer import Transformer transformer = Transformer(model_name='my-previously-trained-model', max_concurrent_transforms=0 instance_count=1, instance_type='ml.m4.xlarge') ```

We'd like to use SKLearn.Preprocessing.OrdinalEncoder. However, it seems that the AWS SKLearn package version SageMaker is one that older than the SKLearn release 1.0.2 that have this encoding available. Any recommendations how to work around it?

based on the documentation provided here , https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html, large dataset can be structured as shown below in a csv file. is it possible to have multiple files in this format for batch inference, is there any configuration that can be set , for it to process multiple files. Also, what other formats , beside csv can the batch inference handle? ``` Record1-Attribute1, Record1-Attribute2, Record1-Attribute3, ..., Record1-AttributeM Record2-Attribute1, Record2-Attribute2, Record2-Attribute3, ..., Record2-AttributeM ... ... RecordN-Attribute1, RecordN-Attribute2, RecordN-Attribute3, ..., RecordN-AttributeM ```

Hi, I keep getting this error: AttributeError: module 'sagemaker' has no attribute 'create_transform_job', when using a batch transform example that AWS graciously had in the notebook instances. Code: ***Also, I updated Sagemaker to the newest package and its still not working. ``` %%time import time from time import gmtime, strftime batch_job_name = "Batch-Transform-" + strftime("%Y-%m-%d-%H-%M-%S", gmtime()) input_location = "s3://{}/{}/batch/{}".format( bucket, prefix, batch_file ) # use input data without ID column output_location = "s3://{}/{}/output/{}".format(bucket, prefix, batch_job_name) request = { "TransformJobName": batch_job_name, "ModelName": 'xgboost-parquet-example-training-2022-03-28-16-02-31-model', "TransformOutput": { "S3OutputPath": output_location, "Accept": "text/csv", "AssembleWith": "Line", }, "TransformInput": { "DataSource": {"S3DataSource": {"S3DataType": "S3Prefix", "S3Uri": input_location}}, "ContentType": "text/csv", "SplitType": "Line", "CompressionType": "None", }, "TransformResources": {"InstanceType": "ml.m4.xlarge", "InstanceCount": 1}, } sagemaker.create_transform_job(**request) print("Created Transform job with name: ", batch_job_name) # Wait until the job finishes try: sagemaker.get_waiter("transform_job_completed_or_stopped").wait(TransformJobName=batch_job_name) finally: response = sagemaker.describe_transform_job(TransformJobName=batch_job_name) status = response["TransformJobStatus"] print("Transform job ended with status: " + status) if status == "Failed": message = response["FailureReason"] print("Transform failed with the following error: {}".format(message)) raise Exception("Transform job failed") ``` Everything else is working well. I've had no luck with this on anyother forum.

Are there any plans to have AWS Rekognition tested by NIST? That's a requirement when selecting facial recognition algorithms for many contracts in my company. Thanks!

I am using the sagemaker python SDK to train a bespoke model. I have defined my `metric_definition` regexes and passed them to the estimator like: ```python num_re = "([0-9\\.]+)(e-?[[01][0-9])?" metrics = [ {"Name": "learning-rate", "Regex": f"lr: {num_re}"}, {"Name": "training:loss", "Regex": f"loss: {num_re}"}, # ... ] estimator = Estimator( image_uri=training_image_uri, # ... metric_definitions=metrics, enable_sagemaker_metrics=True, ) ``` When I run training, these metrics are visible in my logs and I can also see them in SageMaker Studio in `Trial Components > Metrics (tab)` as a grid of numbers like: > Name | Minimum | Maximum | Standard Deviation | Average | Count | Final value > learning-rate | 8.889 | 8.907 | 0.010392304845413657 | 8.898 | 4 |8.907 > ... Which suggests that the regexes are correctly matching on the logs However, I am not able to visualise any graphs for my metrics. I have tried all of: - `Sagemaker Studio > Trial components > charts`. It is only possible to plot things like `learning-rate_min` (i.e. a point value not a time-series metric) - `SageMaker aws console > training > training jobs > <select job> > Scroll to Monitor section`. Here I can see metrics like CPUUtilization over time but for my metrics there is just an empty graph for each metric that I have defined that says 'No data available' - `SageMaker aws console > training > training jobs > <select job> > Scroll to Monitor section > View algorithm metrics (opens in CloudWatch) > Browse > select metric (e.g. learning-rate and 'Add to Graph' `. I filter by the correct time period and go the `Graphed metrics (1) tab`, even after updating the period to `1 second` I am not able to see anything on the graph. I'm not sure what the issue is here but any help would be much appreciated

Hi all, If I've previously indexed a collection of say 100 million photos using the IndexFaces API and I want to run a one to many search against that entire collection of indexed photos for every new applicant that comes in to my system, would I just need to run a single SearchFaces API call? Or would I need to run 100 million SearchFaces or CompareFaces API calls? I'm just trying to estimate the pricing on using Rekognition in a facial recognition system and this answer obviously plays a massive factor in the pricing. Thanks!

based on documentation provided here -> https://docs.aws.amazon.com/cli/latest/reference/sagemaker-runtime/invoke-endpoint-async.html I am passing a custom attributes parameter when calling the invoke-endpoint-async function. ``` invoke-endpoint-async --endpoint-name <value> [--custom-attributes <value>] [--inference-id <value>] --input-location <value> ``` are there any sample/example on how can i read this in my code before/after invoking my model for inference. I am creating a preprocessing file and have input_fn and predict_fn function, can value passed in custom_attributes during api call be read or written to the response here? also, if it can be written to the response like documentation says, where can i see it . as the async endpoint only gives the output location when invoked and later process the request, i don't see the response, i just see the dumped out file in the specified output location. how can i see the full response? ``` sagemaker_model = TensorFlowModel(model_data = 'model_data_path', role = execution_role, framework_version = '1.12', source_dir ='src', entry_point = 'preprocessing.py', ...) ``` file: preprocessing.py ``` def input_fn(request_body, content_type): //read the custom attribute here ```

I was testing AWS Personalize recommenders and I realize that I was using all which charge for real-time recommendations. Is there a way to use them not for charge on real-time?

Is there a recommended Docker image / Dockerfile starting point for running R scripts in AWS Batch? I have looked at a few here, and the starting image is could be any of the following: `rocker`, `python:3.6`, `ubuntu:16.04` or `r-base`: - (FROM rocker) https://github.com/aws-samples/aws-fargate-with-rstudio-open-source/blob/main/docimage-rstudio/Dockerfile - (FROM rocker) https://github.com/aws-samples/aws-fargate-with-rstudio-open-source/blob/main/docimage-shiny/Dockerfile - (FROM python:3.6) https://github.com/aws-samples/sagemaker-studio-custom-image-samples/blob/main/examples/r-image/Dockerfile - (FROM ubuntu:16.04) https://github.com/aws/amazon-sagemaker-examples/blob/main/hyperparameter_tuning/r_bring_your_own/Dockerfile - (FROM r-base) https://github.com/aws-samples/reinvent2020-aim404-productionize-r-using-amazon-sagemaker/blob/main/Dockerfile What are the key points to consider in choosing a good starting point for a Dockerfile for R jobs? For this particular workload, my main R packages are `data.table` and the tidyverse packages (no need for RStudio / Shiny server).

Hi. I have a dataset of price data. It looks like this | Price| Branch| ItemCode| Discount| DateTimeOfPrice| | --- | --- | --- | --- | --- | | 10| 002| 52345436| 0.33| 2022-03-24 14:00| The dataset has about 1M records I feature engineered it in the following way | Price| Discount| ItemCode| Year| Month| Day| Hour| Branch1| Branch2| Branch3| | --- | --- | --- | --- | --- |--- | --- | --- | --- | --- | | 10 | 0.33| 52345436| 2022 | 03 |24 | 14 | 0 | 1 | 0 | Each component of the DateTimeOfPrice got a separate column We have 3 branches. To avoid the situation when algorithm may think that "branch" column is some kind of priority column, I created 3 new column (we have 3 branches). If the item belongs to branch2, the column will get the value 1, if not - it will be 0 I run Linear Learner, XGBoost build-in algorithms and also SageMaker AutoPilot. In all cases I run , the best RMSE was 60 and prediction/ validation gives sometimes a result which is far from the actual value. I tried also to run XGBoost from the notebook with the following parameters ``` hyperparams = { "max_depth": "7", "eta": "0.2", "gamma": "4", "min_child_weight": "6", "subsample": "0.7", "objective": "reg:squarederror", "num_round": "100", "eval_metric":"rmse", "verbosity": "2", } ``` Still, the RMSE is arround 60. Please advice what can be done to improve the mertic and predication

We plan to provide a machine learning algorithm via a [container image](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-byoc-containers.html) and are concerned about. Is it possible that other parties download the docker image for local inspection?

I am using the batch transform function in SageMaker for the inference of my PyTorch model. I am using the same structure as https://github.com/aws/amazon-sagemaker-examples/tree/main/advanced_functionality/scikit_bring_your_own/container. The error is that my job will start multiple times on different workers if I choose multiple workers. Or it will repeat after finish if I choose 1 worker. I think it should be some errors in **timeout** setup. I have tried to increase the **keepalive_timeout** and **proxy_read_timeout** in the serve file and tried the **SAGEMAKER_MODEL_SERVER_TIMEOUT** as an environment variable. But nothing worked. Could some one help? Thanks!

I'm creating a pipeline with multiple steps One to preprocess a dataset and the other one takes the preprocessed one as an input to train a BlazingText model for classification My first `ProcessingStep` outputs augmented manifest files step_process = ProcessingStep( name="Nab3Process", processor=sklearn_processor, inputs=[ ProcessingInput(source=raw_input_data, destination=raw_dir), ProcessingInput(source=categories_input_data, destination=categories_dir) ], outputs=[ ProcessingOutput(output_name="train", source=train_dir), ProcessingOutput(output_name="validation", source=validation_dir), ProcessingOutput(output_name="test", source=test_dir), ProcessingOutput(output_name="mlb_train", source=mlb_data_train_dir), ProcessingOutput(output_name="mlb_validation", source=mlb_data_validation_dir), ProcessingOutput(output_name="mlb_test", source=mlb_data_test_dir), ProcessingOutput(output_name="le_vectorizer", source=le_vectorizer_dir), ProcessingOutput(output_name="mlb_vectorizer", source=mlb_vectorizer_dir) ], code=preprocessing_dir) But I'm having a hard time when I try to feed my `train` output as a `TrainingInput` to the model step to use it to train. step_train = TrainingStep( name="Nab3Train", estimator=bt_train, inputs={ "train": TrainingInput( step_process.properties.ProcessingOutputConfig.Outputs[ "train" ].S3Output.S3Uri, distribution="FullyReplicated", content_type="application/x-recordio", s3_data_type='AugmentedManifestFile', attribute_names=['source', 'label'], input_mode='Pipe', record_wrapping='RecordIO' ), "validation": TrainingInput( step_process.properties.ProcessingOutputConfig.Outputs[ "validation" ].S3Output.S3Uri, distribution="FullyReplicated", content_type='application/x-recordio', s3_data_type='AugmentedManifestFile', attribute_names=['source', 'label'], input_mode='Pipe', record_wrapping='RecordIO' ) }) And I'm getting the following error 'FailureReason': 'ClientError: Could not download manifest file with S3 URL "s3://sagemaker-us-east-1-xxxxxxxxxx/Nab3Process-xxxxxxxxxx/output/train". Please ensure that the bucket exists in the selected region (us-east-1), that the manifest file exists at that S3 URL, and that the role "arn:aws:iam::xxxxxxxxxx:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole" has "s3:GetObject" permissions on the manifest file. Error message from S3: The specified key does not exist.' What Should I do?

Hello everybody, in order to improve the transcription results, we're preparing custom vocabularies. We have a large corpus of text and would like to find all words in our corpus that are not yet in the vocabulary of Amazon Transcribe. Where do we find all words that Amazon Transcribe can recognize? Thank you in advance! Best regards Rustam

The DeepAR documentation page mentions the option of using Parquet format to providing datasets to DeepAR. But the details are extremely terse. I would like to use DeepAR with a fairly large dataset, and all the engineering is already in dataframes, so parquet would be well-suited. What should the input look like? How/where do we put multiple 'dynamic_feat' series? Category? If I have many targets, where I would do one per line in a huge JSON-lines format file, how do I set that up in Parquet?

Hey, After discovering ml.inf machines not supporting multi-model servers (MMS), we were forced to re-deploy our models. Currently, we are struggling to use MMS with EI accelerators. Is it an issue on our side, or are these accelerators not supported on MMS? ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Your Ecr Image pytorch-inference-eia:1.5.1-cpu-py3 does not contain required com.amazonaws.sagemaker.capabilities.multi-models=true Docker label(s). Thanks!

This is a follow up question after this [Question](https://repost.aws/questions/QUeFSnzs3sRAiSRpJFo5GquA/how-to-correctly-get-the-recommendation-if-the-user-data-or-item-data-change-where-the-previous-interactions-were-based-on-the-old-data). I am storing contextual metadata on interaction whose value is an array so it is stored as | separated string. The data stores the users licenses at the time of interaction. User may bought new licenses any time. Say, The definition of the Contextual Metadata on interaction is, ``` { "name": "LICENSES", "type": "string", "categorical": true } ``` And the data can be "L1", "L1|L3",, or "L1|L2|L3" etc. Where L1, L2, L3 are different licenses. Users may have any combination of them. Now I want to get Recommendations using GetRecommendation based on that context data. I would like to pass the current licenses user have on the Context. Can I pass the | separated string and get the desired result? For example an user has license L1 and L2, ``` context = { 'LICENSES': 'L1|L2' } ``` Now my question is, can we pass context like this? And will it work? I want to show items based on the license users have. On the documentation, it never explicitly says that we can provide such categorical data in context. All the examples use a single string, not a | separated array of strings. So I am wondering if I can pass such | separated array of string on context which would work. One might suggest filtering the Items based on license. But we want to show items without filtering in our case.

We have been trying to deploy our multiple models to a multi-model endpoint that uses inference machines (inf.xlarge) without luck. ClientError: An error occurred (ValidationException) when calling the CreateEndpointConfig operation: MultiModel mode is not supported for instance type ml.inf1.xlarge. This isn't good, is that really the case, or have we messed up somewhere during the process? Thanks

Hi All, Greetings!! We have two issues in sagemaker batch transform job. 1. Load balancing is not happening with two instances even after CPU utilization = 200% and GPU utilization = 81% of single instance and 2nd instance was complete idle. Transform job arguments: MaxConcurrentTransforms: 2, MaxPayloadInMB: 1, BatchStrategy: MultiRecord, InstanceCount=2 2. Batch transform job is failing after 20 minutes without any errors in cloud watch logs but noticed that CPU utilization = 200% and GPU utilization = 81% of single instance and 2nd instance was complete idle. Could you please have a look? Thanks, Vinayak

I would like to try out DeepAR for an engineering problem that I have some sensor datasets for, but I am unsure how to set it up for ingestion into DeepAR to get a predictive model. The data is essentially the positions, orientations, and a few other timeseries sensor readings of an assortment of objects (animals, in this case, actually) over time. Data is both noisy and sometimes missing. So, in this case, there are N individuals and for each individual, there are Z variables of interest per individual. None of the variables are "static" (color, size, etc), they are all expected to be time-varying on the same time scale. Ultimately, I would like to try and predict all Z targets for all N individuals. How do I set up the timeseries to feed into DeepAR? The premise is that all these individuals are implicitly interacting in the observed space, so all the target values have some interdependence on each other, which is what I would like to see if DeepAR can take into account to make predictions. Should I be using a category vector of length 2, such that the first cat variable corresponds to the individual, and the second corresponds to one of the variables associated with the individual? Then there would be N*Z targets in my input dataset, each with `cat = [ n , z ]`, where there are N distinct values for n, and z for Z?

I haven't been able to resolve the the following error that keeps showing up at the end of the [Looping over JSON](https://catalog.us-east-1.prod.workshops.aws/workshops/3d705026-9edc-40e8-b353-bdabb116c89c/en-US/loops/lab-6/step-2#looping-over-dictionaries-and-json) portion of Lab 6 in the "Learn Python on AWS Workshop" module: ``` error: the following arguments are required_ --file ``` Following the directions throughout the lab, I created the JSON file called `translate_input.json` and copied the list of dictionaries contained within. Then I created a new python file called `lab_6_step_2_loops.py` , typed in the text as directed, and ran the program with the command in the terminal python `lab_6_step_2_loops.py --file translate_input.json` after which the above error appears. I reached out to some coworkers who were also working on this but none of them have gotten back to me yet. Additionally, I've gone over all of the previous labs of this Python workshop to see what/if I missed anything, read numerous explanations, and watched several tutorials on YouTube regarding argparse and json. All of this was helpful but danced around the general issue without actually helping me resolve it but leads me to think the issue is related to the first section of the [code](https://catalog.us-east-1.prod.workshops.aws/workshops/3d705026-9edc-40e8-b353-bdabb116c89c/en-US/loops/lab-6/step-2#looping-over-dictionaries-and-json). ``` parser = argparse.ArgumentParser(description="Provides translation between one source language and another of the same set of languages.") parser.add_argument( '--file', dest='filename', help="The path to the input file. The file should be valid json", required=True) ``` Leaving this code as is, inputting the file name, file path, destination, or some combination keeps bringing up the same error as above. When I follow the directions in the link and literally just copy and paste- no typing- the text, I still come up with this error. Any thoughts?

I have scheduled an hourly model-quality-monitoring job in AWS SageMaker. both the jobs, ground-truth-merge and model-quality-monitoring completes successfully without any errors. but, all the metrics calculated by the job are "0.0 +/- 0.0" while the confustion matrix gets calculated as expected. I have done everything as mentioned in [this notebook for model-quality-monitoring from sagemaker-examples](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker_model_monitor/model_quality/model_quality_churn_sdk.ipynb) with very few changes and they are: 1. I have changed the model from xgboost churn to model trained on my data. 2. my input to the endpoint was csv like in the example-notebook, but output was json. 3. i have changed the problem-type from BinaryClassfication to MulticlassClassification wherever necessary. confustion matrix was built successfully, but all metrics are 0 for some reason. So, I would like the monitoring job to calculate the multi-classification metrics on data properly. **All Logs** Here's the `statistics.json` file that model-quality-monitor saved to S3 with confustion matrix built, but with 0s in all the metrics: ``` { "version" : 0.0, "dataset" : { "item_count" : 4432, "start_time" : "2022-02-23T03:00:00Z", "end_time" : "2022-02-23T04:00:00Z", "evaluation_time" : "2022-02-23T04:13:20.193Z" }, "multiclass_classification_metrics" : { "confusion_matrix" : { "0" : { "0" : 709, "2" : 530, "1" : 247 }, "2" : { "0" : 718, "2" : 497, "1" : 265 }, "1" : { "0" : 700, "2" : 509, "1" : 257 } }, "accuracy" : { "value" : 0.0, "standard_deviation" : 0.0 }, "weighted_recall" : { "value" : 0.0, "standard_deviation" : 0.0 }, "weighted_precision" : { "value" : 0.0, "standard_deviation" : 0.0 }, "weighted_f0_5" : { "value" : 0.0, "standard_deviation" : 0.0 }, "weighted_f1" : { "value" : 0.0, "standard_deviation" : 0.0 }, "weighted_f2" : { "value" : 0.0, "standard_deviation" : 0.0 }, "accuracy_best_constant_classifier" : { "value" : 0.3352888086642599, "standard_deviation" : 0.003252410977346705 }, "weighted_recall_best_constant_classifier" : { "value" : 0.3352888086642599, "standard_deviation" : 0.003252410977346705 }, "weighted_precision_best_constant_classifier" : { "value" : 0.1124185852154987, "standard_deviation" : 0.0021869336610830254 }, "weighted_f0_5_best_constant_classifier" : { "value" : 0.12965524348784485, "standard_deviation" : 0.0024239410000317335 }, "weighted_f1_best_constant_classifier" : { "value" : 0.16838092925822584, "standard_deviation" : 0.0028615098045768348 }, "weighted_f2_best_constant_classifier" : { "value" : 0.24009212108475822, "standard_deviation" : 0.003326031863819311 } } } ``` Here's how couple of lines of captured data looks like(*prettified for readability, but each line has no tab spaces as shown below*) : ``` { "captureData": { "endpointInput": { "observedContentType": "text/csv", "mode": "INPUT", "data": "0,1,628,210,30", "encoding": "CSV" }, "endpointOutput": { "observedContentType": "application/json", "mode": "OUTPUT", "data": "{\"label\":\"Transfer\",\"prediction\":2,\"probabilities\":[0.228256680901919,0.0,0.7717433190980809]}\n", "encoding": "JSON" } }, "eventMetadata": { "eventId": "a7cfba60-39ee-4796-bd85-343dcadef024", "inferenceId": "5875", "inferenceTime": "2022-02-23T04:12:51Z" }, "eventVersion": "0" } { "captureData": { "endpointInput": { "observedContentType": "text/csv", "mode": "INPUT", "data": "0,3,628,286,240", "encoding": "CSV" }, "endpointOutput": { "observedContentType": "application/json", "mode": "OUTPUT", "data": "{\"label\":\"Adoption\",\"prediction\":0,\"probabilities\":[0.99,0.005,0.005]}\n", "encoding": "JSON" } }, "eventMetadata": { "eventId": "7391ac1e-6d27-4f84-a9ad-9fbd6130498a", "inferenceId": "5876", "inferenceTime": "2022-02-23T04:12:51Z" }, "eventVersion": "0" } ``` Here's couple of lines from my ground-truths that I have uploaded to S3 look like(*prettified for readability, but each line has no tab spaces as shown below*): ``` { "groundTruthData": { "data": "0", "encoding": "CSV" }, "eventMetadata": { "eventId": "1" }, "eventVersion": "0" } { "groundTruthData": { "data": "1", "encoding": "CSV" }, "eventMetadata": { "eventId": "2" }, "eventVersion": "0" }, ``` Here's couple of lines from the ground-truth-merged file look like(*prettified for readability, but each line has no tab spaces as shown below*). this file is created by the ground-truth-merge job, which is one of the two jobs that model-quality-monitoring schedule runs: ``` { "eventVersion": "0", "groundTruthData": { "data": "2", "encoding": "CSV" }, "captureData": { "endpointInput": { "data": "1,2,1050,37,1095", "encoding": "CSV", "mode": "INPUT", "observedContentType": "text/csv" }, "endpointOutput": { "data": "{\"label\":\"Return_to_owner\",\"prediction\":1,\"probabilities\":[0.14512373737373732,0.6597074314574313,0.1951688311688311]}\n", "encoding": "JSON", "mode": "OUTPUT", "observedContentType": "application/json" } }, "eventMetadata": { "eventId": "c9e21f63-05f0-4dec-8f95-b8a1fa3483c1", "inferenceId": "4432", "inferenceTime": "2022-02-23T04:00:00Z" } } { "eventVersion": "0", "groundTruthData": { "data": "1", "encoding": "CSV" }, "captureData": { "endpointInput": { "data": "0,2,628,5,90", "encoding": "CSV", "mode": "INPUT", "observedContentType": "text/csv" }, "endpointOutput": { "data": "{\"label\":\"Adoption\",\"prediction\":0,\"probabilities\":[0.7029623691085284,0.0,0.29703763089147156]}\n", "encoding": "JSON", "mode": "OUTPUT", "observedContentType": "application/json" } }, "eventMetadata": { "eventId": "5f1afc30-2ffd-42cf-8f4b-df97f1c86cb1", "inferenceId": "4433", "inferenceTime": "2022-02-23T04:00:01Z" } } ``` Since, the confusion matrix was constructed properly, I presume that I fed the data to sagemaker-model-monitor the right-way. But, why are all the metrics 0.0, while confustion-matrix looks as expected? EDIT 1: Logs for the job are available [here](https://controlc.com/1e1781d2).

is there a limitation on the size of the model that we can create a model and then eventually serverless endpoint? any documentation? I did some research and ran into something similar here. https://discuss.huggingface.co/t/sagemaker-serverless-inference-for-layoutlmv2-model/14186/3 i as a solution , it is advised to set MMS_DEFAULT_WORKERS_PER_MODEL=1. I'm not sure what exactly does this do? is there any aws documentation around this?

I see Comprehend has an event detection API https://docs.aws.amazon.com/comprehend/latest/dg/how-events.html Is it capable of saying if the event happened ? or didn't happen? Or is in the future?

We have detected a small issue with moderation labels, We are running some images through the rekognition service, We tested it with some images of guns, we have found that close up images of guns does not trigger the moderation labels response. It will only trigger when we lower the minimum confidence to less than 4, which is unfortunately too low for us to run in production. One of the images we have been using is hosted on S3 and can be found at: https://dl-demo.s3.eu-west-2.amazonaws.com/messages/message_87.jpg Can AWS Include such images in their training sets?

I am recreating an endpoint currently working in sagemaker for inference to a serverless endpoint. I am using one of the images ( huggingface-pytorch-inference:1.9.1-transformers4.12.3-cpu-py38-ubuntu20.04) found here -> https://github.com/aws/deep-learning-containers/blob/master/available_images.md. everything works when i choose non serverless, i.e. provisioned option for endpoint configuration , but when i try to create one with serverless option it fails. error messages are below ( from the logs in cloudwatch) . starting with python and log4j error at the end. 'python: can't open file '/user/local/bin/deep_learning_container.py': [Errno 13] permission denied. Requirement already satisfied: transformers in /opt/conda/lib/pythong3.6 ..... ..... Warning: MMS is using non-default JVM parameters: -XX: -UseContainersupport Failed to reap children process log4j: ERROR setfile(null,true) call failed. java.io.FileNotFoundException: logs/mms_log.log (No Such file or directory) why am i getting this error ??? FYI - i have set memory to maximum allowed memory size of 6gb. for the serverless option.

Hello everyone, I've been a software and game developer for well over a decade, and I feel very at home with C, C++ and C#/.NET. I've done a lot of programming with DirectX SDKs and Unity, as well as desktop and mobile development. However, I don't really know much at all about the web and networking. But now I'm becoming very interested in what kinds of things I can do with AWS in game development, as well as with blockchain, AI/ML and cloud computing power. There seems to be more AWS services and packages than I can count though, and I'm really not sure which ones I should be pursuing and trying to learn more about and which ones are irrelevant to me and my job. The list of services, SDKs and packages is as impressive and inspiring as it is overwhelming and confusing! I'd like to be able to deploy some .NET applications to AWS to provide remote APIs for games and applications. To start with and get the hang of it, I just want to make a little minigame that lives on a server that a Unity or Unreal game can interact with through http requests. And I'll incrementally add some features to it as I start to get the hang of it. I'd also like to do some experimentation with blockchain software and services and see what kinds of interesting things can be accomplished with it. And beyond that, I'd eventually like to get into using AI/ML to accomplish goals in games and apps, harness cloud-based CPU/GPU processing power for heavy-lifting and even setup some real-time multiplayer game servers. I just don't know what tools/services/packages I need to dive into and start figuring out. I've found [the Visual Studio 2022 AWS package here](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.AWSToolkitforVisualStudio2022&ssr=false#overview), as well as [this AWS GitHub page](https://github.com/aws/dotnet) for .NET. I've also heard about Lambda. I really have no idea which ones are relevant to me or where I should start with this. To begin with, I want to run a .NET application on a server that applications can query and interact with and have it store and supply data for them, offer some remote APIs and requests, etc. I'd also potentially be interested in running some native C/C++ modules on servers. Can someone point me in the right direction and tell me what I need to set up and dive into first? I'm doing this in my free time when I'm not working on projects for work, so my time is very limited and researching and learning the wrong things would be a big setback for me. Any help or guidance is greatly appreciated!

We're trying to get streaming working with C++ LexRuntimeV2Client. I'm making a successful call to "PutSession," then attempting to start the stream by calling "StartConversationAsync." Code looks like this: ` LexRuntimeV2::Model::StartConversationHandler convoHandler = LexRuntimeV2::Model::StartConversationHandler(); convoHandler.SetTextResponseEventCallback(textEvent); convoHandler.SetIntentResultEventCallback(intentEvent); convoHandler.SetOnErrorCallback(errorEvent); convoHandler.SetHeartbeatEventCallback(heartbeatEvent); startConvo.WithBotId(BotId) .WithBotAliasId(BotAliasId) .WithLocaleId(Locale) .WithSessionId(ConvoSession) .WithEventStreamHandler(convoHandler) .WithConversationMode(Aws::LexRuntimeV2::Model::ConversationMode::TEXT); client.StartConversationAsync(startConvo, readyHandler, responseHandler); I'm getting a callback to "readyHandler" but none of the other callbacks are firing. I've read some sources that suggest that the issue is WinHttp client, but it's not clear how to switch to curl on Windows to test. Currently testing this on Windows, but the target deployment will be a Linux Docker container. Documentation is very very light for this API so I may definitely be calling it incorrectly. I can successfully call "RecognizeText" on the bot and get proper results so I know the client & bot are configured properly. Any help for how to make this call would be appreciated.

based on documentation provided here , https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/using_pytorch.html#model-directory-structure, the model file saved from training is model.pth. I also read that it can be .pt extension or even bin extension. I have seen a example of pytorch_model.bin, but when i tried to serve the model with the pytorch_model.bin, it warns me that .pt or .pth file needs to exist. has anyone run into this?

Hi Team, Greetings!! We are able to deploy on real-time endpoint with elastic inference accelerators. But SM Elastic Inference Accelerators are not available during inference, could you please have a look? Note: 1. We are bringing our own trained model using Pytorch = 1.10.2 on SM. 2. We don't find any errors in cloud watch logs. 3. Converted our trained model to TorchScript and able to load that model as well during inference. 4. Tried both Torchscript's trace and script modes but no luck. **Code** from sagemaker.pytorch import PyTorchModel from sagemaker import get_execution_role endpoint_name = 'ner-bert' pytorch = PyTorchModel(entry_point='deploy_ei.py', source_dir='code', model_data=model_data, role=get_execution_role(), framework_version='1.3.1', py_version='py3', sagemaker_session=sagemaker_session) predictor = pytorch.deploy(initial_instance_count=1, instance_type='ml.m5.large', accelerator_type='ml.eia2.xlarge', endpoint_name=endpoint_name) Thanks, Vinayak

[Continued/simplified from this post.](https://repost.aws/questions/QU-iZHXP55T_6BLKhHPET_0Q/if-i-open-a-web-socket-connection-with-one-of-my-sage-makers-notebook-cells-how-long-will-the-connection-last) Let's say I have a web app UI and I want to connect it to a SageMaker notebook using the Churn Predictor model in SageMaker Sudio. Think of it like using that model hosted in SageMaker as a backend to my web app. I could create an API endpoint but my web app has a very simple solution for opening a web socket which I'd like to use. Question: **Whats the simplest/best way to deploy the SageMaker notebook so that it keeps the kernel running and connection open?** The websocket means I don't need to set up and configure individual API endpoints which is what all the documentation I can find suggests. The desired workflow is: - Configure model in SageMaker notebook environment - Add a cell that establishes a websocket connection - Export notebook as a Python script (Maybe?) - Host & Run Python script on an AWS service With the web socket I can simply establish the connection: `anvil.server.connect(app_key)` And then give each function that I want my web app to call a decorator. i.e: ``` @anvil.server.callable def function(foo): return foo ```

Hi Team, Greetings!! We are not able to deploy on real-time endpoint with elastic inference accelerators. Could you please have a look? SageMaker version: 2.76.0 Code: from sagemaker.pytorch import PyTorchModel from sagemaker import get_execution_role endpoint_name = 'ner-bert' model = PyTorchModel(entry_point='deploy_ei.py', source_dir='code', model_data=model_data, role=get_execution_role(), framework_version='1.10.0', py_version='py38') predictor = model.deploy(initial_instance_count=1, instance_type='ml.m5.xlarge', accelerator_type='ml.eia2.medium', endpoint_name=endpoint_name) Error details: Unsupported pytorch version: 1.10.0. You may need to upgrade your SDK version (pip install -U sagemaker) for newer pytorch versions. Supported pytorch version(s): 1.3.1, 1.5.1, 1.3, 1.5. Note: We are able to deploy without elastic accelerator in above code and want to use Python 3.8 version because we have some dependency libraries which supports only Python 3.8 version. I looked at "Available DL containers" at https://github.com/aws/deep-learning-containers/blob/master/available_images.md and by looking at this section "SageMaker Framework Containers (SM support only)", SM support PyTorch 1.10.0 with Python 3.8 version. But we would like to deploy on Elastic Inference and by looking at this section "Elastic Inference Containers" in above URL, EI containers supports only PyTorch 1.5.1 with Python 3.6. Why these containers are so outdated? What could be the solution? Can we specify the latest version of Python in requirements.txt file and get it installed? Thanks, Vinayak

I was working on a Sagemaker notebook when suddenly it stopped working. I stopped and notebook and started it again, but now I cannot access it, getting the following error: *AccessDeniedException User: arn:aws:sts::755460267215:assumed-role/voclabs/user1672114=9540446637 is not authorized to perform: sagemaker:ListNotebookInstances with an explicit deny in an identity-based policy* Can somebody please help? I should note that I am pretty new to the Sagemaker environment.

Hi All, Good day!! How to invoke the SageMaker model endpoint directly from Postman without Lambda and API gateway setup? We have deployed the real-time endpoint on SM and data scientist (has AWS account) would like to invoke that endpoint via Postman from their local machine and test? Method: Post URL: https://runtime.sagemaker.ap-southeast-1.amazonaws.com/endpoints/my-endpoint-name/invocations We know that we can invoke SM endpoint using SM boto3 SDK like below, but we would like to invoke the endpoint directly via Postman. import boto3 smr = boto3.client('sagemaker-runtime') resp = smr.invoke_endpoint(EndpointName=endpoint_name, Body=b'.345,0.224414,.131102,0.042329,.279923,-0.110329,-0.099358,0.0', ContentType='text/csv') Thanks, Vinayak

Hi All, Good day!! How to invoke the SageMaker model endpoint directly from Postman without Lambda and API gateway setup? We know that we can invoke SM endpoint using SM boto3 SDK like below, but we would like to invoke the endpoint directly via Postman. import boto3 smr = boto3.client('sagemaker-runtime') resp = smr.invoke_endpoint(EndpointName=endpoint_name, Body=b'.345,0.224414,.131102,0.042329,.279923,-0.110329,-0.099358,0.0', ContentType='text/csv') Thanks, Vinayak

Hi Team, Greetings!! Could you please confirm, how can we define concurrency in SageMaker real-time inference? This is how we define concurrency in SageMaker serverless inference. “ServerlessConfig”: { “MemorySizeInMB”: 4096, “MaxConcurrency”: 10, } Thanks, Vinayak

The notebook instance is `ml.m5d.2xlarge` with 32GB of memory. However, we are encountering some OOM errors in SageMaker Notebooks when generating embeddings with Tensorflow: ``` tf_hub_embedding_layer = hub.KerasLayer("https://tfhub.dev/google/universal-sentence-encoder/4", trainable=False, name="universal_sentence_encoder") embeddings = tf_hub_embedding_layer(lr_cleaned_df.text.values) ``` When retrieving the universal sentence encoder, the virtual memory size is about 7GB. The size of `lr_cleaned_df.text.values` contains 60K snippet of texts and is about 122MB in memory. Is there a default allocation of memory to a Jupyter notebook? If so, can this be overwritten?

I'm using one of the images listed here https://github.com/aws/deep-learning-containers/blob/master/available_images.md, to create an model such that I can tie that up with a sagemaker serverless endpoint , but I keep getting "failed reason: Image size 15136109518 is greater that suppported size 1073741824" . this work when the endpoint configuration is not serverless. is there any documentation around image/container size for aws managed images?

Hi, our company is working on creating an innovative security system to prevent theft of goods from parking trucks. Our goal is to equip every truck with radar sensors and a central control unit which handles the incoming data. We want to upload this data to a cloud so that we can use machine learning in order to analyse the risk of a theft at any given moment. Our concept is: 1. Send the data to the cloud 2. Store the data on the cloud 3. Analyse and use machine learning on the data We are looking to set our central unit (Currently a raspberry pi 4) as an AWS IoT device. The main questions are: What are the differences between IoT Core and Greengrass? What are the pros and cons of each of these services? Which one would fit most to our use-case? Please send us a detailed description and any kind of documentation you think would help us in achieving our goals.

Hi All, Greetings!! Please address my below queries. 1. Does the pre and post-processing need to be incorporate in SM? 2. Isn't SM supposed to be used for inference only? Why I'm asking these questions because I am using PyTorch model server for pre-processing, predictions and post-processing for NER use cases. 3. We use utokenize for creating tokens which supports >= Python 3.8 but PyTorch model server supports <= 3.6, what would be the solution? How to install Python 3.8 in PyTorch model server? 4. Don't we have basic word piece tokenization as pre-processing in SM? Thanks, Vinayak

When I was building model for analyzing in SageMaker Canvas, it just run for 1h 2m and then I got this notification: Model building failed: Failed to run Neo compilation or generate explainability report. client_request_id is f76d5bf7-6780-4257-9631-500101632b1e Where should I contact the admin for this issue? Thank you so much!

I am doing this on dekstop. I have created a dataset with training and testing, added all my labels to the images etc. Here is the error I am getting. "Amazon Rekognition Custom Labels couldn't train your model. Check that your IAM policy and your S3 bucket policy are correct. For more information, see the Amazon Rekognition Developer Guide." I have been searching the forums, tried creating a policy but I still cannot figure out the issue. Even a point in the right direction would be very awesome. I am very new to AWS and this is project I have to do for class. Thanks!

Hi All, Good day!! Key point to note here is, we have pre-processing script for the text document, deserialize which is required for prediction then we have post-processing script for generating NER (entitites). I went through SageMaker material and decided to try following options. 1. Option 1: Bring our own model, write a inference script and deploy it on SM real-time endpoint using Pytorch container. I went through Suman video (https://www.youtube.com/watch?v=D9Qo5OpG4p8) which is really good, need to try with our pre-processing and post-processing scripts then see if it works fine or not. 2. Option 2: Bring our own model, write a inference script and deploy it on SM real-time endpoint using Huggingface container. I went through Huggingface docs (https://huggingface.co/docs/sagemaker/inference#deploy-a-%F0%9F%A4%97-transformers-model-trained-in-sagemaker) but there is no reference for how to use own pre and post-processing scripts to setup inference pipeline. If you know any examples on using our own pre and post-processing scripts using Huggingface container then please share it. 3. Option 3: Bring our own model, write a inference script and deploy it on SM Serverless inference/endpoint using Huggingface container. I went through Julien video (https://www.youtube.com/watch?v=cUhDLoBH80o&list=PLJgojBtbsuc0E1JcQheqgHUUThahGXLJT&index=35) which is excellent but he has not shown how to use our own pre and post-processing scripts using Huggingface container. Please share if you know any examples. Could you please help? Thanks, Vinayak