Memory error occurs in amazon sagemaker when preprocessing 400 mb of data which is stored in s3. No problem in loading the data. Dimension of data is 8-9 million rows and 4 columns, but I get 7000 columns after applying One hot encoding. One tying to train de model I get the following memory error: MemoryError: Unable to allocate 946. MiB for an array with shape (123962752,) and data type float64 Notebook instance is ml.t2.medium. How to solve this issue?

This post is both a bug report and a question. We are trying to use SageMaker to train a model and everything is quite standard. Since we have a lot of images, we'll suffer from a super long image downloading time if we don't change the input_mode to FastFile. Then I struggled to successfully load image in the container. In my dataset there are a lot of samples whose name contains Chinese. When I started debugging because I could not properly load files, I found that when sagemaker mounts the data from s3, it didn't take care of the encoding correctly. Here is an image name and the image path inside the training container: `七年级上_第10章分式_七年级上_第10章分式_1077759_title_0-0_4_mathjax` `/opt/ml/input/data/validation/\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_1077759_title_0-0_4_mathjax.png` This is not neat but still I can have the right path in the container. The problem is that I'm not able to read the file even though the path exists: what I mean is `os.path.exists('/opt/ml/input/data/validation/\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_1077759_title_0-0_4_mathjax.png')` gives true but `cv2.imread('/opt/ml/input/data/validation/\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_1077759_title_0-0_4_mathjax.png')` returns None. Then I tried to open the file and fortunately it gives an error The code is `with open('/opt/ml/input/data/validation/\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_\u4E03\u5E74\u7EA7\u4E0A_\u7B2C10\u7AE0\u5206\u5F0F_1077759_title_0-0_4_mathjax.png', 'rb') as f: a = f.read() ` and it gives me the error `OSError: [Errno 107] Transport endpoint is not connected` I tried to load a file in the same folder whose name doesn't contain any Chinese. Everything works well in this case so I'm sure that the Chinese characters in the filenames are causing problems. I wonder if there is a quick walk around so I don't need to rename maybe 80% of the data in s3.

Hi- I’m using WZone plug-in for Amazon Associates with the remote images option on. My side has around 500 products now and the site has become super slow, with products taking up to 30 to 60 seconds to load! I’ve been told that using remote images is the main issue causing such slow site speed. Does anyone know if the Amazon Associates rules require use of remote images or can the images be stored on my server? Do you think adding products with remote images off will speed up the site significantly? I’ve tried almost everything so far & this issue has persisted for many months! Any feedback would be greatly appreciated! Thanks!!!

Hello folks, I have been translating manually more than 3 million words and I just wanted to use that to accelerate my translations. I guess AWS translation, and the parallel feature, is just what I need but when I tried to input my files I discovered that you can only upload CSV, Memory Translations and another format that is new to me. I only have the translations in 2 websites, each one in a different format, and with some patience I can create Word files, but, How can I transform it to CSV? Or, What can I do? The problem with CSV is that sometimes I join 2 English sentences in just 1 Spanish sentence, so it is not a 1 to 1 correlation. Any ideas? Thanks

Sagemaker Studio Lab has stopped working. First, all kernels fail with unknown error and can't restart. OR get stuck in a loop restarting themselves. I am unable to make a new kernel. Second, this error keeps popping up: "Unexpected error while saving file: aws/notebooks/MLA-TAB-DAY3-AUTOML.ipynb [Errno 28] No space left on device: '" And when checking resources it shows that 100% of storage is used at /hom/studio-lab-user but I am unable to delete any of the files in that directory or the directory itself. > /dev/nvme0n1p1 50G 17G 34G 33% /opt/.sagemakerinternal > /dev/nvme1n1 25G 25G 36K 100% /home/studio-lab-user How can I just reset the entire Sagemaker Studio Lab profile to default, erase all data that's clogging up storage/memory, and start fresh?

We deployed a LighGBM Regression model and endpoint using Sagemaker Jumpstart. We have attempted to configure this endpoint as 'asynchronous' via the console. Receiving Error: ValidationException-Network Isolation is not supported when specifying an AsyncInferenceConfig. Looking at the model's network details the model has Enable Network Isolation set as 'True'. This was default output setting set by JumpStart. How can we diasble Network Isolation to in order to make this endpoint asynchronous?

We have been testing the celebrity recognition service and have noticed one issue when we upload the picture of a famous Indian actor, Salman Khan. The last name that comes up in the response is actually an abusive word of local language in India. You can test it for yourself by uploading the following image. https://www.bollywoodhungama.com/wp-content/uploads/2021/10/salman-khan.jpg How the response can be fixed, if at all?

We are integrated the AWS lex into our Slack public channel as an answer bot. But if someone talks in the channel with a context length that is higher than 1024, the AWS lex bot will return an error that is kind of annoying and not related. Is there any way to turn off the error handling or make it mute for error reporting in Slack? Thanks for any help/support.

I have a vehicle re identification (in multiple cameras) task and I want to label some of my own data with the right identity considering each vehicle timestamp and direction. Is this something I can use sagemaker's ground truth for? and if it's possible how much will it cost for about 500 images?

I encountered following erorr when I tried to create a new project in SageMaker Studio. I tried to add `AdminAccess` to `role/service-role/AmazonSageMakerServiceCatalogProductsLaunchRole`, but still doesn't help. What could be the problem? ``` Error getting the details of Service Catalog Provisioning Parameters. Error message: ValidationException: Access denied while assuming the role arn:aws:iam::XXXXXXXXX:role/service-role/AmazonSageMakerServiceCatalogProductsLaunchRole. Args: {"productId":"prod-xxxxxxxx","provisioningArtifactId":"pa-xxxxxxxxx","pathId":"lpv2-xxxxxxxxxx"} ``` [printscreen](https://raw.githubusercontent.com/qinjie/picgo-images/main/qerpoijsdfijsadfjaosdjfqwefasdfasdfasdfasdf.png)

Hi, based on https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb I've created a docker image which contains the model code implemented in Python. This model image is deployed as async inference endpoint. Everything works well, the endpoint can be invoked via InvokeEndpointAsync, custom attributes are provided to the model code via "X-Amzn-SageMaker-Custom-Attributes" header, the input located on S3 is provided as http body. After succesful processing the result is uploaded to S3 with an automatically generated output filename ("*.out"). The container behavior is logged to CloudWatch via Python logging module, initialized as: ``` import logging logging.basicConfig(level=logging.INFO) logger = logging.getLogger() ``` However, the request arriving inside the container neither contains an "X-Amzn-SageMaker-Inference-Id" nor "X-Amzn-SageMaker-InputLocation" field (mentioned here https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpointAsync.html). Is there a way to access the inference_id/output filename from inside the container? Or any other opportunity to link input filename/container log/output filename? Otherwise it's not possible to fully track and examine requests. Thanks in advance for your advice!

How is High availability addressed in aws panorama appliance? what is the recommended approach? What if one appliance fails? what is the SLA for panorama appliance if it stops working?

I work in SM Studio, and I do not understand why CPU and memory usage do not appear in the notebook toolbar. These metrics should be there, at least given this description: https://docs.amazonaws.cn/en_us/sagemaker/latest/dg/notebooks-menu.html When I open a notebook in SM Studio, I see the same toolbar but without CPU and memory usage listed. Moreover, I see 'cluster' before the kernel's name in my toolbar. Has anyone experienced sth similar? I assume an alternative for me would be to use CloudWatch.

I have followed the boto3 documentation version 1.23.1 and used analyze_document method for synchronous analysis from lambda. I have followed the code snippet given [here](https://aws.amazon.com/blogs/machine-learning/specify-and-extract-information-from-documents-using-the-new-queries-feature-in-amazon-textract/). I need to pass custom queries to textract via QueriesConfig parameter, but it raises "Unknown parameter" error. I cannot find any reference for this error [here](https://docs.aws.amazon.com/textract/latest/dg/API_AnalyzeDocument.html).

Facing issue - AWS FACE RECANTATION not detecting many faces from video. Wr are loosing many expected faces from video. In our application we are using AWS face recognition services to detect faces from video. Earlier(4 months back) it is working fine and was able to detect many faces but now the recognised faces count is very less. We are losing many faces that we can see are present in video by naked eye. Service should be able to detect those faces. So my question is, **Is their any changes in service or is their any restriction added to the service? ** we are using these services: 1. start_face_detection (To Start video processing at amazon end) 2. get_face_detection (To Get detected faces )

Actually Rekognition works fine but when I use a filename containing a colon (:) for the S3Object, it makes an error. It is very problematic for me because all my files already have colons and I can't change their names. So if use this It works fine: ``` { "Image":{ "S3Object":{ "Bucket":"console-sample-images", "Name":"skateboard.jpg" } ``` but if i use a name with a colon like this It gives me an error. ``` { "Image":{ "S3Object":{ "Bucket":"console-sample-images", "Name":"skate:board.jpg" } ``` Error output: `{"name":"Error","content":"{\"__type\":\"InvalidS3ObjectException\",\"Code\":\"InvalidS3ObjectException\",\"Message\":\"Unable to get object metadata from S3. Check object key, region and/or access permissions.\"}","message":"faultCode:Server.Error.Request faultString:'null' faultDetail:'null'","rootCause":{"errorID":2032,"target":{"bytesLoaded":174,"dataFormat":"text","bytesTotal":174,"data":"{\"__type\":\"InvalidS3ObjectException\",\"Code\":\"InvalidS3ObjectException\",\"Message\":\"Unable to get object metadata from S3. Check object key, region and/or access permissions.\"}"},"text":"Error #2032: Stream Error. URL: https://rekognition.eu-west-1.amazonaws.com","currentTarget":{"bytesLoaded":174,"dataFormat":"text","bytesTotal":174,"data":"{\"__type\":\"InvalidS3ObjectException\",\"Code\":\"InvalidS3ObjectException\",\"Message\":\"Unable to get object metadata from S3. Check object key, region and/or access permissions.\"}"},"type":"ioError","bubbles":false,"eventPhase":2,"cancelable":false},"errorID":0,"faultCode":"Server.Error.Request","faultDetail":null,"faultString":""}` Is there a workaround for this problem? (encoding the ':' a certain way?) Thank you for your help.

When creating a Ground Truth labelling job, some of the files in S3 are not displaying in the browser. The files (.png image and .flac audio file) are available in S3 and are able to be seen / heard when downloaded. Any ideas on what is wrong?

I am trying to send a query with the function analyze_document (or start_document_analysis) as described in the docs , however I get the error: `Unknown parameter in input: "QueriesConfig", must be one of: Document, FeatureTypes, HumanLoopConfig ` If I then remove "QueriesConfig", I get the error : `An error occurred (InvalidParameterException) when calling the AnalyzeDocument operation: QueriesConfig must be used with QUERIES FeatureType. ` I do not know hence what to do. I leave the full script code. ``` client = boto3.client('textract') image_binary = stream.getvalue() response = client.analyze_document(Document={'Bytes': image_binary}, FeatureTypes=["QUERIES"], QueriesConfig={ 'Queries': [ { 'Text': 'what is the document date?' } ] }) ``` Thanks!

Good day! **My main purpose:** Easy way to collect information about different failure scenarios in SageMaker TrainingJob. **What do I use currently?** Sagemaker SKLearn Estimator(TrainingJobs are inside) **Where will my model train?** Different datasets. So, I need control and collect all information about all training processes and their final statuses on different datasets. **Which failure scenarios do I have?** There are plenty of them. I have create my own python Errors for them. For example: 1. There are labels only for one class. 2. Too small dataset(by my own criterions) 3. Missing data for crucial columns 4. e.t.c. **Where am I stuck?** After failed training I can't get own errors from training job response. All of them are "ExecuteUserScriptError" I can't pass my own info in FailureReason or ErrorMessage(always it's empty). I see which error was raised in CloudWatchLogs and TrainingJobTraceback(from SagemakerNotebook). So, bad solution is parse all CloudWatchLogs in case of failure. **Question: How to provide my own ErrorMessage or FailureReason? ** May be I am digging in the wrong direction. Anyway, I need your advice. Thank you so much for possibility to ask an advice here)

Hello! We started using Sagemaker Jupyter Lab to run a few Depp Learning experiments we previously ran on GoogleColabPro+. The training starts fine and everything seems to work, however, the progress bar appears as follows: **Validation sanity check: 0it [00:00, ?it/s] Training: 0it [00:00, ?it/s]** The progress bar was working fine on GoogleColab. I tried uninstalling ipywidgets as [suggested here](https://github.com/PyTorchLightning/pytorch-lightning/issues/11208), but still no luck. Anyone has an idea of how to fix the problem? Below you will find a copy of the TrainerFunction I am using. ``` class T5FineTuner(pl.LightningModule): def __init__(self, hparams): super(T5FineTuner, self).__init__() self.hparams = hparams self.model = T5ForConditionalGeneration.from_pretrained(hparams['model_name_or_path']) self.tokenizer = T5Tokenizer.from_pretrained(hparams['tokenizer_name_or_path']) def hparams(self): return self.hparams def is_logger(self): return True #self.trainer.proc_rank <= 0 def forward( self, input_ids, attention_mask=None, decoder_input_ids=None, decoder_attention_mask=None, labels=None ): return self.model( input_ids, attention_mask=attention_mask, decoder_input_ids=decoder_input_ids, decoder_attention_mask=decoder_attention_mask, labels=labels, ) def _step(self, batch): labels = batch["target_ids"] labels[labels[:, :] == self.tokenizer.pad_token_id] = -100 outputs = self( input_ids=batch["source_ids"], attention_mask=batch["source_mask"], labels=labels, decoder_attention_mask=batch['target_mask'] ) loss = outputs[0] return loss def training_step(self, batch, batch_idx): loss = self._step(batch) tensorboard_logs = {"train_loss": loss} return {"loss": loss, "log": tensorboard_logs} def training_epoch_end(self, outputs): avg_train_loss = torch.stack([x["loss"] for x in outputs]).mean() tensorboard_logs = {"avg_train_loss": avg_train_loss} # return {"avg_train_loss": avg_train_loss, "log": tensorboard_logs, 'progress_bar': tensorboard_logs} def validation_step(self, batch, batch_idx): loss = self._step(batch) return {"val_loss": loss} def validation_epoch_end(self, outputs): avg_loss = torch.stack([x["val_loss"] for x in outputs]).mean() tensorboard_logs = {"val_loss": avg_loss} return {"avg_val_loss": avg_loss, "log": tensorboard_logs, 'progress_bar': tensorboard_logs} def configure_optimizers(self): "Prepare optimizer and schedule (linear warmup and decay)" model = self.model no_decay = ["bias", "LayerNorm.weight"] optimizer_grouped_parameters = [ { "params": [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)], "weight_decay": self.hparams['weight_decay'], }, { "params": [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)], "weight_decay": 0.0, }, ] optimizer = AdamW(optimizer_grouped_parameters, lr=self.hparams['learning_rate'], eps=self.hparams['adam_epsilon']) self.opt = optimizer return [optimizer] def optimizer_step(self, epoch=None, batch_idx=None, optimizer=None, optimizer_idx=None, optimizer_closure=None, second_order_closure=None, on_tpu=False, using_native_amp=False, using_lbfgs=False): # if self.trainer.use_tpu: # xm.optimizer_step(optimizer) # else: optimizer.step(closure=optimizer_closure) optimizer.zero_grad() self.lr_scheduler.step() def get_tqdm_dict(self): tqdm_dict = {"loss": "{:.3f}".format(self.trainer.avg_loss), "lr": self.lr_scheduler.get_last_lr()[-1]} return tqdm_dict def train_dataloader(self): train_dataset = get_dataset(tokenizer=self.tokenizer, type_path="translated_train", args=self.hparams) dataloader = DataLoader(train_dataset, batch_size=self.hparams['train_batch_size'], drop_last=True, shuffle=True, num_workers=4) t_total = ( (len(dataloader.dataset) // (self.hparams['train_batch_size'] * max(1, self.hparams['n_gpu']))) // self.hparams['gradient_accumulation_steps'] * float(self.hparams['num_train_epochs']) ) scheduler = get_linear_schedule_with_warmup( self.opt, num_warmup_steps=self.hparams['warmup_steps'], num_training_steps=t_total ) self.lr_scheduler = scheduler return dataloader def val_dataloader(self): val_dataset = get_dataset(tokenizer=self.tokenizer, type_path="test_2k", args=self.hparams) return DataLoader(val_dataset, batch_size=self.hparams['eval_batch_size'], num_workers=4) logger = logging.getLogger(__name__) class LoggingCallback(pl.Callback): def on_validation_end(self, trainer, pl_module): logger.info("***** Validation results *****") if pl_module.is_logger(): metrics = trainer.callback_metrics # Log results for key in sorted(metrics): if key not in ["log", "progress_bar"]: logger.info("{} = {}\n".format(key, str(metrics[key]))) def on_test_end(self, trainer, pl_module): logger.info("***** Test results *****") if pl_module.is_logger(): metrics = trainer.callback_metrics # Log and save results to file output_test_results_file = os.path.join(pl_module.hparams["output_dir"], "test_results.txt") with open(output_test_results_file, "w") as writer: for key in sorted(metrics): if key not in ["log", "progress_bar"]: logger.info("{} = {}\n".format(key, str(metrics[key]))) writer.write("{} = {}\n".format(key, str(metrics[key]))) ```

It is mentioned in the AWS EI docs that we can use EI in US EAST, WEST, ASIA PACIFIC, EU. But when i launch an instance in ohio or Seoul, it shows this error > ``` Instance launch failed The Availability Zone [us-east-2c] does not support Elastic Inference Accelerators. Choose another Availability Zone. Launch log Initializing requests Succeeded Launch initiation Failed ``` Why is this happenging?? Is this a problem from my end?

I am building a ml workflow using step function following [this](https://sagemaker-examples.readthedocs.io/en/latest/step-functions-data-science-sdk/machine_learning_workflow_abalone/machine_learning_workflow_abalone.html). However, when I start the state machine, I got error ``` AlgorithmError: framework error ... SM_CHANNEL_TRAIN ...exit code: 1 ``` Does anyone know how to fix it? or how to set SM_CHANNEL_TRAIN? Thank you

I am exploring the Sagemaker Built-in algorithms, and I am curious to learn more about the details of the algorithms. However, I am surprised that it is hard to find any references for the research background and implementation details in the numerous documents and tutorials for particular algorithms. If such information exists somewhere, I would highly appreciate a pointer. Thanks a lot in advance!

Hi Dears, I am building ML model using DeepAR Algorithm. I faced this error while i reached to this point : Error : ClientError: An error occurred (UnknownOperationException) when calling the CreateHyperParameterTuningJob operation: The requested operation is not supported in the called region. ------------------- Code: from sagemaker.tuner import ( IntegerParameter, CategoricalParameter, ContinuousParameter, HyperparameterTuner, ) from sagemaker import image_uris container = image_uris.retrieve(region= 'af-south-1', framework="forecasting-deepar") deepar = sagemaker.estimator.Estimator( container, role, instance_count=1, instance_type="ml.m5.2xlarge", use_spot_instances=True, # use spot instances max_run=1800, # max training time in seconds max_wait=1800, # seconds to wait for spot instance output_path="s3://{}/{}".format(bucket, output_path), sagemaker_session=sess, ) freq = "D" context_length = 300 deepar.set_hyperparameters( time_freq=freq, context_length=str(context_length), prediction_length=str(prediction_length) ) Can you please help in solving the error? I have to do that in af-south-1 region. Thanks Basem hyperparameter_ranges = { "mini_batch_size": IntegerParameter(100, 400), "epochs": IntegerParameter(200, 400), "num_cells": IntegerParameter(30, 100), "likelihood": CategoricalParameter(["negative-binomial", "student-T"]), "learning_rate": ContinuousParameter(0.0001, 0.1), } objective_metric_name = "test:RMSE" tuner = HyperparameterTuner( deepar, objective_metric_name, hyperparameter_ranges, max_jobs=10, strategy="Bayesian", objective_type="Minimize", max_parallel_jobs=10, early_stopping_type="Auto", ) s3_input_train = sagemaker.inputs.TrainingInput( s3_data="s3://{}/{}/train/".format(bucket, prefix), content_type="json" ) s3_input_test = sagemaker.inputs.TrainingInput( s3_data="s3://{}/{}/test/".format(bucket, prefix), content_type="json" ) tuner.fit({"train": s3_input_train, "test": s3_input_test}, include_cls_metadata=False) tuner.wait()

The QueryResultItem includes a ScoreAttributes of VERY_HIGH, HIGH, MEDIUM and LOW. Is the actual confidence result available, or an indication of the threshold of these values?

We are wondering whether we can enable workers to go back to earlier items in annotation tasks. Currently, using the default template for customizing the text classification job, we do not see a button of going back. In addition, we are wondering how to add files for Angular to Ground Truth application.

I'm trying to understand how to code my Terraform for MSK, and noticing that the "current_version" changed after my changes were applied. That makes sense since it did change the PCA configuration... but which kinds of changes actually cause that property to change? Does every `aws kafka update-cluster-configuration` change it?

My IAM works, but when I login, it logs me into us-east-1. My project is on the S3 instance (sagemaker tool). How can I access buckets in S3 if I am logged into us-east-1 by default.

Hi all, I am asking if it's possible to use `framework processor` inside a `sagemaker pipeline`. I am asking because the to submit the source_dir for the framework processor, we have to do so when calling the .run() method, when wrapping the processor inside a `sagemaker.workflow.steps.ProcessingStep`, there isn't an available argument to specify the `source_dir`. Thank you! Best, Ruoy

Hi, we are developing an invoice OCR, so we decided to use Textract to get info from the invoices and exepenses useing the AnalyzeExpense/StartExpenseAnalysis commands. In some cases there are fields that are not detected by them. We saw that there is an option to implement Textract with A2I to train our a model. But as we can see in the API documentantion it seems to only be available with AnalyzeDocument. Is there any option to integrate with the requests in which we are interested (AnalyzeExpense/StartExpenseAnalysis) ?

I have a dashboard and I do not want to have any access restrictions for Amazon employees. Is it possible? any ANT group that provides access to all Amazon employees?

Hi I received my aws sagemaker studio approval to create an account 1 hr ago When I go to the link to create an account it says my email has not been approved Even though I have an email to the contrary How do I contact amazon to sort this out?

I recently tried the Polly console set to Neural and SSML and keyed in some text and tried several voices with the Whisper action and it works for the Mathew voice but does not work for any other one. If its set to Normal instead of Neural the whisper does work for other voices but does not sound good. Is there a reason the whisper only works for that one voice using the Neural setting?

Hello, I tried using [Inf1 EC2 instance](https://aws.amazon.com/ec2/instance-types/inf1/) for deploying my ML model. I need to monitor the GPU usage of the ML model. I could find the CPU usage in the aws console, but not gpu usage. Already tried: 1. https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-gpu-monitoring-gpumon.html This didn't work. It threw this error ``` (python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ python3 gpumon.py Traceback (most recent call last): File "gpumon.py", line 146, in <module> nvmlInit() File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1450, in nvmlInit nvmlInitWithFlags(0) File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 1440, in nvmlInitWithFlags _nvmlCheckReturn(ret) File "/home/ubuntu/anaconda3/envs/python3/lib/python3.8/site-packages/pynvml/nvml.py", line 765, in _nvmlCheckReturn raise NVMLError(ret) pynvml.nvml.NVMLError_DriverNotLoaded: Driver Not Loaded ``` Also, `nvidia-smi` didn't work ``` (python3) ubuntu@ip-xxx-mm-yy-zzz:~/tools/GPUCloudWatchMonitor$ nvidia-smi NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running. ``` Kindly provide some help to monitor GPU usage.

Hi all, I've followed the guide provided in this medium blog by senior AWS ML SA: https://towardsdatascience.com/hosting-vs-code-in-sagemaker-studio-f211385e25f7 I've created the lifecycle configuration (jupyterlab app) for the following bash script: ``` #!/bin/bash mkdir -p vscode cd vscode sudo su curl -fsSL https://code-server.dev/install.sh | sh exit ``` However, when I attach the configuration to the domain or a specific user's jupyter server in the SM Studio, the JupyterServer App fails to be created with the following message displayed on the loading page of SM Studio's jupyterlab: > "The JupyterServer app default encountered a problem and was stopped.Details: ConfigurationError: LifecycleConfig execution failed with non zero exit code 1 for script arn:aws:sagemaker:eu-west-3:615740825886:studio-lifecycle-config/install-vscode-on-jupyterserver. Check https://docs.aws.amazon.com/sagemaker/latest/dg/studio-lcc-debug.html for debugging instructions." I checked the page for debugging instructions, specifically the part where I am instructed to check the cloudwatch logs for the log group: aws/sagemaker/studio The problem is this log group does not exist in CloudWatch (I have admin permission). I would like to have this set up for the entire team, Any help would be greatly appreciated! Best, Ruoy

Tried hitting analyze_expense API from the codebase and uploaded the same receipt in the AWS consoles analyze_expense form, I am getting different output in the console and the API call from the codebase AWS console shows a better output of amount extraction than the API call from the codebase with the same API (analyze_expense) is there any possibility of a different version of analyze_expense() API serving AWS console and a different version serving API integrated into our codebase? I've noticed this behaviour with a few invoices and also can reproduce the same can someone please help with this

Hello, I have configured lex bot with an intent that consists of one sample utterance. While testing, if user utterance is lengthy, it triggers fallback intent even though it is matching with configured intent with confidence score of '0.78' which is greater than default threshold(0.4). Help me with this issue. **Steps to Reproduce**: 1. Create a custom bot in lex v2 console with default configuration and a intent named "PolicyChange". 2. Add this sample utterance - "I want my husband's name removed from the policy". 3. Configure fulfillment successful message for that intent with a message something like - "Ok we will update policy" 4. Build the bot and test it in test console with utterance - "My husband died last month due to covid. Now I want his name removed from the policy. Can you help me with that." 5. Inspect the JSON input and output response in test console. You can see "FallbackIntent" is triggered. But, in Interpretations the "PolicyChange" intent has confidence score of "0.78".

Using a web console for Amazon Personalize for creating item datasets, it throws an error "GENRES field cannot be of type null" when trying to create an item schema with GENRES that contains type "null". But the documentation https://docs.aws.amazon.com/personalize/latest/dg/how-it-works-dataset-schema.html has GENRES with type null for imperfect data. Also, it throws an error "schema is missing required field [genres]" if it's not included in the schema, which is odd, because the documentation says that the item schema requires ITEM_ID, CREATION_TIMESTAMP, and 1 metadata field where the table doesn't even mention genres. https://docs.aws.amazon.com/personalize/latest/dg/custom-datasets-and-schemas.html#dataset-requirements However, we could create the schema without genres when using the AWS SDK. Is this a bug in AWS console? or is it a new requirement that hasn't been updated on SDK?

I am following the V2 Lex Developer Guide to create a bot from the OrderFlowers Blueprint. When I add the Lambda function to the bot (page 145 of the V2 Guide), I receive the error below. Any idea why this is happening? I have followed the steps in the V2 Developer Guide. Invalid Lambda Response: Your request to Amazon Lex couldn't be completed. Lambda function arn:aws:lambda:us-east-1:994504927747:function:OrderFlowersCodeHook:OrderFlowersCodeHook encountered a problem while processing Amazon Lex request 6406cf16-5cd5-4c72-8a2a-fed1717e5e3a. The error message from the Lambda function is Unhandled. Check the Lambda function log for error details, then try your request again after fixing the error.

Overview: I'm trying to run an XGboost model on a bunch of parquet files sitting in S3 using dask by setting up a fargate cluster and connecting it to a Dask cluster. Total dataframe size totals to about 140 GB of data. I scaled up a fargate cluster with properties: Workers: 40 Total threads: 160 Total memory: 1 TB So there should be enough data to hold the data tasks. Each worker has 9+ GB with 4 Threads. I do some very basic preprocessing and then I create a DaskDMatrix which does cause the task bytes per worker to get a little high, but never above the threshold where it would fail. Next I run xgb.dask.train which utilizes the xgboost package not the dask_ml.xgboost package. Very quickly, the workers die and I get the error `XGBoostError: rabit/internal/utils.h:90: Allreduce failed`. When I attempted this with a single file with only 17MB of data, I would still get this error but only a couple workers die. Does anyone know why this happens since I have double the memory of the dataframe? ``` X_train = X_train.to_dask_array() X_test = X_test.to_dask_array() y_train = y_train y_test = y_test ``` dtrain = xgb.dask.DaskDMatrix(client,X_train, y_train) output = xgb.dask.train( client, {"verbosity": 1, "tree_method": "hist", "objective": "reg:squarederror"}, dtrain, num_boost_round=100, evals=[(dtrain, "train")])`

Hi All, I have integrated Polly into my WordPress blog and so far so good. The only issue I have is that it reads everything and I’d like to leave out certain things like twitter embeds etc. I’ve seen that you can insert SSML tags in to the content to do this but this is where I’m completely clueless. Am I inserting this into WordPress? If so, how and where? Or is it something I need to do in the Polly Dashboard (where the file lives in the S3 bucket perhaps)? Thanks in advance for any pointers.

Hi, I have created and endpoint in sagemaker using boto3 but never finishes creation, is stuck in Creating status for few days now. I have tried to delete it using the aws cli api but i get the message: *An error occurred (ValidationException) when calling the DeleteEndpoint operation: Cannot update in-progress endpoint* Usually endpoint fails after some time and can deleted but this time doesn't fail. Is there any way to force deletion?

Hello. I'm using AWS Augmented AI to create Human review workflows, I do have custom templates set up and workflows configured, do have AWS worker portal configured and can authenticate there with Cognito identity. Now, whenever I start working on a Human Loop task with my custom template I'd like to know the identity of the worker currently working on a task. Is there a way I can get a cookie or auth code or the full JWT token from the task page? I explored both the page and the nested iframe but couldn't find any useful cookies or any other related information there. Long story short what I'm trying to make is to create a lambda behind the secured API Gateway endpoint that will return the information specific to a worker that is currently working on a Human Loop task and to render this information in my template. I thought I can get a JWT token from the cookies and use it to access the API Gateway. Can you please clarify whether I can do this or if any other options are available. Thanks.

Hello, i need some help regarding my question title. i am working on a project that needs to get a video stored on a website which i have created to work with my machine learning code which is running on pycharm it all seems to be working locally and schedulely but i want it to run it on cloud what type of instance do i need because the model i am using takes a lot of time on cpu and gpu based instance will certainly help. if you need more details regarding this let me know thanks.

I am following the v2 Lex Developer Guide to create the OrderFlowers Blueprint Lex. I followed the steps to add a Lamba function (page 145) in the guide (https://docs.aws.amazon.com/lexv2/latest/dg/lex2.0.pdf). When I execute the Lex Bot it throws the error below. Do you have any suggestions what is causing this? Thank You! Invalid Lambda Response: Your request to Amazon Lex couldn't be completed. Lambda function arn:aws:lambda:us-east-1:994504927747:function:OrderFlowersCodeHook:OrderFlowersCodeHook encountered a problem while processing Amazon Lex request 4c270a5e-4883-4df4-98b7-996be1cbb627. The error message from the Lambda function is Unhandled. Check the Lambda function log for error details, then try your request again after fixing the error.

> The problem is related to registering a **AWS DeepLens v1.1 Camera** I am failing to register a AWS Deeplens camera using machines running Ubuntu 20.4 (focal) and Mint 19.3 (tessa) or ubuntu bionic, with the error: **Device not detected**. This is when trying to follow the *Register an AWS DeepLens v1.1 device * steps on the AWS web console. Trying to go via other forums, I am made to believe that there is a need to have the **awscam** package installed on the linux machines. I ama failing to install the package as well, with the error below. Is the a awscam package for debian distributions, and more precisely for Ubuntu 20.4? **sudo apt-get install awscam** ``` sudo apt-get install awscam Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package awscam ``` **lsb_release -a** ``` lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.4 LTS Release: 20.04 Codename: focal ```

Hi, I am working on Hugging Face example in Sagemaker labs, and I got the below error "ResourceLimitExceeded: An error occurred (ResourceLimitExceeded) when calling the CreateTrainingJob operation: The account-level service limit 'ml.p3.2xlarge for training job usage' is 0 Instances, with current utilization of 0 Instances and a request delta of 1 Instances. Please contact AWS support to request an increase for this limit." Code: Can you please enable 1 instance to help me continue with Sagemaker Lab example of Hugging face model. Thank you

Hi, I'm trying to put a lexicon in Amazon Polly console in order to censor some stopwords. My .xml file is accepted, but there is no result with the spoken input. This is my script <?xml version="1.0" encoding="UTF-8"?> <lexicon version="1.0" xmlns="http://www.w3.org/2005/01/pronunciation-lexicon" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2005/01/pronunciation-lexicon http://www.w3.org/TR/2007/CR-pronunciation-lexicon-20071212/pls.xsd" alphabet="ipa" xml:lang="it-IT"> <lexeme> <grapheme>parola uno</grapheme> <alias></alias> </lexeme> <lexeme> <grapheme>parola due</grapheme> <alias></alias> </lexeme> <lexeme> <grapheme>parola tre</grapheme> <alias></alias> </lexeme> <lexeme> <grapheme>parola quattro</grapheme> <alias></alias> </lexeme> </lexicon> ******* I can't understand the error, Thank you very much if you can help me!

Any expert that is familiar with the AI topics for QA Automation and Rendering optimization?

Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning (ML) from weeks to minutes in Amazon SageMaker Studio, the first fully integrated development environment (IDE) for ML. With SageMaker Data Wrangler, you can simplify the process of data preparation and feature engineering, and complete each step of the data preparation workflow, including data selection, cleansing, exploration, and visualization, from a single visual interface. SageMaker Data Wrangler runs on ml.m5.4xlarge by default. SageMaker Data Wrangler includes built-in data transforms and analyses written in PySpark so you can process large data sets (up to hundreds of gigabytes (GB) of data) efficiently on the default instance. Starting today, you can use additional M5 or R5 instance types with more CPU or memory in SageMaker Data Wrangler to improve performance for your data preparation workloads. Amazon EC2 M5 instances offer a balance of compute, memory, and networking resources for a broad range of workloads. Amazon EC2 R5 instances are the memory optimized instances. Both M5 and R5 instance types are well suited for CPU and memory intensive applications such as running built-in transforms for very large data sets (up to terabytes (TB) of data) or applying custom transforms written in Panda on medium data sets (up to tens of GBs). To learn more about the newly supported instances with Amazon SageMaker Data Wrangler, visit the [blog ](https://aws.amazon.com/blogs/machine-learning/process-larger-and-wider-datasets-with-amazon-sagemaker-data-wrangler/) or the [AWS document](https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler-data-flow.html), and the[ pricing page](https://aws.amazon.com/sagemaker/pricing/). To get started with SageMaker Data Wrangler, visit the [AWS documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler.html).

I have created a chatbot using AWS Lex and I have integrated it with slack and they respond to the mentions in channels(control via lambda). Currently, the bot responds as a new message however I want it to respond in the slack thread of the original message posted by the user. Does anyone have any idea on how to solve this? Really appreciate your help, thanks.

hello everyone, i want to deploy a pretrained model on the AWS Server using endpoint so that i can fetch it on the mobile device. i am also sharing the github link of the model which i want to deploy on the aws. its the pretrained object detection of tensorflow. For further explanation you can ask me. https://github.com/tauqueerdanish/object_detection/blob/main/Task_01.ipynb

I've got two CSV files that are loaded into Data Wrangler that are intended to augment each other. The tables have some columns that are the same (in name) and some that are not, many of the rows are missing entries for many of the columns. The two tables represent separate datasets. Consider the example below: Table 1: | Filename | LabelA | LabelB | | --- | --- | --- | | ./A/001.dat | 1 | 1 | | ./A/002.dat | 0 | 1 | Table 2: | Filename | LabelB | LabelC | | --- | --- | --- | | ./B/001.dat | | 0 | | ./B/002.dat | 0 | 1 | I am looking to merge / concatenate the two table. The problem is that neither Data Wrangler join nor concatenate seems to work (at least as expected). Desired result: | Filename | LabelA | LabelB | LabelC | | --- | --- | --- | --- | | ./A/001.dat | 1 | 1 | | | ./A/002.dat | 0 | 1 | | | ./B/001.dat | | | 0 | | ./B/002.dat | | 0 | 1 | When using a "Full Outer" join and ask to combine "Filename" and "LabelB" columns, it will take all the values from Table 1 OR Table 2 even if Table 1 does not have that entry (for example, some rows will have Filename = <nothing> rather than Filename = ./B/001.dat). When using concatenate, Data Wrangler errors on the fact that it cannot match EVERY column between the tables. Now in my example there are many columns and many rows which precludes a manual process of joining without merging columns and then going through a renaming and merging process one-by-one. How do get these tables to simply merge? I feel I must be missing something obvious. I am about to give up on Data Wrangler and do it all in a python script using pandas, but I thought I should give Data Wrangler a try while learning the MLops process.

I am trying to create a custom slot that can recognise time format `10:00:00` for example, so i created a custom slot "extended slot type" and tried using regex like this `[0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2}` it failed with "Check the regular expression and try your request again" error so i tried using the unicode character for "colon" like `[0-9]{1,2}\u003A[0-9]{1,2}\u003A[0-9]{1,2}` even this fails with the same error i tried using other unicode and special characters like `-` which works absolutely fine. so, How can i use a colon special character in regex validation ?

Sagemaker Canvas allows updating files in S3 bucket either by drag'n drop or using "Select files from your computer" features. When the s3 bucket used (similar to sagemaker-<region>-<account-number>) in encrypted with custom KMS key, the upload fails with "Upload fails" (HTTP 400). Is using KMS key encrypted bucket possible? We already have a KMS key policy in place allowing these actions for canvas IAM role: "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:Encrypt*", "kms:Describe*", "kms:Decrypt*" Something else needed?

Unable to extract fields from Analyze Expense Demo for pdfs. However, same file in jpeg or png format is working fine. For pdf, getting "We can’t find any summary fields"

I am using AWS Sagemaker services and delete any instances that belong to the services. However, i still get charged per day even though I dont use the service anymore. I sent the case to the customer support center but no clear explanation yet. Please any kind of help for my account to stop the cost.

I am using AWS Sagemaker services and delete any instances that belong to the services. However, i still get charged per day even though I dont use the service anymore. I sent the case to the customer support center but no clear explanation yet. Please any kind of help for my account to stop the cost.

I have a scanned PDF with handwritten text and I am trying to search through it using AWS Kendra. However, Kendra doesn't search through the handwritten text. I tried using textract and comprehend, but there seem to be limitations to PDF size and number of pages with both services. Is there a way to allow Kendra to search through this handwritten text?

How can you use the functions of SageMaker Canvas with python or outside the user interface?

We are working on Automation of CCTV camera videos (RTSP link) using image processing techniques such as back ground subtraction, contour detection, annotation using opencv-python for 80 sites.so we have to run 80 python scripts in parallel manner. I would like to know which AMI suits for this project without any drop in outputs. The outputs are images in jpg format. These code would run 24*7.kindly suggest best AMI for this so that we would suggest the same to our management for purchase.

After create model in Redshift i can load the generated model in python via Sagemaker. The features are generated automatically e.g. f0, f1 and so on. Altough i have only 10 columns i get maybe 500 features. This comes from some categories but where can i see the mapping of the features to the original category names? For example f100 is the category "XY" of column 5.

What I want to do is from the domain name vapeprivate.com make vapeprivate.com/fr, vapeprivate.com/es, vapeprivate.com/it to create several versions of the website at the language level. Can you give me the example of how to do in my case concretely. Currently I have a bucket created in the name of vapeprivate.com connected to the domain name vapeprivate.com with route 53

I run neural network tensorflow train on studiolab. and I got: ``` Epoch 145/4000 1941/1941 - 10s - ... - 10s/epoch - 5ms/step ``` then I try to make a train job with script_mode with `ml.c5.xlarge` ``` estimator = TensorFlow(entry_point='untitled.py', source_dir='./training/', instance_type='ml.c5.xlarge', instance_count=1, output_path="s3://sagemaker-[skip]", role=sagemaker.get_execution_role(), framework_version='2.8.0', py_version='py39', hyperparameters={...}, metric_definitions=[...], script_mode=True) ``` and its got: ``` Epoch 19/4000 1941/1941 - 49s - ... - 49s/epoch - 25ms/step ``` Why is it 5 times slower than studiolab notebook? Is it because instance type?

After creating and finishing the model training in Redshift we can predict with the new model. How can we see the target function of the model or feature importance of the single columns which were used as an input? To describe it in another way we want to see the correlations to the target column after the model training.

Hi everyone! Currently, I've accomplished detecting all the faces from a collection and then generating sub-galleries of each individual subject with all their photos associated with the ruby SDK '~> 1.65' In order to do this, I've indexed the faces of all photos within a collection, list all the faces (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#list_faces-instance_method), then grabbing each face_id recognized and search the faces related to that face_id (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#search_faces-instance_method), and delete the face id used to do the API call and all the returned ones to tell when a new detected subject starts and ends. My issue is that some indexed faces are returning a wrong bound box not fitting the face properly with a few images, sometimes not even close to the face (I've seen a brick as a bound box). This is weird because internally those "wrong faces" seem to be associated properly with other faces as if the service were just returning wrong box values but doing all the AI jobs properly. Is there a way to prevent or tweak the call params to prevent this? or is it normal for the API to return sometimes wrong values due to the multiple decimals? Thanks in advance!

Hi everyone! Currently, I've accomplished detecting all the faces from a collection and then generating sub-galleries of each subject with all their photos associated with the ruby SDK '~> 1.65' To do this, I've indexed the faces of all photos within a collection, list all the faces (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#list_faces-instance_method), then grabbing each face_id recognized and search the faces related to that face_id (https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Rekognition/Client.html#search_faces-instance_method), and delete the face id used to do the API call and all the returned ones to tell when a new detected subject starts and ends. My issue is that the search faces API returns different results depending on which face id param you are doing the request with. For example, if there are 10 faces ids detected that belong to a person (1, 2, 3, ..., 10), the search faces call with face id = 1 param, should return the faces id (2, 3, 4, ..., 10) but if you continue to do this with the other face ids this is not always the case with some scenarios where the search faces call with face id = 3 has returned a subset of the previously mentioned like just (4, 5, 6). Is there any other way to achieve this to prevent this kind of "error"? if not, this is a real concern for us because it depends on the order in which we call the search faces with different face ids, and sometimes it seems like there is more than 1 subject detected with almost the same photos when in reality it's the same person. Thanks in advance!

In a sagemaker notebook with an associated git repository, when I try to create a boto3 session client using verify = False, I get the following : `SSLError: SSL validation failed for {service_name } [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)` The error resolves if I allow the default value of verify = None (meaning SSL certs are verified). My problem is that this session is created as part of a function call on the associated git repository and I don't want to change the behavior in the repo. I don't understand why this error occurs only when I specify *not* to validate SSL certificates. Any ideas of what explains this behavior?

Hello, I'd like to deploy SageMaker's built-in algorithm, BlazingText model on Fargate instead of Sagemaker endpoint. So, I tried to make an ECS task using BlazingText docker path. Here is my CDK code for it. const loadBalancedFargateService = new ecsPatterns.ApplicationLoadBalancedFargateService(this, 'Service', { memoryLimitMiB: 1024, desiredCount: 1, cpu: 512, taskImageOptions: { image: ecs.ContainerImage.fromRegistry("811284229777.dkr.ecr.us-east-1.amazonaws.com/blazingtext:1"), }, }); However, I got an error: CannotPullContainerError: inspect image has been retried 1 time(s): failed to resolve ref "811284229777.dkr.ecr.us-east-1.amazonaws.com/blazingtext:1": pulling from host 811284229777.dkr.ecr.us-east-1.amazonaws.com failed with status code [manifests 1]... Is it impossible to pull docker container of sagemaker built-in algorithm from ECS?

I need to extract text from some .PDFs documents. The formatting of these texts varies (column where it starts and ends) but the beginning and end are always similar, it starts with the word **DECRETO** and ends with the word **CHEFE + something** In this link you can see an example of the original text: https://i.postimg.cc/Xvk0TXJ9/texto.png Is it possible to do this with AWS tools? What's the best way? **OBS: Text language is PT-BR**

I'm going through the process of registering my DeepLens. The moment I go to the page for registering my lens with V1, I get a message of "The security token included in the request is invalid". If I instead go through the V1.1 process, I get the same message after hooking up my DeepLens, and setting up the internet. Under either path, the lens fails to register. Any ideas? **Trouble Shooting So Far:** + Tried issuing new security key + Tried using root account and IAM role + Tried clearing cache + Tried using a different account + Tried using a different computer

Hello Community, Does the following AWS services: CloudWatch RUM CloudWatch Evidently AWS DevOps Guru CodeGuru Profiler Are SOC 1,2,3 Supported? or any plan to be supported in thew future? They don't appear in the updated list of the AWS Services in Scope by [Compliance Program Page](https://aws.amazon.com/compliance/services-in-scope/).

After following all of the steps outlined in the LexV2 - Slack integration developer guide (https://docs.aws.amazon.com/lexv2/latest/dg/deploy-slack.html), Lex is not receiving any messages sent to our created bot within Slack. When testing Lex in the console, I receive the expected output. However, there is no response from the bot when a message is sent in Slack, and the Fulfillment Lambda function never even receives the event (even without a Fulfillment Lambda, no response from Lex via Slack). I have correctly set up the channel integration, providing the client id, client secret, and verification token from my created Slack app. The channel is configured on an alias which is tied to the latest version of my Lex bot. My team has some existing bots that were configured in the same way months ago, and they are still functional. I copied all possible configuration from those Slack-Lex integrations to no avail. I have verified that the im_message event is being sent by Slack by providing an alternative event subscription request URL. However, the callback URLs from the Slack channel integration do not work. Any help would be appreciated, thanks.

I have a private GitLab and would like to connect it to Sagemaker Studio and/or Notebooks. Is that supported? What is it is through a VPC? If so, how can it be done?

I have a sagemaker notebook instance having two jupyter notebook ipynb files. When I had one jupyter notebook, I was able to run it automatically with one lambda function trigger and lifecycle configuration. Now I have two jupyter notebooks and corresponding two lambda function triggers. How can I run them based on the trigger by changing the lifecycle configuration script. The trigger is file uploading into S3. Based on what location the file is added, the corresponding jupyter notebook should run

I have a sagemaker notebook instance having two jupyter notebook ipynb files. When I had one jupyter notebook, I was able to run it automatically with one lambda function trigger and lifecycle configuration. Now I have two jupyter notebooks and corresponding two lambda function triggers. How can I run them based on the trigger by changing the lifecycle configuration script.

We are trying to deploy a Lambda with some code which works in a Notebook. The code is rather simple and uses D-Wave — DW_2000Q_6. The problem is that when we execute the lambda (container lambda due to size problems), it give us the following error: ```json { "errorMessage": "[Errno 30] Read-only file system: '/home/sbx_user1051'", "errorType": "OSError", "stackTrace": [ " File \"/var/lang/lib/python3.8/imp.py\", line 234, in load_module\n return load_source(name, filename, file)\n", " File \"/var/lang/lib/python3.8/imp.py\", line 171, in load_source\n module = _load(spec)\n", " File \"<frozen importlib._bootstrap>\", line 702, in _load\n", " File \"<frozen importlib._bootstrap>\", line 671, in _load_unlocked\n", " File \"<frozen importlib._bootstrap_external>\", line 843, in exec_module\n", " File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n", " File \"/var/task/lambda_function.py\", line 6, in <module>\n from dwave.system.composites import EmbeddingComposite\n", " File \"/var/task/dwave/system/__init__.py\", line 15, in <module>\n import dwave.system.flux_bias_offsets\n", " File \"/var/task/dwave/system/flux_bias_offsets.py\", line 22, in <module>\n from dwave.system.samplers.dwave_sampler import DWaveSampler\n", " File \"/var/task/dwave/system/samplers/__init__.py\", line 15, in <module>\n from dwave.system.samplers.clique import *\n", " File \"/var/task/dwave/system/samplers/clique.py\", line 32, in <module>\n from dwave.system.samplers.dwave_sampler import DWaveSampler, _failover\n", " File \"/var/task/dwave/system/samplers/dwave_sampler.py\", line 31, in <module>\n from dwave.cloud import Client\n", " File \"/var/task/dwave/cloud/__init__.py\", line 21, in <module>\n from dwave.cloud.client import Client\n", " File \"/var/task/dwave/cloud/client/__init__.py\", line 17, in <module>\n from dwave.cloud.client.base import Client\n", " File \"/var/task/dwave/cloud/client/base.py\", line 89, in <module>\n class Client(object):\n", " File \"/var/task/dwave/cloud/client/base.py\", line 736, in Client\n @cached.ondisk(maxage=_REGIONS_CACHE_MAXAGE)\n", " File \"/var/task/dwave/cloud/utils.py\", line 477, in ondisk\n directory = kwargs.pop('directory', get_cache_dir())\n", " File \"/var/task/dwave/cloud/config.py\", line 455, in get_cache_dir\n return homebase.user_cache_dir(\n", " File \"/var/task/homebase/homebase.py\", line 150, in user_cache_dir\n return _get_folder(True, _FolderTypes.cache, app_name, app_author, version, False, use_virtualenv, create)[0]\n", " File \"/var/task/homebase/homebase.py\", line 430, in _get_folder\n os.makedirs(final_path)\n", " File \"/var/lang/lib/python3.8/os.py\", line 213, in makedirs\n makedirs(head, exist_ok=exist_ok)\n", " File \"/var/lang/lib/python3.8/os.py\", line 213, in makedirs\n makedirs(head, exist_ok=exist_ok)\n", " File \"/var/lang/lib/python3.8/os.py\", line 223, in makedirs\n mkdir(name, mode)\n" ] } ``` It seems that the library tries to write to some files which are not in /tmp folder. I'm wondering if is possible to do this, and if not, what are the alternatives. imports used: ```python import boto3 from braket.ocean_plugin import BraketDWaveSampler from dwave.system.composites import EmbeddingComposite from neal import SimulatedAnnealingSampler ```

Can we play a notification sound for every new message both at client and agent side in AWS Chatbot. Do we have some inbuilt capability in Amazon CCP

Note: I am a senior developer, but am very new to the topic of machine learning. I have two frozen TensorFlow model weight files: `weights_face_v1.0.0.pb` and `weights_plate_v1.0.0.pb`. I also have some python code using Tensorflow 2, that loads the model and handles basic inference. The models detect respectively faces and license plates, and the surrounding code converts an input image to a numpy array, and applies blurring to the images in areas that had detections. I want to get a SageMaker endpoint so that I can run inference on the model. I initially tried using a regular Lambda function (container based), but that is too slow for our use case. A SageMaker endpoint should give us GPU inference, which should be much faster. I am struggling to find out how to do this. From what I can tell reading the documentation and watching some YouTube video's, I need to create my own docker container. As a start, I can use for example `763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.8.0-gpu-py39-cu112-ubuntu20.04-sagemaker`. However, I can't find any solid documentation on how I would implement my other code. How do I send an image to SageMaker? Who tells it to convert the image to numpy array? How does it know the tensor names? How do I install additional requirements? How can I use the detections to apply blurring on the image, and how can I return the result image? Can someone here please point me in the right direction? I searched a lot but can't find any example code or blogs that explain this process. Thank you in advance! Your help is much appreciated.

I have a CSV file, on S3, in the same region, but when I'm trying to run an autopilot experiment, I'll get the following error: "Unable to fetch column headers. Your dataset file must be a .csv or .parquet file and currently, be in your S3 bucket." Can someone please help me with how I should fix this issue? Thank you

As per standard entity recognition from AWS Comprehend, it recognizes all currency, numbers, duration as QUANTITY - Is there anyway to get category of the quantity which AWS Comprehend is recognizing?

Hi, Greetings!! I found some errors in cloud watch logs while executing a training job via SageMake pipelines but unfortunately training job did not fail. Hence I tried stopping a training job using boto3 APIs below and AWS CLI as well but training job is in stopping status for a long time and it's not stopping. stop_pipeline_execution() stop_training_job() stop-training-job --training-job-name <value> How to kill the training jobs forcefully? Thanks

Hello, I have submitted an entry to Deep Racer April Open league from a temporary AWS account which has been deleted/removed. How I get the entry (or the model) moved/ported to my new/permanent AWS account. I cannot access the temporary AWS account anymore nor I have saved my model to submit again from new/permanent account. Please help. Regards, Raghavendra

Our task interfaces in Ground Truth have links to additional content that an annotator can use to validate additional information for the task. We've included these as links with target=_blank but they don't work because the labeling interface hasn't enabled the 'allow-popups' permission for the iframe. Having additional detail as a link to a new tab is a really common pattern in annotation tasks; is there an alternate way to do this?

Assume a short video recommendation system with a million short videos and users. The users can watch & like videos. Using a receipt such as USER_PERSONALIZATION (Users, Video as items, and Watched/Liked as interactions) the model will include thousands of interactions per user. To filter out the watched videos before recommending videos to the user, we can use Amazon Filter with: EXCLUDE ItemID WHERE Interactions.EVENT_TYPE IN ("watched"). However according to https://docs.aws.amazon.com/personalize/latest/dg/frequently-asked-questions.html "...when filtering, Amazon Personalize considers up to 200 historical interactions for a user, and up to 100 streamed interactions". Therefore, for such use case, we might still recommend watched videos, as there will be more than 200 watched videos per user and the filter only considers up to 200 watched video (interactions). Does it mean that Amazon Personalize is not considered a suitable solution for short video apps?

I have been creating a customer journey using V2 to ask for the reason for a call and then an account number. In V2 when calling from lex the additional required slot of account number is not executed. So the question of account number is never requested from the customer. I thought I was something I was doing but reverted to a test in V1 and it worked fine. Has anyone also come across this or am I missing something in version 2?

We have several desktop (Win/Mac/Nix) monolithic applications that generate JSON events exposing these events via POST. Currently it is a one way deal, outbound from app to the stream for nothing but analytics. We are now looking to analyze those events to determine if there are actionable "data sequences" that start to appear. **My Question is (and opinions are welcomed...)** **What is the most efficient (#1 - performant, #2 - cost effective, #3 - resilient) way to stream those records (250-500 byte msg, at a peak of 1,000 / second) to AWS for the purpose of:** 1. Transforming the record data to a known model (i.e. std date format, simple XForms) 2. Analyzing the normalized record (ML - predictive analytics or iterative learning) 3. Persisting the record 4. Based on the analysis, sending back a payload (or pointer to a CDN stored payload) to the original calling app? In a nutshell looking to do a type of on-preises, *near real-time* application performance analytics of the applications using a single on-premises service spewing JSON out to AWS native cloud services that do all teh work, and return info as timely as possible. Thanks. <><

I am having difficulty in getting started on SageMaker Studio using the Standard setup wizard of SageMaker Domain creation. The wizard prevents me from moving forward at the RStudio setup step even though RStudio is indicated to be optional and which I don't need or want to pay for. I have tried a lot of things (looked at the videos, searched the web) to no avail. Tried as a root user as well as a IAM user and have the same issue. Thanks for any help.

I'm trying to register my AWS DeepLens v1.0 but I'm failing at step 1 "Autorizations". As soon as I load the page, there's a red panel showing "The security token included in the request is invalid". And if I enter a name then click the "create roles" button, same thing. The browser JS console is showing "POST https://iam.amazonaws.com/ 403", so I suppose I could manually add some roles in my IAM user profile, but I'm new with AWS, and don't have any clue. Can someone help me ?

I'm having trouble using the GPU for a Deep learning GPU EC2 instance. The specs of the instance are: - Deep Learning AMI GPU PyTorch 1.11.0 (Amazon Linux 2) 20220328 - amazon/Deep Learning AMI GPU PyTorch 1.11.0 (Amazon Linux 2) 20220328 When I log into the instance and I run `nvidia smi`, I get the error: `NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.` Similarly, when I run a pytorch (pre-installed) command to check whether it can see a GPU, it returns False: `(pytorch) [ec2-user@ip-172-31-86-58 ~]$ python3` `Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:25:59)` `[GCC 10.3.0] on linux` `Type "help", "copyright", "credits" or "license" for more information.` `>>> import torch` `>>> torch.cuda.is_available()` `False` The GPU set up should have worked out of the box but how do I fix this?