By using AWS re:Post, you agree to the Terms of Use

DevOps

AWS provides a set of flexible services designed to enable companies to more rapidly and reliably build and deliver products using AWS and DevOps practices. These services simplify provisioning and managing infrastructure, deploying application code, automating software release processes, and monitoring your application and infrastructure performance.

Recent questions

see all
1/18

Using HuggingFace in Sagemaker Studio as part of a project

TLDR: if we are trying to use a HuggingFaceProcessor/Estimator in a Sagemaker Studio project, what are the requirements for the `train.py` file in terms of how it refers to the assembled training data, and where it should store the results of the operations it performs( e.g. compiled model, datae etc.) ----------------------- FULL DETAILS ------------------------ So our high level goal is to be able to deploy some kind of non-XGB model from a sagemaker studio project, given that the templates provided are all XGB. As outlined in [an earlier question](https://repost.aws/questions/QUdd2zOBY0Q4CEG1ZdbgNsgA/using-transformers-module-with-sagemaker-studio-project-module-not-found-error-no-module-named-transformers) we'd started with TensorFlow, but since our TensorFlow model wraps a HuggingFace model we thought let's try something even simpler, just a HuggingFace model using the HuggingFaceProcessor. So following docs on [HuggingFaceProcessor](https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job-frameworks-hugging-face.html) and a [HuggingFace Estimator](https://github.com/huggingface/notebooks/blob/main/sagemaker/02_getting_started_tensorflow/sagemaker-notebook.ipynb) example we started to adjust the abalone (project template) pipeline.py to look like this (full code can be provided on request): ``` # processing step for feature engineering hf_processor = HuggingFaceProcessor( role=role, instance_count=processing_instance_count, instance_type=processing_instance_type, transformers_version='4.4.2', pytorch_version='1.6.0', base_job_name=f"{base_job_prefix}/frameworkprocessor-hf", sagemaker_session=pipeline_session, ) step_args = hf_processor.run( outputs=[ ProcessingOutput(output_name="train", source="/opt/ml/processing/train"), ProcessingOutput(output_name="validation", source="/opt/ml/processing/validation"), ProcessingOutput(output_name="test", source="/opt/ml/processing/test"), ], code=os.path.join(BASE_DIR, "preprocess.py"), arguments=["--input-data", input_data], ) step_process = ProcessingStep( name="PreprocessTopicData", step_args=step_args, ) # training step for generating model artifacts model_path = f"s3://{sagemaker_session.default_bucket()}/{base_job_prefix}/TopicTrain" hf_train = HuggingFace(entry_point='train.py', source_dir=BASE_DIR, base_job_name='huggingface-sdk-extension', instance_type=processing_instance_type, instance_count=processing_instance_count, transformers_version='4.4', pytorch_version='1.6', py_version='py36', role=role, ) hf_train.set_hyperparameters( epochs=3, train_batch_size=16, learning_rate=1.0e-5, model_name='distilbert-base-uncased', ) step_args = hf_train.fit( inputs={ "train": TrainingInput( s3_data=step_process.properties.ProcessingOutputConfig.Outputs[ "train" ].S3Output.S3Uri, content_type="text/csv", ), "validation": TrainingInput( s3_data=step_process.properties.ProcessingOutputConfig.Outputs[ "validation" ].S3Output.S3Uri, content_type="text/csv", ), }, ) ``` Finding that pushing to master doesn't provide any feedback on issues arising from pipeline.py, we realised that trying to get the pipeline from a notebook was a better way of debugging these sorts of changes, assuming one remembered to restart the kernel each time to ensure changes to the pipeline.py file was available to the notebook. So using the following code in the notebook we worked through a series of issues trying to bash the code into shape such that it would compile: ``` from pipelines.topic.pipeline import get_pipeline pipeline = get_pipeline( region=region, role=role, default_bucket=default_bucket, model_package_group_name=model_package_group_name, pipeline_name=pipeline_name, ) ``` We needed to change the default processing and training instance types to avoid a "cpu" unsupported issue: ``` processing_instance_type="ml.p3.xlarge", training_instance_type="ml.p3.xlarge", ``` and add a train.py script: ``` from transformers import AutoTokenizer from transformers import TFAutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased") model = TFAutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased", num_labels=18) import pandas as pd import tensorflow as tf from sklearn.model_selection import train_test_split from transformers import ( DistilBertTokenizerFast, TFDistilBertForSequenceClassification, ) DATA_COLUMN = 'text' LABEL_COLUMN = 'label' MAX_SEQUENCE_LENGTH = 512 LEARNING_RATE = 5e-5 BATCH_SIZE = 16 NUM_EPOCHS = 3 NUM_LABELS = 15 if __name__ == "__main__": # -------------------------------------------------------------------------------- # Tokenizer # -------------------------------------------------------------------------------- tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased') def tokenize(sentences, max_length=MAX_SEQUENCE_LENGTH, padding='max_length'): """Tokenize using the Huggingface tokenizer Args: sentences: String or list of string to tokenize padding: Padding method ['do_not_pad'|'longest'|'max_length'] """ return tokenizer( sentences, truncation=True, padding=padding, max_length=max_length, return_tensors="tf" ) # -------------------------------------------------------------------------------- # Load data # -------------------------------------------------------------------------------- from keras.utils import to_categorical from sklearn.preprocessing import LabelEncoder labelencoder_Y_1 = LabelEncoder() yy = labelencoder_Y_1.fit_transform(train_data[LABEL_COLUMN].tolist()) yy = to_categorical(yy) print(len(yy)) print(yy.shape) train_dat, validation_dat, train_label, validation_label = train_test_split( train_data[DATA_COLUMN].tolist(), yy, test_size=0.2, shuffle=True ) # -------------------------------------------------------------------------------- # Prepare TF dataset # -------------------------------------------------------------------------------- train_dataset = tf.data.Dataset.from_tensor_slices(( dict(tokenize(train_dat)), # Convert BatchEncoding instance to dictionary train_label )).shuffle(1000).batch(BATCH_SIZE).prefetch(1) validation_dataset = tf.data.Dataset.from_tensor_slices(( dict(tokenize(validation_dat)), validation_label )).batch(BATCH_SIZE).prefetch(1) # -------------------------------------------------------------------------------- # training # -------------------------------------------------------------------------------- model = TFDistilBertForSequenceClassification.from_pretrained( 'distilbert-base-uncased', num_labels=NUM_LABELS ) optimizer = tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE) model.compile( optimizer=optimizer, loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True), ) ``` However we are now stuck on this error when trying to get the pipeline from a notebook. ```TypeError Traceback (most recent call last) <ipython-input-3-be38b3dda75f> in <module> 7 default_bucket=default_bucket, 8 model_package_group_name=model_package_group_name, ----> 9 pipeline_name=pipeline_name, 10 ) 11 # !conda list ~/topic-models-no-monitoring-p-rboparx6tdeg/sagemaker-topic-models-no-monitoring-p-rboparx6tdeg-modelbuild/pipelines/topic/pipeline.py in get_pipeline(region, sagemaker_project_arn, role, default_bucket, model_package_group_name, pipeline_name, base_job_prefix, processing_instance_type, training_instance_type) 228 "validation" 229 ].S3Output.S3Uri, --> 230 content_type="text/csv", 231 ), 232 }, /opt/conda/lib/python3.7/site-packages/sagemaker/workflow/pipeline_context.py in wrapper(*args, **kwargs) 246 return self_instance.sagemaker_session.context 247 --> 248 return run_func(*args, **kwargs) 249 250 return wrapper /opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in fit(self, inputs, wait, logs, job_name, experiment_config) 1059 self._prepare_for_training(job_name=job_name) 1060 -> 1061 self.latest_training_job = _TrainingJob.start_new(self, inputs, experiment_config) 1062 self.jobs.append(self.latest_training_job) 1063 if wait: /opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in start_new(cls, estimator, inputs, experiment_config) 1956 train_args = cls._get_train_args(estimator, inputs, experiment_config) 1957 -> 1958 estimator.sagemaker_session.train(**train_args) 1959 1960 return cls(estimator.sagemaker_session, estimator._current_job_name) /opt/conda/lib/python3.7/site-packages/sagemaker/session.py in train(self, input_mode, input_config, role, job_name, output_config, resource_config, vpc_config, hyperparameters, stop_condition, tags, metric_definitions, enable_network_isolation, image_uri, algorithm_arn, encrypt_inter_container_traffic, use_spot_instances, checkpoint_s3_uri, checkpoint_local_path, experiment_config, debugger_rule_configs, debugger_hook_config, tensorboard_output_config, enable_sagemaker_metrics, profiler_rule_configs, profiler_config, environment, retry_strategy) 611 self.sagemaker_client.create_training_job(**request) 612 --> 613 self._intercept_create_request(train_request, submit, self.train.__name__) 614 615 def _get_train_request( # noqa: C901 /opt/conda/lib/python3.7/site-packages/sagemaker/session.py in _intercept_create_request(self, request, create, func_name) 4303 func_name (str): the name of the function needed intercepting 4304 """ -> 4305 return create(request) 4306 4307 /opt/conda/lib/python3.7/site-packages/sagemaker/session.py in submit(request) 608 def submit(request): 609 LOGGER.info("Creating training-job with name: %s", job_name) --> 610 LOGGER.debug("train request: %s", json.dumps(request, indent=4)) 611 self.sagemaker_client.create_training_job(**request) 612 /opt/conda/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw) 236 check_circular=check_circular, allow_nan=allow_nan, indent=indent, 237 separators=separators, default=default, sort_keys=sort_keys, --> 238 **kw).encode(obj) 239 240 /opt/conda/lib/python3.7/json/encoder.py in encode(self, o) 199 chunks = self.iterencode(o, _one_shot=True) 200 if not isinstance(chunks, (list, tuple)): --> 201 chunks = list(chunks) 202 return ''.join(chunks) 203 /opt/conda/lib/python3.7/json/encoder.py in _iterencode(o, _current_indent_level) 429 yield from _iterencode_list(o, _current_indent_level) 430 elif isinstance(o, dict): --> 431 yield from _iterencode_dict(o, _current_indent_level) 432 else: 433 if markers is not None: /opt/conda/lib/python3.7/json/encoder.py in _iterencode_dict(dct, _current_indent_level) 403 else: 404 chunks = _iterencode(value, _current_indent_level) --> 405 yield from chunks 406 if newline_indent is not None: 407 _current_indent_level -= 1 /opt/conda/lib/python3.7/json/encoder.py in _iterencode_dict(dct, _current_indent_level) 403 else: 404 chunks = _iterencode(value, _current_indent_level) --> 405 yield from chunks 406 if newline_indent is not None: 407 _current_indent_level -= 1 /opt/conda/lib/python3.7/json/encoder.py in _iterencode(o, _current_indent_level) 436 raise ValueError("Circular reference detected") 437 markers[markerid] = o --> 438 o = _default(o) 439 yield from _iterencode(o, _current_indent_level) 440 if markers is not None: /opt/conda/lib/python3.7/json/encoder.py in default(self, o) 177 178 """ --> 179 raise TypeError(f'Object of type {o.__class__.__name__} ' 180 f'is not JSON serializable') 181 TypeError: Object of type ParameterInteger is not JSON serializable ``` Which is telling us that some aspect of the training job (?) is not serializable, and it's not clear how to debug further. What would be enormously helpful is project templates for sagemaker studio showing the use of all the Processors, e.g. HuggingFace, TensorFlow and so on, but failing that we'd be most grateful is anyone could point us to documentation on what the requirements are for the `train.py` file that we need to specifiy for the HuggingFace Estimator. many thanks in advance
0
answers
0
votes
5
views
asked 2 hours ago

Cannot add environment variable through Ebextensions

I'm using .ebextensions to create VPCEndpoints so in the **Resources** section I've addded the needed section for the VPCEndpoint. Then after that in the **option_settings** section I'm trying to add an environment variable in my elastic beanstalk application referencing the created VPCEndpoint, but when i check the environment variables from the elastic beanstalk console the value is added as a plain text not the Ref of the VPCEndpoint (Check the screenshot) So how can i make it interpret the Ref of the endpoint ? ![Enter image description here](/media/postImages/original/IMkQEkAlsLRyCG5pYs1i8hkA) ``` Resources: NewsonarVPCEndpoint: Type: AWS::EC2::VPCEndpoint Properties: PrivateDnsEnabled: false SecurityGroupIds: - {"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "ALLOW_INBOUND_FROM_VPC_SECURITY_GROUP", "DefaultValue": "default_value"}} ServiceName: { "Fn::Join": [ "", [ "com.amazonaws.vpce.",{"Fn::GetOptionSetting": {"Namespace": "aws:elasticbeanstalk:application:environment", "OptionName": "AWS_REGION", "DefaultValue": "us-east-1"}},".",{"Ref": "sonarVPCEndpointService"}]] } SubnetIds: - { "Ref": "Subnet1Id" } - { "Ref": "Subnet2Id" } - { "Ref": "Subnet3Id" } VpcEndpointType: Interface VpcId: { "Ref": "VpcId" } option_settings: aws:elasticbeanstalk:application:environment: VPC_ENDPOINT: '`{"Ref" : "NewsonarVPCEndpoint"}`' ```
0
answers
0
votes
4
views
asked 6 hours ago

Automatically stop CodeDeploy ECS Blue/Green deployment on unhealthy containers

We are writing a CI/CD setup where we remotely trigger a CodePipeline pipeline which fetches its task definition and appspec.yaml from S3 and includes a CodeDeploy ECS Blue/Green step for updating an ECS service. Images are pushed to ECR also remotely. This setup works and if the to-be-deployed application is not faulty and well configured the deployment succeeds in under 5 minutes. However, if the application does not pass health checks, or the task definition is broken, CodeDeploy will continuously re-deploy this revision during its "Install" step without end, creating tens of stopped tasks in the ECS Service. According to some this should time out after an hour, however we have not tested this. What we would like to achieve is automatic stops and rollbacks of these failing deployments. Ideally CodeDeploy should try only once to deploy the application and if that fails, immediately cancel the deployment and thus the pipeline run. According to the AWS documentation no options for this exist in CodeDeploy or the appspec.yaml that we upload to S3, so we are unsure of how to configure this if it is at all possible. We had two wanted scenarios in mind: 1. After one health check failure, the deployment stops and rolls back; 2. The deployment times out after a period shorter than one hour; ideally < 10 minutes. We currently have no alarms attached to the CodeDeploy deployment group, but it was my understanding that these alarms only trigger before the installation step to verify that the deployment can proceed instead of running alongside the deployment. In short; how would we configure either of those scenarios or at least prevent CodeDeploy from endlessly deploying replacement task sets?
0
answers
0
votes
17
views
asked 5 days ago

using transformers module with sagemaker studio project: ModuleNotFoundError: No module named 'transformers'

So as mentioned in my [other recent post](https://repost.aws/questions/QUAL9Vn9abQ6KKCs2ASwwmzg/adjusting-sagemaker-xgboost-project-to-tensorflow-or-even-just-different-folder-name), I'm trying to modify the sagemaker example abalone xgboost template to use tensorfow. My current problem is that running the pipeline I get a failure and in the logs I see: ``` ModuleNotFoundError: No module named 'transformers' ``` NOTE: I am importing 'transformers' in `preprocess.py` not in `pipeline.py` Now I have 'transformers' listed in various places as a dependency including: * `setup.py` - `required_packages = ["sagemaker==2.93.0", "sklearn", "transformers", "openpyxl"]` * `pipelines.egg-info/requires.txt` - `transformers` (auto-generated from setup.py?) but so I'm keen to understand, how can I ensure that additional dependencies are available in the pipline itself? Many thanks in advance ------------ ------------ ------------ ADDITIONAL DETAILS ON HOW I ENCOUNTERED THE ERROR From one particular notebook (see [previous post](https://repost.aws/questions/QUAL9Vn9abQ6KKCs2ASwwmzg/adjusting-sagemaker-xgboost-project-to-tensorflow-or-even-just-different-folder-name) for more details) I have succesfully constructed the new topic/tensorflow pipeline and run the following steps: ``` pipeline.upsert(role_arn=role) execution = pipeline.start() execution.describe() ``` the `describe()` method gives this output: ``` {'PipelineArn': 'arn:aws:sagemaker:eu-west-1:398371982844:pipeline/topicpipeline-example', 'PipelineExecutionArn': 'arn:aws:sagemaker:eu-west-1:398371982844:pipeline/topicpipeline-example/execution/0aiczulkjoaw', 'PipelineExecutionDisplayName': 'execution-1664394415255', 'PipelineExecutionStatus': 'Executing', 'PipelineExperimentConfig': {'ExperimentName': 'topicpipeline-example', 'TrialName': '0aiczulkjoaw'}, 'CreationTime': datetime.datetime(2022, 9, 28, 19, 46, 55, 147000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2022, 9, 28, 19, 46, 55, 147000, tzinfo=tzlocal()), 'CreatedBy': {'UserProfileArn': 'arn:aws:sagemaker:eu-west-1:398371982844:user-profile/d-5qgy6ubxlbdq/sjoseph-reg-genome-com-273', 'UserProfileName': 'sjoseph-reg-genome-com-273', 'DomainId': 'd-5qgy6ubxlbdq'}, 'LastModifiedBy': {'UserProfileArn': 'arn:aws:sagemaker:eu-west-1:398371982844:user-profile/d-5qgy6ubxlbdq/sjoseph-reg-genome-com-273', 'UserProfileName': 'sjoseph-reg-genome-com-273', 'DomainId': 'd-5qgy6ubxlbdq'}, 'ResponseMetadata': {'RequestId': 'f949d6f4-1865-4a01-b7a2-a96c42304071', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'f949d6f4-1865-4a01-b7a2-a96c42304071', 'content-type': 'application/x-amz-json-1.1', 'content-length': '882', 'date': 'Wed, 28 Sep 2022 19:47:02 GMT'}, 'RetryAttempts': 0}} ``` Waiting for the execution I get: ``` --------------------------------------------------------------------------- WaiterError Traceback (most recent call last) <ipython-input-14-72be0c8b7085> in <module> ----> 1 execution.wait() /opt/conda/lib/python3.7/site-packages/sagemaker/workflow/pipeline.py in wait(self, delay, max_attempts) 581 waiter_id, model, self.sagemaker_session.sagemaker_client 582 ) --> 583 waiter.wait(PipelineExecutionArn=self.arn) 584 585 /opt/conda/lib/python3.7/site-packages/botocore/waiter.py in wait(self, **kwargs) 53 # method. 54 def wait(self, **kwargs): ---> 55 Waiter.wait(self, **kwargs) 56 57 wait.__doc__ = WaiterDocstring( /opt/conda/lib/python3.7/site-packages/botocore/waiter.py in wait(self, **kwargs) 376 name=self.name, 377 reason=reason, --> 378 last_response=response, 379 ) 380 if num_attempts >= max_attempts: WaiterError: Waiter PipelineExecutionComplete failed: Waiter encountered a terminal failure state: For expression "PipelineExecutionStatus" we matched expected path: "Failed" ``` Which I assume is corresponding to the failure I see in the logs: ![buildl pipeline error message on preprocessing step](/media/postImages/original/IMMpF6LeI6TgWxp20TnPZbUw) I did also run `python setup.py build` to ensure my build directory was up to date ... here's the terminal output of that command: ``` sagemaker-user@studio$ python setup.py build /opt/conda/lib/python3.9/site-packages/setuptools/dist.py:771: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead warnings.warn( /opt/conda/lib/python3.9/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead. warnings.warn(msg, warning_class) running build running build_py copying pipelines/topic/pipeline.py -> build/lib/pipelines/topic running egg_info writing pipelines.egg-info/PKG-INFO writing dependency_links to pipelines.egg-info/dependency_links.txt writing entry points to pipelines.egg-info/entry_points.txt writing requirements to pipelines.egg-info/requires.txt writing top-level names to pipelines.egg-info/top_level.txt reading manifest file 'pipelines.egg-info/SOURCES.txt' adding license file 'LICENSE' writing manifest file 'pipelines.egg-info/SOURCES.txt' ``` It seems like the dependencies are being written to `pipelines.egg-info/requires.txt` but are these not being picked up by the pipeline?
1
answers
0
votes
46
views
asked 6 days ago

adjusting sagemaker xgboost project to tensorflow (or even just different folder name)

I have sagemaker xgboost project template "build, train, deploy" working, but I'd like to modify if to use tensorflow instead of xgboost. First up I was just trying to change the `abalone` folder to `topic` to reflect the data we are working with. I was experimenting with trying to change the `topic/pipeline.py` file like so ``` image_uri = sagemaker.image_uris.retrieve( framework="tensorflow", region=region, version="1.0-1", py_version="py3", instance_type=training_instance_type, ) ``` i.e. just changing the framework name from "xgboost" to "tensorflow", but then when I run the following from a notebook: ``` from pipelines.topic.pipeline import get_pipeline pipeline = get_pipeline( region=region, role=role, default_bucket=default_bucket, model_package_group_name=model_package_group_name, pipeline_name=pipeline_name, ) ``` I get the following error ``` ValueError Traceback (most recent call last) <ipython-input-5-6343f00c3471> in <module> 7 default_bucket=default_bucket, 8 model_package_group_name=model_package_group_name, ----> 9 pipeline_name=pipeline_name, 10 ) ~/topic-models-no-monitoring-p-rboparx6tdeg/sagemaker-topic-models-no-monitoring-p-rboparx6tdeg-modelbuild/pipelines/topic/pipeline.py in get_pipeline(region, sagemaker_project_arn, role, default_bucket, model_package_group_name, pipeline_name, base_job_prefix, processing_instance_type, training_instance_type) 188 version="1.0-1", 189 py_version="py3", --> 190 instance_type=training_instance_type, 191 ) 192 tf_train = Estimator( /opt/conda/lib/python3.7/site-packages/sagemaker/workflow/utilities.py in wrapper(*args, **kwargs) 197 logger.warning(warning_msg_template, arg_name, func_name, type(value)) 198 kwargs[arg_name] = value.default_value --> 199 return func(*args, **kwargs) 200 201 return wrapper /opt/conda/lib/python3.7/site-packages/sagemaker/image_uris.py in retrieve(framework, region, version, py_version, instance_type, accelerator_type, image_scope, container_version, distribution, base_framework_version, training_compiler_config, model_id, model_version, tolerate_vulnerable_model, tolerate_deprecated_model, sdk_version, inference_tool, serverless_inference_config) 152 if inference_tool == "neuron": 153 _framework = f"{framework}-{inference_tool}" --> 154 config = _config_for_framework_and_scope(_framework, image_scope, accelerator_type) 155 156 original_version = version /opt/conda/lib/python3.7/site-packages/sagemaker/image_uris.py in _config_for_framework_and_scope(framework, image_scope, accelerator_type) 277 image_scope = available_scopes[0] 278 --> 279 _validate_arg(image_scope, available_scopes, "image scope") 280 return config if "scope" in config else config[image_scope] 281 /opt/conda/lib/python3.7/site-packages/sagemaker/image_uris.py in _validate_arg(arg, available_options, arg_name) 443 "Unsupported {arg_name}: {arg}. You may need to upgrade your SDK version " 444 "(pip install -U sagemaker) for newer {arg_name}s. Supported {arg_name}(s): " --> 445 "{options}.".format(arg_name=arg_name, arg=arg, options=", ".join(available_options)) 446 ) 447 ValueError: Unsupported image scope: None. You may need to upgrade your SDK version (pip install -U sagemaker) for newer image scopes. Supported image scope(s): eia, inference, training. ``` I was skeptical that the upgrade suggested by the error message would fix this, but gave it a try: ``` ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. pipelines 0.0.1 requires sagemaker==2.93.0, but you have sagemaker 2.110.0 which is incompatible. ``` So that seems like I can't upgrade sagemaker without changing pipelines, and it's not clear that's the right thing to do - like this project template may be all designed around those particular ealier libraries. But so is it that the "framework" name should be different, e.g. "tf"? Or is there some other setting that needs changing in order to allow me to get a tensorflow pipeline ...? However I find that if I use the existing `abalone/pipeline.py` file I can change the framework to "tensorflow" and there's no problem running that particular step in the notebook. I've searched all the files in the project to try and find any dependency on the `abalone` folder name, and the closest I came was in `codebuild-buildspec.yml` but that hasn't helped. Has anyone else successfully changed the folder name from `abalone` to something else, or am I stuck with `abalone` if I want to make progress? Many thanks in advance p.s. is there a slack community for sagemaker studio anywhere? p.p.s. I have tried changing all instances of the term "Abalone" to "Topic" within the `topic/pipeline.py` file (matching case as appropriate) to no avail p.p.p.s. I discovered that I can get an error free run of getting the pipeline from a unit test: ``` import pytest from pipelines.topic.pipeline import * region = 'eu-west-1' role = 'arn:aws:iam::398371982844:role/SageMakerExecutionRole' default_bucket = 'sagemaker-eu-west-1-398371982844' model_package_group_name = 'TopicModelPackageGroup-Example' pipeline_name = 'TopicPipeline-Example' def test_pipeline(): pipeline = get_pipeline( region=region, role=role, default_bucket=default_bucket, model_package_group_name=model_package_group_name, pipeline_name=pipeline_name, ) ``` and strangely if I go to a different copy of the notebook, everything runs fine, there ... so I have two seemingly identical ipynb notebooks, and in one of them when I switch to trying to get a topic pipeline I get the above error, and in the other, I get no error at all, very strange p.p.p.p.s. I also notice that `conda list` returns very different results depending on whether I run it in the notebook or the terminal ... but the conda list results are identical for the two notebooks ...
1
answers
0
votes
24
views
asked 6 days ago

Django Daphne Websocket Access Denied

We need to establish a "Web socket connection" to our AWS servers using Django, Django channels, Redis, and Daphne Nginx Config. Currently local and on-premises config is configured properly and needs help in configuring the same communication with the staging server. We tried adding the above config to our servers but got an error of access denied with response code 403 from the server for web socket request. below is the **Nginx config** for staging ``` server { listen 80; server_name domain_name.com domain_name_2.com; root /var/www/services/project_name_frontend/; index index.html; location ~ ^/api/ { rewrite ^/api/(.*) /$1 break; proxy_pass http://unix:/var/www/services/enerlly_backend/backend/backend.sock; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $host; proxy_read_timeout 30; proxy_connect_timeout 30; proxy_send_timeout 30; send_timeout 30; proxy_redirect ~^/(.*) $scheme://$host/api/$1; } location /ws { try_files $uri @proxy_to_ws; } location @proxy_to_ws { proxy_pass http://127.0.0.1:8001; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_redirect off; } location ~ ^/admin/ { proxy_pass http://unix:/var/www/services/project_name_backend/backend/backend.sock; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $host; proxy_read_timeout 30; proxy_connect_timeout 30; proxy_send_timeout 30; send_timeout 30; proxy_redirect off; } location /staticfiles/ { alias /var/www/services/project_name_backend/backend/staticfiles/; } location /mediafiles/ { alias /var/www/services/project_name_backend/backend/mediafiles/; } location / { try_files $uri /index.html; } } ``` and **Systemctl service** to execute Django Daphne service ``` [Unit] Description=Backend Project Django WebSocket daemon After=network.target [Service] User=root Group=www-data WorkingDirectory=/var/www/services/project_name_backend ExecStart=/home/ubuntu/project_python_venv/bin/python /home/ubuntu/project_python_venv/bin/daphne -b 0.0.0.0 -p 8001 project_name_backend.prod_asgi:application [Install] WantedBy=multi-user.target ``` **Below is the Load Balancer security group config inbound rules** ![Enter image description here](/media/postImages/original/IMN2LT2BlTSmK0PHEAu5dwHQ) **Listner Config for Load Balancer** ![Enter image description here](/media/postImages/original/IMxBGKpaJOSrSsOQyn5FEt-Q) ![Enter image description here](/media/postImages/original/IMktSIYK0ZSOy8GzYyR-DI_w)
0
answers
0
votes
8
views
asked 7 days ago

Recent articles

see all
1/1

Popular users

see all
1/18