By using AWS re:Post, you agree to the Terms of Use

Management & Governance

In the past, organizations have had to choose between innovating faster and maintaining control over cost, compliance, and security. With AWS Management and Governance services, customers don’t have to choose between innovation and control—they can have both. With AWS, customers can enable, provision, and operate their environment for both business agility and governance control.

Recent questions

see all
1/18

using transformers module with sagemaker studio project: ModuleNotFoundError: No module named 'transformers'

So as mentioned in my [other recent post](https://repost.aws/questions/QUAL9Vn9abQ6KKCs2ASwwmzg/adjusting-sagemaker-xgboost-project-to-tensorflow-or-even-just-different-folder-name), I'm trying to modify the sagemaker example abalone xgboost template to use tensorfow. My current problem is that running the pipeline I get a failure and in the logs I see: ``` ModuleNotFoundError: No module named 'transformers' ``` Now I have 'transformers' listed in various places as a dependency including: * `setup.py` - `required_packages = ["sagemaker==2.93.0", "sklearn", "transformers", "openpyxl"]` * `pipelines.egg-info/requires.txt` - `transformers` (auto-generated from setup.py? but so I'm keen to understand, how can I ensure that additional dependencies are available in the pipline itself? Many thanks in advance ------------ ------------ ------------ ADDITIONAL DETAILS ON HOW I ENCOUNTERED THE ERROR From one particular notebook (see [previous post](https://repost.aws/questions/QUAL9Vn9abQ6KKCs2ASwwmzg/adjusting-sagemaker-xgboost-project-to-tensorflow-or-even-just-different-folder-name) for more details) I have succesfully constructed the new topic/tensorflow pipeline and run the following steps: ``` pipeline.upsert(role_arn=role) execution = pipeline.start() execution.describe() ``` the `describe()` method gives this output: ``` {'PipelineArn': 'arn:aws:sagemaker:eu-west-1:398371982844:pipeline/topicpipeline-example', 'PipelineExecutionArn': 'arn:aws:sagemaker:eu-west-1:398371982844:pipeline/topicpipeline-example/execution/0aiczulkjoaw', 'PipelineExecutionDisplayName': 'execution-1664394415255', 'PipelineExecutionStatus': 'Executing', 'PipelineExperimentConfig': {'ExperimentName': 'topicpipeline-example', 'TrialName': '0aiczulkjoaw'}, 'CreationTime': datetime.datetime(2022, 9, 28, 19, 46, 55, 147000, tzinfo=tzlocal()), 'LastModifiedTime': datetime.datetime(2022, 9, 28, 19, 46, 55, 147000, tzinfo=tzlocal()), 'CreatedBy': {'UserProfileArn': 'arn:aws:sagemaker:eu-west-1:398371982844:user-profile/d-5qgy6ubxlbdq/sjoseph-reg-genome-com-273', 'UserProfileName': 'sjoseph-reg-genome-com-273', 'DomainId': 'd-5qgy6ubxlbdq'}, 'LastModifiedBy': {'UserProfileArn': 'arn:aws:sagemaker:eu-west-1:398371982844:user-profile/d-5qgy6ubxlbdq/sjoseph-reg-genome-com-273', 'UserProfileName': 'sjoseph-reg-genome-com-273', 'DomainId': 'd-5qgy6ubxlbdq'}, 'ResponseMetadata': {'RequestId': 'f949d6f4-1865-4a01-b7a2-a96c42304071', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'f949d6f4-1865-4a01-b7a2-a96c42304071', 'content-type': 'application/x-amz-json-1.1', 'content-length': '882', 'date': 'Wed, 28 Sep 2022 19:47:02 GMT'}, 'RetryAttempts': 0}} ``` Waiting for the execution I get: ``` --------------------------------------------------------------------------- WaiterError Traceback (most recent call last) <ipython-input-14-72be0c8b7085> in <module> ----> 1 execution.wait() /opt/conda/lib/python3.7/site-packages/sagemaker/workflow/pipeline.py in wait(self, delay, max_attempts) 581 waiter_id, model, self.sagemaker_session.sagemaker_client 582 ) --> 583 waiter.wait(PipelineExecutionArn=self.arn) 584 585 /opt/conda/lib/python3.7/site-packages/botocore/waiter.py in wait(self, **kwargs) 53 # method. 54 def wait(self, **kwargs): ---> 55 Waiter.wait(self, **kwargs) 56 57 wait.__doc__ = WaiterDocstring( /opt/conda/lib/python3.7/site-packages/botocore/waiter.py in wait(self, **kwargs) 376 name=self.name, 377 reason=reason, --> 378 last_response=response, 379 ) 380 if num_attempts >= max_attempts: WaiterError: Waiter PipelineExecutionComplete failed: Waiter encountered a terminal failure state: For expression "PipelineExecutionStatus" we matched expected path: "Failed" ``` Which I assume is corresponding to the failure I see in the logs: ![buildl pipeline error message on preprocessing step](/media/postImages/original/IMMpF6LeI6TgWxp20TnPZbUw) I did also run `python setup.py build` to ensure my build directory was up to date ... here's the terminal output of that command: ``` sagemaker-user@studio$ python setup.py build /opt/conda/lib/python3.9/site-packages/setuptools/dist.py:771: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead warnings.warn( /opt/conda/lib/python3.9/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead. warnings.warn(msg, warning_class) running build running build_py copying pipelines/topic/pipeline.py -> build/lib/pipelines/topic running egg_info writing pipelines.egg-info/PKG-INFO writing dependency_links to pipelines.egg-info/dependency_links.txt writing entry points to pipelines.egg-info/entry_points.txt writing requirements to pipelines.egg-info/requires.txt writing top-level names to pipelines.egg-info/top_level.txt reading manifest file 'pipelines.egg-info/SOURCES.txt' adding license file 'LICENSE' writing manifest file 'pipelines.egg-info/SOURCES.txt' ``` It seems like the dependencies are being written to `pipelines.egg-info/requires.txt` but are these not being picked up by the pipeline?
1
answers
0
votes
22
views
asked 2 days ago

Failed to stabilize Instance with id

I have this problem: Failed to stabilize Instance with id. My CF looks like: Resources: DocumentDBSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupName: 'DocumentDB SG' GroupDescription: !Sub 'Security Group for the DocumentDb' VpcId: !Ref VPC DocumentDBSubnetGroup: Type: AWS::DocDB::DBSubnetGroup Properties: DBSubnetGroupDescription: "Subnet group for Document DB cluster" DBSubnetGroupName: "document-db-subnet-group" SubnetIds: !Ref PrivateSubnetIds DocumentDBParameterGroup: Type: AWS::DocDB::DBClusterParameterGroup Properties: Description: "Parameter group for Document DB cluster" Family: docdb4.0 Name: "document-db-paramater-group" Parameters: audit_logs: "disabled" DocumentDBCluster: Type: AWS::DocDB::DBCluster Properties: BackupRetentionPeriod: 7 DBClusterIdentifier: "docdb" DBSubnetGroupName: !Ref DocumentDBSubnetGroup DBClusterParameterGroupName: !Ref DocumentDBParameterGroup Port: 27017 PreferredBackupWindow: "07:00-09:30" PreferredMaintenanceWindow: "tue:07:00-tue:11:00" VpcSecurityGroupIds: - !Ref DocumentDBSecurityGroup StorageEncrypted: true DocumentDBInstance: Type: AWS::DocDB::DBInstance DependsOn: - DocumentDBCluster Properties: DBClusterIdentifier: !Ref DocumentDBCluster DBInstanceClass: db.t3.medium DBInstanceIdentifier: "docdb" PreferredMaintenanceWindow: "tue:07:00-tue:11:00" If i search this problem i find information about RDS (snapshot), but i don't use snapshot in this deployment..
1
answers
0
votes
20
views
asked 2 days ago

Recent articles

see all
1/3

Popular users

see all
1/18

Learn AWS faster by following popular topics

1/2