AWS Cloud Quest : Fine-Tunning LLM on Amazon SageMaker DIY

Question

The DIY Section where you instantiate

Step 6.1: Instantiate SageMaker parameters
Initialize a SageMaker session and retrieve information related to the AWS environment such as SageMaker role and AWS region. You also specify the image URI for a specific version of the "djl-deepspeed" framework using the SageMaker session's region. The image URI is a unique identifier for a specific Docker container image that can be used in various AWS services, such as Amazon SageMaker or Elastic Container Registry (ECR).

```
import boto3
import json
import sagemaker.djl_inference
from sagemaker.session import Session
from sagemaker import image_uris
from sagemaker import Model

sagemaker_session = Session()
print("sagemaker_session: ", sagemaker_session)

aws_role = sagemaker_session.get_caller_identity_arn()
print("aws_role: ", aws_role)

aws_region = boto3.Session().region_name
print("aws_region: ", aws_region)

image_uri = image_uris.retrieve(framework="djl-deepspeed",
                                version="0.22.1",
                                region=sagemaker_session._region_name)
print("image_uri: ", image_uri)
```

is returning the following error and there is no instruction on how or where to get it fixed

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in :3                                                                                    │
│                                                                                                  │
│    1 import boto3                                                                                │
│    2 import json                                                                                 │
│ ❱  3 import sagemaker.djl_inference                                                              │
│    4 from sagemaker.session import Session                                                       │
│    5 from sagemaker import image_uris                                                            │
│    6 from sagemaker import Model                                                                 │
│                                                                                                  │
│ /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/__init__.py:18 in   │
│                                                                                          │
│                                                                                                  │
│   15                                                                                             │
│   16 import importlib_metadata                                                                   │
│   17                                                                                             │
│ ❱ 18 from sagemaker import estimator, parameter, tuner  # noqa: F401                             │
│   19 from sagemaker.amazon.kmeans import KMeans, KMeansModel, KMeansPredictor  # noqa: F401      │
│   20 from sagemaker.amazon.pca import PCA, PCAModel, PCAPredictor  # noqa: F401                  │
│   21 from sagemaker.amazon.lda import LDA, LDAModel, LDAPredictor  # noqa: F401                  │
│                                                                                                  │
│ /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/estimator.py:30 in  │
│                                                                                          │
│                                                                                                  │
│     27 from six.moves.urllib.parse import urlparse                                               │
│     28                                                                                           │
│     29 import sagemaker                                                                          │
│ ❱   30 from sagemaker import git_utils, image_uris, vpc_utils, s3                                │
│     31 from sagemaker.analytics import TrainingJobAnalytics                                      │
│     32 from sagemaker.config import (                                                            │
│     33 │   ESTIMATOR_DEBUG_HOOK_CONFIG_PATH,                                                     │
│                                                                                                  │
│ /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/image_uris.py:24 in │
│                                                                                          │
│                                                                                                  │
│    21 from packaging.version import Version                                                      │
│    22                                                                                            │
│    23 from sagemaker import utils                                                                │
│ ❱  24 from sagemaker.jumpstart.constants import DEFAULT_JUMPSTART_SAGEMAKER_SESSION              │
│    25 from sagemaker.jumpstart.enums import JumpStartModelType                                   │
│    26 from sagemaker.jumpstart.utils import is_jumpstart_model_input                             │
│    27 from sagemaker.spark import defaults                                                       │
│                                                                                                  │
│ /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/jumpstart/constants │
│ .py:27 in                                                                                │
│                                                                                                  │
│    24 │   MIMEType,                                                                              │
│    25 │   JumpStartModelType,                                                                    │
│    26 )                                                                                          │
│ ❱  27 from sagemaker.jumpstart.types import JumpStartLaunchedRegionInfo, JumpStartS3FileType     │
│    28 from sagemaker.base_serializers import (                                                   │
│    29 │   BaseSerializer,                                                                        │
│    30 │   CSVSerializer,                                                                         │
│                                                                                                  │
│ /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker/jumpstart/types.py: │
│ 20 in                                                                                    │
│                                                                                                  │
│     17 from copy import deepcopy                                                                 │
│     18 from enum import Enum                                                                     │
│     19 from typing import Any, Dict, List, Optional, Set, Union                                  │
│ ❱   20 from sagemaker_core.shapes import ModelAccessConfig as CoreModelAccessConfig              │
│     21 from sagemaker.model_card.model_card import ModelCard, ModelPackageModelCard              │
│     22 from sagemaker.utils import (                                                             │
│     23 │   S3_PREFIX,                                                                            │
│                                                                                                  │
│ /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker_core/shapes/__init_ │
│ _.py:1 in                                                                                │
│                                                                                                  │
│ ❱ 1 from ..main.shapes import *                                                                  │
│   2                                                                                              │
│                                                                                                  │
│ /home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/sagemaker_core/main/shapes.py │
│ :2511 in                                                                                 │
│                                                                                                  │
│    2508 │   """                                                                                  │
│    2509                                                                                          │
│    2510                                                                                          │
│ ❱  2511 class MonitoringDatasetFormat(Base):                                                     │
│    2512 │   """                                                                                  │
│    2513 │   MonitoringDatasetFormat                                                              │
│    2514 │     Represents the dataset format used when running a monitoring job.                  │
│                                                                                                  │
│ in pydantic.main.ModelMetaclass.__new__:186                                                      │
│                                                                                                  │
│ in pydantic.utils.validate_field_name:168                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
NameError: Field name "json" shadows a BaseModel attribute; use a different field name with "alias='json'".

This is the last task on the ML course. I need to complete this step to pass the course

re:Post Agent · Accepted Answer

To resolve the error you're encountering in the AWS Cloud Quest: Fine-Tuning LLM on Amazon SageMaker DIY section, try the following steps:

1. Update the SageMaker SDK to the latest version using pip:
   pip install --upgrade sagemaker

2. If the issue persists, create a new conda environment with the required dependencies:
   conda create -n sagemaker_env python=3.8
   conda activate sagemaker_env
   pip install sagemaker boto3

3. If you're still facing problems, try using a modified version of the code that doesn't rely on the sagemaker.djl_inference module. Here's an alternative approach using the Hugging Face framework:

import boto3
   import json
   from sagemaker.session import Session
   from sagemaker import get_execution_role
   from sagemaker import image_uris
   from sagemaker import Model

sagemaker_session = Session()
   print("sagemaker_session: ", sagemaker_session)

aws_role = get_execution_role()
   print("aws_role: ", aws_role)

aws_region = boto3.Session().region_name
   print("aws_region: ", aws_region)

image_uri = image_uris.retrieve(framework="huggingface",
                                   region=aws_region,
                                   version="4.17.0",
                                   image_scope="inference",
                                   base_framework_version="pytorch1.10.2")
   print("image_uri: ", image_uri)

This modified version should be more widely compatible and help you proceed with the course.

If you continue to experience issues, it's recommended to reach out to AWS support or your course instructor for specific guidance, as there may be particular requirements or configurations for the Cloud Quest environment that require their expertise.  
**Sources**  
[Fine-tuning an LLM using QLoRA in AWS GovCloud (US) | AWS Public Sector Blog](https://aws.amazon.com/blogs/publicsector/fine-tuning-an-llm-using-qlora-in-aws-govcloud-us/)  
[Improve RAG accuracy with fine-tuned embedding models on Amazon SageMaker | AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/improve-rag-accuracy-with-fine-tuned-embedding-models-on-amazon-sagemaker/)

AWS Cloud Quest : Fine-Tunning LLM on Amazon SageMaker DIY

Add your answer

Relevant content