Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Need help in getting Started with SageMaker

0

Hello, I am new to ML and just trying to get started with Sagemaker. I am trying to

import pandas as pd
import numpy as np
import boto3
from sagemaker import get_execution_role
conn = boto3.client('s3')
bucket = 'sagemakerkaggle'
content = conn.list_objects(Bucket=bucket)['Contents']
data_key= 'montocarlo/train.csv'
data_location= 's3://{}/{}'.format(bucket,data_key)

train = pd.read_csv(data_location, index_col= False)

I am getting an import error However, I can see the contents of my S3 Bucket. So, I don't think there is an issue related to access.

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File /opt/conda/lib/python3.10/site-packages/fsspec/registry.py:226, in get_filesystem_class(protocol)
    <a href='/opt/conda/lib/python3.10/site-packages/fsspec/registry.py:225'>225</a> try:
--> <a href='/opt/conda/lib/python3.10/site-packages/fsspec/registry.py:226'>226</a>     register_implementation(protocol, _import_class(bit["class"]))
    <a href='/opt/conda/lib/python3.10/site-packages/fsspec/registry.py:227'>227</a> except ImportError as e:
ImportError: Install s3fs to access S3
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...

After installing s3fs I am getting an error again

...
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
datasets 2.20.0 requires fsspec[http]<=2024.5.0,>=2023.1.0, but you have fsspec 2024.9.0 which is incompatible.
jupyter-scheduler 2.7.1 requires fsspec==2023.6.0, but you have fsspec 2024.9.0 which is incompatible.
Successfully installed fsspec-2023.6.0 s3fs-2024.9.0
Topics
Machine Learning & AI
Tags
Amazon SageMakerAmazon SageMaker Studio
Language
English

asked 2 years ago866 views

2 Answers
1

Hello,

Error: You were getting an error because you needed to install the s3fs library to access S3 files in your SageMaker notebook.

Let's do:

  1. Install s3fs: Use !pip install s3fs in your notebook.

Use !pip install s3fs==2023.6.0 to install a compatible version.

  • Access data: Use pd.read_csv('s3://your_bucket/your_file.csv') to read your CSV file from S3.

2.Use a virtual environment:

  • Create a new environment with conda create -n my_env python=3.7.
  • Activate it with conda activate my_env.
  • Install required packages like pandas, numpy, boto3, sagemaker, fsspec[http].
EXPERT

answered 2 years ago

EXPERT

reviewed 2 years ago

  • Aditi
    2 years ago

    Now I am getting this error

    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
amazon-sagemaker-jupyter-scheduler 3.1.2 requires aiobotocore<3,>=2.7, but you have aiobotocore 2.5.4 which is incompatible.
amazon-sagemaker-sql-editor 0.1.10 requires aiobotocore<3,>=2.7.0, but you have aiobotocore 2.5.4 which is incompatible.
amazon-sagemaker-sql-editor 0.1.10 requires botocore<2,>=1.31.64, but you have botocore 1.31.17 which is incompatible.
boto3 1.34.131 requires botocore<1.35.0,>=1.34.131, but you have botocore 1.31.17 which is incompatible.
s3transfer 0.10.2 requires botocore<2.0a.0,>=1.33.2, but you have botocore 1.31.17 which is incompatible.
sagemaker 2.227.0 requires boto3<2.0,>=1.34.142, but you have boto3 1.34.131 which is incompatible.
sagemaker-jupyterlab-extension-common 0.1.19 requires aiobotocore>=2.7.0, but you have aiobotocore 2.5.4 which is incompatible.

    Also, since I am installing within Sagemaker notebook, I am doing

    !conda create -n my_env python=3.7

    However, during installation, it is asking me Proceed [y]/no? On my local computer I can simply press enter on Anaconda prompt to install the packages. How is this done within a notebook?

  • NARRAVULA MUNI SAI TEJA EXPERT
    2 years ago
    1. Install Necessary Libraries:
    • Use %%pip install pandas numpy boto3 s3fs fsspec[http]==2023.6.0 in your SageMaker notebook.'
    1. Access S3 Data:
    • import libraries: import pandas as pd, boto3.
    • Get data location: data_location = 's3://your_bucket/your_file.csv'.
    • Read data: train = pd.read_csv(data_location).
0

Can you provide more details about how do you run the notebook? If you work with SageMaker studio, you can select the relevant kernel with basic dependancies. Check this video for very basic "getting started". https://www.youtube.com/watch?v=oBx_o57gDGY

AWS

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content