error while reading csv from S3 using pandas

0

Hi Team, I am getting an error "Install s3fs to access S3" while trying to access a file from S3 in AWS lambda. However I tried importing all possible packages but still I am not able to read the file.

  • import json import boto3 import pandas as pd

    def lambda_handler(event, context):

    df1 = pd.read_csv('s3://himanrcctest/size_and_fit/Logs_2023-07-15.csv001',delimiter="\t")
    return str(df1
    

    I have already run below commands: pip3 install s3fs pip3 install fs_s3fs however I got this error while running above commnd - awscli 1.29.60 requires botocore==1.31.60, but you have botocore 1.31.17 which is incompatible. boto3 1.28.60 requires botocore<1.32.0,>=1.31.60, but you have botocore 1.31.17 which is incompatible.

gefragt vor 8 Monaten329 Aufrufe
1 Antwort
0

Hello.

What if you run the command below to update botocore to the latest version?
https://docs.aws.amazon.com/efs/latest/ug/install-botocore.html

sudo pip3 install botocore --upgrade

Also, since neither pandas nor s3fs are included in the Lambda package, you need to create a layer using the steps in the document below.

mkdir python
pip3 install pandas -t ./python/
pip3 install s3fs -t ./python/
zip -r lambda_layer.zip python/
aws lambda publish-layer-version --layer-name "lambda_layer_name" --zip-file fileb://lambda_layer.zip --compatible-runtimes python3.11
profile picture
EXPERTE
beantwortet vor 8 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen