AWS Lambda Python Code

0

Hi Dears, Hope all is good

I have lambda function written in python (event driven ETL), I use two packaged functions to handel one csv file. I want to use the input csv file (S3 object uploaded to the bucket) in the two functions and put two json files in two different s3 buckets as an output.

But When I used the code as below, It convert the first function(df_out_1) successfully but in second function (df_out_2) give me error message that " No columns to parse from file " for the second function :

The code as below:

def lambda_handler(event, context):

s3_client = boto3.client('s3')
s3 = boto3.resource('s3')

if event:
    try:
        obj = event['Records'][0]
        bucket_name = obj['s3']['bucket']['name']
        file = obj['s3']['object']['key']
        csv_object = s3_client.get_object(Bucket = bucket_name , Key = file)
        csv_content = csv_object['Body'].read()
        bite = io.BytesIO(csv_content)

        df_out_1 = csv_json_function_alex_dep(bite)
        s3.Object('midibus-sagemaker-ai-engine-2' , jsonkey ).put(Body=json.dumps(df_out_1))
        print('The file is converted successfully for model-1 input!')
        
        #change input later to be jsonkey
        df_out_2 = csv_json_function_alex_arri(bite)
        s3.Object('midibus-sagemaker-ai-engine-2',  jsonkey ).put(Body=json.dumps(df_out_2))
        print('The file is converted successfully for model-2 input!')

    
        return {
        'statusCode': 200,
        'body': json.dumps('The file is converted successfully!')
        }
    except BaseException as err:
        print("Oops! Something went wrong ! \\n")
        print(err)
        print(err.args)
        traceback.print_exc(limit=5)

        return {
        'statusCode': 400,
        'body': json.dumps('Something went wrong!')
        }

thanks, Basem

2개 답변
1
수락된 답변

Have you tried doing bite = io.BytesIO(csv_content) again before the second function call?

profile pictureAWS
전문가
답변함 2년 전
  • Thanks!! It works well, Why we put it again with the second function?

0

If your first function uses bite.read() to get the content of the buffer, the internal buffer cursor will be advanced to the end of the buffer and a subsequent read() call will return an empty buffer. You can reset the cursor to the beginning of the buffer using bite.seek(0). Alternatively, if you can control how the functions you invoke are reading data from the buffer, you could usebite.getvalue().

AWS
전문가
답변함 2년 전
  • Thanks for your proposed solution, I used the above answer and it works well now.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠