Lambda concurrent invocations - collision using same '/tmp/yourfilename.csv'?

1

We plan to use Lambda invoced by SNS to read json data from S3, process it and save in csv fromat to another bucket. Current approach to create temporary file, then use boto3 upload_file.

df.to_csv('/tmp/csv_file.csv', index=False, encoding='utf-8')
s3.upload_file('/tmp/csv_file.csv', OUTPUT_BUCKET, + key)

But I wonder would it create collisions if I will have several concurrent invocations of the lambda, both trying to write to the same /tmp/csv_file.csv?
Or /tmp/.. will be different for each Lambda invocation?

3 Answers
3
Accepted Answer

Each concurrent Lambda invocation runs in its own run time environment with its own /tmp folder. There will not be collisions. Subsequent invocation may reuse existing environments, so the file may still be there in the /tmp, but again, it is from a previous invocation and not from a concurrent one.

profile pictureAWS
EXPERT
Uri
answered 2 years ago
profile pictureAWS
EXPERT
reviewed 2 years ago
3

Lambda will not have parallel invocations that use the same container. Containers will be reused if they have previously been provisioned and are idle (before being reclaimed by the service after some time), but no container will serve multiple requests at the same time. In the case of multiple requests, separate containers will be provisioned for each and there will be no race conditions on your file. Just make sure your code pulls down a fresh copy of the CSV every time the function is invoked.

AWS
answered 2 years ago
profile pictureAWS
EXPERT
reviewed 2 years ago
0

You can always add a random number job id or use aws_request_id (https://docs.aws.amazon.com/lambda/latest/dg/python-context.html) to your csv file name to make them unique.

AWS
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions