Lambda concurrent invocations - collision using same '/tmp/yourfilename.csv'?

Question

We plan to use Lambda invoced by SNS to read json data from S3, process it and save in csv fromat to another bucket.
Current approach to create temporary file, then use `boto3` `upload_file`.
```
df.to_csv('/tmp/csv_file.csv', index=False, encoding='utf-8')
s3.upload_file('/tmp/csv_file.csv', OUTPUT_BUCKET, + key)
```

But I wonder would it create collisions if I will have several concurrent invocations of the lambda, both trying to write to the same `/tmp/csv_file.csv`?  
Or `/tmp/..` will be different for each Lambda invocation?

Accepted Answer

Each concurrent Lambda invocation runs in its own run time environment with its own /tmp folder. There will not be collisions. Subsequent invocation may reuse existing environments, so the file may still be there in the /tmp, but again, it is from a previous invocation and not from a concurrent one.

Answer

Lambda will not have parallel invocations that use the same container.  Containers will be reused if they have previously been provisioned and are idle (before being reclaimed by the service after some time), but no container will serve multiple requests at the same time.  In the case of multiple requests, separate containers will be provisioned for each and there will be no race conditions on your file.  Just make sure your code pulls down a fresh copy of the CSV every time the function is invoked.

Answer

You can always add a random number job id or use aws_request_id (https://docs.aws.amazon.com/lambda/latest/dg/python-context.html) to your csv file name to make them unique.

Lambda concurrent invocations - collision using same '/tmp/yourfilename.csv'?

Contenuto pertinente