How to provide unique filename to each file in each event in aws lambda python?

Question

Hi There,

I have a lambda function which trigger as soon as AWS data pipeline writes a directory with multiple files to s3 bucket. A lambda function which is written in python reads that directory files and convert into csv and write back to different bucket folder.

The problem is when i pass the object name as file name it creates the same name file and upload to s3 bucket and it does  for every file this creates mess because everyday i have to point to the new file. I want to give unique name to these files so everyday the same file being replaced instead created new one.

EDIT:

Here is my code

if event:

print(f"Got Events")
    #Reading file 
    df = pd.read_json(obj['Body'], lines=True, orient='records', dtype='dict')
    csv_buffer = StringIO()
    df.to_csv(csv_buffer, index=False)
    filename = 'file1.csv'
    client.put_object(Bucket=bucket, Key=filename,Body=csv_buffer.getvalue())

Please help...!

Answer

I'm assuming you are using the boto3 S3 Client.

The `put_object`, `upload_file` and related methods accept a `Key` parameter so you can name the file in S3 whatever you'd like.

Answer

You have control on which file name to use. If you want to use a fixed name, just use that in your S3 API.

Answer

Hello,

One possible approach is to append date time to your filename to generate a unique filename before upload in your lambda function. I'm not a Python or code expert but see if following helps.

```
from datetime import datetime
now = datetime.now() # current date and time
dt = now.strftime("%Y-%m-%d-%H-%M-%S") #store date time in string
file = "file-" + dt #append the name to a string to generate filename

```

You can test the code on a local machine first and see if it meets the requirement.

How to provide unique filename to each file in each event in aws lambda python?

Relevant content