- 新しい順
- 投票が多い順
- コメントが多い順
If this is your entire code, you seem to be missing a handler function (https://docs.aws.amazon.com/lambda/latest/dg/python-programming-model-handler-types.html). Code at the top level in a lambda function only gets run once, on instance initialization, but the handler function is called each time. Additionally, you don't need to tag every object on every invocation, only the object mentioned in the event passed to the handler function.
As a side note, when posting code here, it's best to surround it with {\code}(omit the backslashes) tags so it keeps formatting, like this:
def myfunc():
return 'foo'
A invocation occurs for every object uploaded to S3. If an instance is already processing an event, and a new event comes in, a new instance will be created to handle the new event.
Thanks @Ellison , so is there a limit to number of instances it will spin up for continuous S3 event?
So if 10 objects are created by a file transfer tool and each object takes just 2 seconds to upload then 10 lambda functions will be invoked?
The default limit is 1,000 concurrent invocations, though this can be raised by submitting a ticket to support. https://docs.aws.amazon.com/lambda/latest/dg/limits.html
Assuming that the uploads for all 10 objects completed at about the same time, 10 lambda instances will be created to handle them. But it can vary a little depending on timing.
Thanks @Ellison, However I am having some issue, when lambda function is invoked its able to tag only few objects which are created, rest of them aren't tagged, whereas in when I run the python code from an EC2 instance it works perfectly fine tagging all the objects, so code works it's just that lambda is unable to tag the files as fast as they are uploaded.
Any thoughts?
It is possible that the lambda is exiting before the S3 tagging has completed, thus aborting the tagging. Review the following link to see if adding async/promise/callback makes sense in your scenario.
Link: https://stackoverflow.com/questions/55012396/write-to-s3-bucket-using-async-await-in-aws-lambda
Hi Ellison,
Below is my code.
import boto3
Create an S3 client
s3 = boto3.client("s3")
all_objects = s3.list_objects(Bucket = 'Bucket-Name')
tag={ 'TagSet1' : { 'FileType1': { 'Key1': 'Value1', 'Key2': 'Value2', 'Key3': 'Value3' } , 'FileType2': { 'Key1': 'Value1', 'Key2': 'Value2', 'Key3': 'Value3' } , 'FileType3': { 'Key1': 'Value1', 'Key2': 'Value2', 'Key3': 'Value3' } }, 'TagSet2' : { 'FileType4': { 'Key1': 'Value1', 'Key2': 'Value2', 'Key3': 'Value3' } , 'FileType5': { 'Key1': 'Value1', 'Key2': 'Value2', 'Key3': 'Value3' } , 'FileType6': { 'Key1': 'Value1', 'Key2': 'Value2', 'Key3': 'Value3' } } }
def tag_function(Folder,Object,tag):
tag_set = []
for key, value in tag[Folder][Object].items():
tagging = {}
tagging.update({'Key' : key})
tagging.update({'Value' : value})
tag_set.append(tagging)
return tag_set
for obj in all_objects['Contents']:
object_path=obj['Key']
try:
remove_extn=(object_path.split('.')[0])
if remove_extn:
file=(remove_extn.split('/')[-1])
if file:
Folder=(file.split('')[0])
Object=(file.split('')[1])
if Folder in ['TagSet1', 'TagSet2']:
response = s3.put_object_tagging(
Bucket='Bucket-Name',
Key=object_path,
Tagging={
'TagSet': tag_function(Folder,Object,tag)
}
)
except IndexError:
pass
Thanks Ellison, New to lambda functions, will make a point to post code the way you suggested.
