Send set of Python files to AWS S3

Question

I need help because I need to create an executable in Python that is running all the time and that every 100 photos I send them and so on until I reach a thousand. To give it context, a system was made in Python (arduino, jetson, etc), which through a camera takes photos from 1 to 1,000 but the photos are all uploaded to a folder created by the same system when the photos are finished. photos. I try to find a way to send the photos taken by groups of 10-20 or 100 and not wait for the 1000 photos to finish taking and then upload them. And the connection is through Boto3, if there is any other better connection or that can help me, it is also helpful

Answer

Hello 
A good understanding of the system is needed to be able to suggest possible solutions. The boto3 lib can be plugged into the system to collect and upload these file. By and large how the system works is needed to help.

Answer

You can first start by creating a python script that will iterate thorough the set of python files and then upload them to S3 in groups of 100.
The script should have:
- necessary libraries and Boto3
- create Boto3 session
- initialize S3 client
- obtain list of python files in the set
- Iterate through the list of Python files and upload them to AWS S3 in groups of 100
- close the Boto3 session

Then you can create a cron job that will run the python script every 10 minutes to ensure that the python files are uploaded to S# in goups of 100 every 10 minutes.
- create the cron job using the commad:
       crontab -e
- in the crontab editor add:
       */10 * * * * python /path/to/python/script.py

Send set of Python files to AWS S3

Relevant content