server access logging in S3 - granularity per file

Question

When enabling server access logging on an S3 bucket, logs are being delivered to the destination bucket as a single file for each request made to the source bucket. This generates a high number of files which is not optimal for our workflow.
I would like to know if there is a way to receive log files containing information about multiple requests instead of just one or if there is another option to consolidate these logs in bigger files

Jamesson Barreto · Answer

Hello,

AWS S3 server access logging does not provide a native way to control the log file size or consolidate multiple requests into single log files.

Riku_Kobayashi · Answer

Hello.

As far as I know, there is no setting in the S3 access log settings to aggregate logs into one.  
Therefore, you will need to create a system to aggregate logs using Lambda or similar, as explained in the following Japanese blog.  
https://qiita.com/take_o/items/c6fb98a4a5f86e42feec

server access logging in S3 - granularity per file

Add your answer

Relevant content