stdout logging from Kubernetes pods

0

Hi,

I ran into an issue with our logging from Kubernetes pods. We store logging in JSON format, but saw that with big JSON logs it was breaking. So I investigated the isssue. Found out that when you are sending big logs from the pods with for example cat log/test.log > /proc/1/fd/1

It will ends in multiple parts into the node log folder (/var/log/pods/pod) 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

This breaks the JSON. How can we avoid that sending big log files to /proc/1/fd/1, will be splitted into multiple parts?

Thank you!

已提問 1 年前檢視次數 667 次
1 個回答
0

I believe this is by design, files larger than 16K. Fluent has options like (https://docs.fluentbit.io/manual/v/1.3/input/tail#docker_mode) docker mode if you are using that which tries to recombine.

profile pictureAWS
Roly
已回答 1 年前
  • Yes, I'm using Fluent-Bit, and also has the docker mode. But I find that it stuggles to combine those again.

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南