stdout logging from Kubernetes pods

0

Hi,

I ran into an issue with our logging from Kubernetes pods. We store logging in JSON format, but saw that with big JSON logs it was breaking. So I investigated the isssue. Found out that when you are sending big logs from the pods with for example cat log/test.log > /proc/1/fd/1

It will ends in multiple parts into the node log folder (/var/log/pods/pod) 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

This breaks the JSON. How can we avoid that sending big log files to /proc/1/fd/1, will be splitted into multiple parts?

Thank you!

已提问 1 年前667 查看次数
1 回答
0

I believe this is by design, files larger than 16K. Fluent has options like (https://docs.fluentbit.io/manual/v/1.3/input/tail#docker_mode) docker mode if you are using that which tries to recombine.

profile pictureAWS
Roly
已回答 1 年前
  • Yes, I'm using Fluent-Bit, and also has the docker mode. But I find that it stuggles to combine those again.

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则