stdout logging from Kubernetes pods

0

Hi,

I ran into an issue with our logging from Kubernetes pods. We store logging in JSON format, but saw that with big JSON logs it was breaking. So I investigated the isssue. Found out that when you are sending big logs from the pods with for example cat log/test.log > /proc/1/fd/1

It will ends in multiple parts into the node log folder (/var/log/pods/pod) 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

This breaks the JSON. How can we avoid that sending big log files to /proc/1/fd/1, will be splitted into multiple parts?

Thank you!

feita há um ano667 visualizações
1 Resposta
0

I believe this is by design, files larger than 16K. Fluent has options like (https://docs.fluentbit.io/manual/v/1.3/input/tail#docker_mode) docker mode if you are using that which tries to recombine.

profile pictureAWS
Roly
respondido há um ano
  • Yes, I'm using Fluent-Bit, and also has the docker mode. But I find that it stuggles to combine those again.

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas