stdout logging from Kubernetes pods

0

Hi,

I ran into an issue with our logging from Kubernetes pods. We store logging in JSON format, but saw that with big JSON logs it was breaking. So I investigated the isssue. Found out that when you are sending big logs from the pods with for example cat log/test.log > /proc/1/fd/1

It will ends in multiple parts into the node log folder (/var/log/pods/pod) 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY 2023-03-24T11:53:33.107458625Z stdout P YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

This breaks the JSON. How can we avoid that sending big log files to /proc/1/fd/1, will be splitted into multiple parts?

Thank you!

gefragt vor einem Jahr666 Aufrufe
1 Antwort
0

I believe this is by design, files larger than 16K. Fluent has options like (https://docs.fluentbit.io/manual/v/1.3/input/tail#docker_mode) docker mode if you are using that which tries to recombine.

profile pictureAWS
Roly
beantwortet vor einem Jahr
  • Yes, I'm using Fluent-Bit, and also has the docker mode. But I find that it stuggles to combine those again.

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen