- Newest
- Most votes
- Most comments
Logging is currently controlled by the service and sent with a fixed JSON format to CloudWatch. Logs from Job Manager and all Task Managers are sent to the same CloudWatch stream. The logging level can be set as part of the application configuration, to DEBUG, INFO, WARN, or ERROR and applies to all loggers . At the moment MSF ignores any logging configuration file or runtime properties set by the application.
The service doesn't natively support package-level log configuration like you would in a standard Java application. The logging level is set at the application level.
[+] https://docs.aws.amazon.com/managed-flink/latest/java/logging.html
In Amazon Managed Service for Apache Flink, you can set the global log level using the LogLevel parameter when creating or updating an application (via CreateApplication or UpdateApplication actions). The available log levels are ERROR, WARN, INFO, and DEBUG.
However, there are important considerations when working with logging in Managed Service for Apache Flink:
-
The service doesn't natively support package-level log configuration like you would in a standard Java application. The logging level is set at the application level.
-
Extensive logging can significantly impact performance. The logging subsystem needs to collect and forward log entries to CloudWatch Logs, and excessive logging can cause your Flink application to fall behind.
-
The DEBUG log level generates a large amount of traffic and can create backpressure. It should only be used temporarily while actively investigating issues.
For your specific need (having third-party code log at WARN while your code logs at DEBUG), you have a few options:
-
Set the global log level to WARN and programmatically increase the verbosity for your specific classes using Log4j or SLF4J in your code.
-
Instead of using the logging system for detailed debugging of your code, consider creating a separate DataStream within your Flink application and use a proper sink to send your debug data to Amazon S3 or CloudWatch. This approach is more aligned with Flink's architecture for high throughput and low latency.
-
If you must use logging for debugging, keep the global level at WARN for production and only temporarily increase it to DEBUG when actively investigating issues.
Remember that Flink is optimized for high throughput and low latency, but the logging subsystem is not. Logging exceptions and warnings is appropriate, but generating log messages for every processed message is not recommended through the standard logging system.
Sources
Logging in Managed Service for Apache Flink - Managed Service for Apache Flink
Set CloudWatch metrics reporting levels - Managed Service for Apache Flink
Write custom messages to CloudWatch Logs - Managed Service for Apache Flink
What do you mean when you say "increase the verbosity for your specific classes using Log4j or SLF4J in your code"? I tried Configurator.setLevel("com.example", Level.DEBUG); Configurator.setLevel("com.example.foo.MyClass", Level.DEBUG); This commands do exactly what I want when running locally but seem to have no effect in Managed Flink environment.
Relevant content
- asked 3 months ago
I certainly understand the need for AWS to control the format for CloudWatch, but it sure would be nice if they provided a way for the dev to control the levels for individual packages. There's plenty of cases where you need to crank up the level on your code but don't want to see it for 3rd party libs. 25 years of log framework development are centered on that as the most important feature of a log framework, so it's annoying that they've ignored that.