- Newest
- Most votes
- Most comments
I think as your startup logs appear in the new log stream but invocation logs do not (despite a successful 200 OK response), this is likely not a permissions or buffering issue, but rather a problem with how the AgentCore Runtime manages Python’s standard streams during session transitions.
Potential Root Causes
1. Handler Context Isolation:
The @app.entrypoint decorator may be executing the handler in a spawned process or a thread where sys.stdout and sys.stderr are not correctly mapped to the CloudWatch-linked file descriptors of the new MicroVM. If the runtime "hijacks" these streams during the first session, it might fail to re-bind them for subsequent sessions.
2. The uv / Python 3.11 Slim Environment:
While you have PYTHONUNBUFFERED=1 set, the ARM64 slim image combined with the uv package manager might be interacting with the Firecracker (MicroVM) logging driver in a way that drops the stream if the handler doesn't explicitly flush to the underlying system file descriptor.
3. Logging Hierarchy:
If you are using the standard logging module, the BedrockAgentCoreApp might be resetting the root logger configuration after the first initialization.
Troubleshooting to confirm it’s a bug:
- Test with Low-Level Writes: Inside your handler, try writing directly to the file descriptor to bypass Python's internal buffering entirely:
import os
os.write(1, b"Direct FD1 Write: Handler is executing\n")
If this appears in CloudWatch but print() does not, I think the issue is within Python’s stream wrapping.
-
Explicit Logger Re-initialization: Instead of relying on a global logger, try configuring a StreamHandler(sys.stdout) inside the entrypoint function specifically for non-first sessions.
-
Verify MicroVM Recycling: Check if the "New Session" logs show a different
request_idorhost_idin the startup logs. If the Runtime is reusing a "warm" container but the entrypoint is losing its connection to the log-shipper, if so this is a bug that only AWS Support can fix. -
https://github.com/hashicorp/terraform-provider-aws/issues/44898
-
https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/runtime-troubleshooting.html
When you encounter missing CloudWatch Logs from your invocation handler for sessions after the first one, there are several troubleshooting steps to consider:
First, verify you're checking the correct log group. AgentCore stores logs under your agent's application logs or under your own custom log group when CloudWatch Logs is used as the destination. Make sure you're looking at the right location for all session logs.
If logs appear to be missing or empty, consider running your agent container locally to verify that logging is working as expected in your code. This can help isolate whether the issue is with your application logic or the AgentCore integration.
You may also want to enable verbose logging in your configuration to capture more detailed information about what's happening during invocations across different sessions.
Since AgentCore provides structured application logs that include fields like timestamp, session_id, trace_id, request_payload, and response_payload, you should be able to correlate logs across sessions. These logs are designed to help you gain visibility into agent runtime invocations and session-level resource consumption.
If you're still experiencing issues after these steps, it's worth noting that AgentCore can output logs to multiple destinations including CloudWatch Logs, Amazon S3, or Firehose stream. You might consider testing with an alternative destination to see if the behavior persists.
Additionally, ensure that your execution role has the proper permissions to write logs to CloudWatch, as permission issues can sometimes cause intermittent log delivery problems that might manifest differently across sessions.
For comprehensive troubleshooting, set up monitoring using CloudWatch metrics and alarms, enable comprehensive logging, and use structured error handling to better understand what's happening across all your sessions.
Sources
AgentCore generated runtime observability data - Amazon Bedrock AgentCore
Troubleshoot AgentCore Runtime - Amazon Bedrock AgentCore
