Skip to content

AgentCore Runtime is retrying every minute, and I don't know why

0

Hi everyone, As title suggests, I deployed a custom agent built with Strands on AgentCore

Here is the structure of my code

app = BedrockAgentCoreApp()

bedrock_client = session.client("bedrock-runtime", region_name=AWS_REGION)

@tool
tool1

@tool
tool2

def process_user_input(user_input: str):
    """
    user input processing
    """
    
    agent = Agent(
        model=model,
        system_prompt=system_prompt,
        tools=[tool1, tool2],
        trace_attributes={"session.id": SESSION_ID},
        callback_handler= None
    )

    response = agent(user_input)
    
    if hasattr(response, 'message') and 'content' in response.message:
        response_text = response.message['content'][0]['text']
    else:
        response_text = str(response)

    return response_text


@app.entrypoint
def main(payload: dict):
    logger.info("starting processing")
    logger.info(f"Received payload: {payload}")
    
    user_input = payload.get("prompt", "").strip()
    
    response_text = process_user_input(user_input)
    
    logger.info(f"Response: {response_text}")
    return {"result": response_text}
    
if __name__ == "__main__":
    app.run()

What happens is that, through the CloudWatch of the AgentCore Runtime, I see that the entrypoint is called after every 60 seconds, preventing the user to have the results if the processing takes more than 60 seconds.

However, when deployed locally, this problem doesn't arise.

How can I deal with this situation? I don't find docs on AWS presenting this "retry feature" of AgentCore.

2 Answers
0

The issue you're experiencing is related to Amazon Bedrock AgentCore's session timeout mechanism. By default, AgentCore automatically terminates sessions after 15 minutes of inactivity. However, what's likely happening in your case is that your agent is processing tasks that take longer than expected, but you haven't implemented a way to tell AgentCore that your agent is still actively working.

For long-running operations, AgentCore needs to be informed that your agent is still processing through a "ping" mechanism. Without this, AgentCore might assume your agent is idle or stuck and attempt to retry the operation.

To fix this issue, you should implement a ping handler for your long-running tasks. Here's how you can modify your code:

import asyncio
from bedrock_agentcore.runtime import BedrockAgentCoreApp

app = BedrockAgentCoreApp()

@app.entrypoint
async def main(payload: dict, context):
logger.info("starting processing")
logger.info(f"Received payload: {payload}")

# Create a flag to track if processing is still ongoing
task_running = True

# Define ping handler to keep the session alive
async def ping_handler():
while task_running:
await context.ping(status="HEALTHY_BUSY")
await asyncio.sleep(30)  # Ping every 30 seconds

# Start ping handler in background
ping_task = asyncio.create_task(ping_handler())

try:
user_input = payload.get("prompt", "").strip()
response_text = await process_user_input(user_input)
logger.info(f"Response: {response_text}")
return {"result": response_text}
finally:
# Signal ping handler to stop and clean up
task_running = False
ping_task.cancel()

You'll need to make your process_user_input function asynchronous as well. This implementation tells AgentCore that your agent is still actively processing by sending "HEALTHY_BUSY" status pings every 30 seconds.

Alternatively, if you're using the latest SDK version, you can use the API-based task management approach:

@app.entrypoint
async def main(payload: dict, context):
logger.info("starting processing")
logger.info(f"Received payload: {payload}")

# Register the task as async
task_id = context.add_async_task("Processing user input")

try:
user_input = payload.get("prompt", "").strip()
response_text = await process_user_input(user_input)
logger.info(f"Response: {response_text}")
return {"result": response_text}
finally:
# Mark task as complete
context.complete_async_task(task_id)

The reason you don't see this issue locally is likely because your local environment doesn't enforce the same timeout mechanisms as the deployed AgentCore Runtime service.
Sources
Troubleshoot AgentCore Runtime - Amazon Bedrock AgentCore
Handle asynchronous and long running agents with Amazon Bedrock AgentCore Runtime - Amazon Bedrock AgentCore

answered 9 months ago

0

Hey,

Hope you're keeping well.

AgentCore invokes your entrypoint via the Bedrock agent runtime, which has a default request timeout of 60 seconds. If your processing exceeds that, the runtime will treat it as a failure and retry according to its delivery policy, which is why you see the repeated calls in CloudWatch. To handle long-running tasks, you’ll need to offload work to an asynchronous workflow such as invoking an AWS Step Functions state machine or publishing to an SQS queue, then return an immediate acknowledgment and deliver results separately. You can also review your Agent manifest configuration in the Bedrock console to see if the timeout parameter can be adjusted for your agent.

Thanks and regards,
Taz

answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.