Amazon Bedrock Streaming API Response Contains Unexpected Control Characters & Event Metadata

The issue you're experiencing is related to how you're processing the streaming response from Amazon Bedrock. The raw response you're seeing contains Server-Sent Events (SSE) format, which includes event metadata and control characters. This is expected behavior for streaming responses, but it requires proper parsing to extract the actual content.

To correctly handle this response, you need to parse the SSE format. Each event in the stream starts with "data:" and ends with two newline characters. The content between these markers is JSON-encoded data that you need to parse.

Here's how you can modify your code to properly handle the streaming response:

1. Use a library that supports SSE parsing, such as sseclient-py.
2. Parse each event and extract the JSON content.
3. Process the JSON content based on the event type.

Here's an example of how you can modify your code:

import json
import sseclient
import requests
from requests_aws4auth import AWS4Auth

# AWS Credentials and setup (same as before)

# Function to handle the streaming response
def handle_sse_stream(url, headers, payload, auth):
    response = requests.post(url, stream=True, headers=headers, json=payload, auth=auth)
    client = sseclient.SSEClient(response)
    
    for event in client.events():
        if event.data:
            try:
                data = json.loads(event.data)
                if 'contentBlockIndex' in data and 'delta' in data:
                    print(data['delta'].get('text', ''), end='')
                elif 'metrics' in data:
                    print("\nMetrics:", data['metrics'])
                    print("Usage:", data['usage'])
            except json.JSONDecodeError:
                print("Error decoding JSON:", event.data)

# Make the request
handle_sse_stream(url, headers, payload, aws_auth)

This code will properly parse the SSE stream, extract the JSON content, and handle different types of events (content deltas and metrics). It will print the text content as it arrives and display the metrics at the end.

The unexpected control characters and event metadata you were seeing are part of the SSE format, which is used to structure the streaming response. By using a proper SSE client and parsing the events, you can extract the meaningful content and ignore the SSE-specific formatting.

Remember to install the sseclient-py library (pip install sseclient-py) before running this code.

This approach should resolve the issues you were experiencing and provide you with a clean output of the model's response.

Sources
Community | Process Amazon Bedrock's Response Stream with JavaScript
InvokeFlow - Amazon Bedrock

Hello,

The error you're encountering is due to receiving raw encypted data from Amazon Bedrock's Streaming API. Your current code is printing this raw data directly, which includes control characters, event information, and other metadata alongside the actual response content without proper parsing of the raw data which comes in chunks in case of Bedrock's Nova model while using Converse Stream.

Here is a Code which is designed to handle streaming responses from Bedrock's Nova model, where the "extract_text" function searches for text between each received chunk of data (1024 bytes), then decodes and displays it in real-time while also accumulating the complete response.

import requests
import json
from requests_aws4auth import AWS4Auth
import struct

# AWS Credentials
AWS_ACCESS_KEY = "Your_Access_key"
AWS_SECRET_KEY = "Your_secter_key"
REGION = "us-east-1"
MODEL_ID = "amazon.nova-pro-v1:0"


# API Endpoint
url = f"https://bedrock-runtime.{REGION}.amazonaws.com/model/{MODEL_ID}/converse-stream"

# AWS4Auth for request signing
auth = AWS4Auth(AWS_ACCESS_KEY, AWS_SECRET_KEY, REGION, 'bedrock')

# Request Headers
headers = {
    "Content-Type": "application/json",
    "Accept": "application/json"
}

# Request Payload
payload = {
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "text": "Tell me about artificial intelligence."
                }
            ]
        }
    ],
    "system": [{"text": "You are an economist with access to lots of data"}],
    "inferenceConfig": {
        "maxTokens": 1000,
        "temperature": 0.5
    }
}

def extract_text(data):
    try:
        # Look for the text field in the JSON data
        json_start = data.find(b'"text":"')
        if json_start != -1:
            json_start += 8  # Length of '"text":"'
            json_end = data.find(b'"', json_start)
            if json_end != -1:
                return data[json_start:json_end].decode('utf-8', errors='ignore')
    except Exception as e:
        print(f"Error extracting text: {e}")
    return None

def converse_stream():
    try:
        print("Sending request...")
        response = requests.post(url, auth=auth, headers=headers, json=payload, stream=True)
        print(f"Response status code: {response.status_code}")
        
        if response.status_code != 200:
            print(f"Error response: {response.text}")
            return

        print("Streaming response:")
        full_response = ""
        print(vars(response))
        # print("above is response")

        for chunk in response.iter_content(chunk_size=1024):
            if chunk:
                text = extract_text(chunk)
                if text:
                    full_response += text
                    print(text, end='', flush=True)

        print("\n\nFull response:")
        print(full_response)

    except requests.exceptions.RequestException as e:
        print(f"An error occurred: {str(e)}")
        if hasattr(e, 'response') and e.response is not None:
            print(f"Status code: {e.response.status_code}")
            print(f"Response content: {e.response.text}")

if __name__ == "__main__":
    converse_stream()

Using this code, you'll be able to interact with Amazon Bedrock's Nova model through a streaming API connection, which enables real-time communication with the AI model. Instead of waiting for the entire response to be generated and delivered at once, the code receives and processes the response in chunks of 1024 bytes, immediately displaying each piece of text as it arrives. This creates a more dynamic and interactive experience. The code also maintains error handling and keeps track of the full response while providing this streaming functionality.