lambda error when getting a trigger event from an S3 bucket in another account and reading the object

The error you're encountering is likely due to a combination of factors related to cross-account access and error handling. Let's break down the issues and potential solutions:

1. Cross-account access: When accessing an S3 bucket from another account, you need to ensure that proper permissions are in place. The Lambda function's execution role must have the necessary permissions to access the S3 bucket in the other account, and the bucket policy must allow access from your Lambda function's account.

2. Error handling: Your current error handling approach is not capturing the specific exception that occurs when calling get_object. This results in the "local variable 'obj' referenced before assignment" error because when an exception occurs, the 'obj' variable is never assigned.

3. Connection issues: The error suggests there might be intermittent connection issues when trying to access the S3 bucket.

To address these issues, consider the following improvements:

1. Verify cross-account permissions: Ensure that the Lambda function's execution role has the necessary permissions to access the S3 bucket in the other account, and that the bucket policy allows access from your Lambda function's account.

2. Improve error handling: Catch specific exceptions and log them for better debugging. For example:

try:
    obj = s3.get_object(Bucket=bucket, Key=key)
except s3.exceptions.NoSuchKey:
    print(f"Object {key} not found in bucket {bucket}")
    return
except s3.exceptions.AccessDenied:
    print(f"Access denied to object {key} in bucket {bucket}")
    return
except Exception as e:
    print(f"An error occurred: {str(e)}")
    return

# Only proceed if obj was successfully retrieved
content = obj['Body'].read()

3. Implement retries: Consider implementing a retry mechanism with exponential backoff for transient errors:

import boto3
from botocore.exceptions import ClientError
import time

def get_object_with_retry(s3_client, bucket, key, max_retries=3):
    for attempt in range(max_retries):
        try:
            return s3_client.get_object(Bucket=bucket, Key=key)
        except ClientError as e:
            if e.response['Error']['Code'] in ['SlowDown', 'ProvisionedThroughputExceededException']:
                if attempt < max_retries - 1:
                    time.sleep(2 ** attempt)  # Exponential backoff
                    continue
            raise

4. Check S3 client initialization: Ensure that the S3 client is properly initialized before the handler function. If there's a connection error during client initialization, your current code will set s3 to None, causing issues later.

By implementing these improvements, you should be able to handle errors more gracefully and potentially reduce the number of failures when reading objects from the cross-account S3 bucket.
Sources
Lambda function cannot access S3 bucket, even after granting IAM role/policy | AWS re:Post
Lambda examples using SDK for JavaScript (v3) - AWS SDK Code Examples

Does this work? Move the S3 object into the handler

import boto3
import gzip
import os
from io import BytesIO


def lambda_handler(event, context):
    s3 = boto3.client('s3')

    # Log the event for debugging
    print(f"Received event: {event}")
    
    # Extract bucket name and object key from the event
    bucket_name = event['Records'][0]['s3']['bucket']['name']
    object_key = event['Records'][0]['s3']['object']['key']
    
    # Generate the GZIP file key
    gzip_key = f"{object_key}.gz"
    
    try:
        # Download the file from S3
        response = s3.get_object(Bucket=bucket_name, Key=object_key)
        file_content = response['Body'].read()
        
        # Compress the file content using GZIP
        with BytesIO() as gzip_buffer:
            with gzip.GzipFile(fileobj=gzip_buffer, mode='wb') as gz_file:
                gz_file.write(file_content)
            
            gzip_content = gzip_buffer.getvalue()
        
        # Upload the GZIP file back to the same bucket
        s3.put_object(Bucket=bucket_name, Key=gzip_key, Body=gzip_content, ContentType='application/gzip')
        
        print(f"Compressed and uploaded {gzip_key} to {bucket_name}")
        return {
            'statusCode': 200,
            'body': f"Compressed {object_key} and saved as {gzip_key} in {bucket_name}"
        }
    except Exception as e:
        print(f"Error processing file {object_key} from bucket {bucket_name}: {e}")
        return {
            'statusCode': 500,
            'body': f"Error compressing file: {e}"
        }