How to Programmatically Delete Amazon Q Data Integrations in Amazon Connect using Python and AWS Lambda

8 minute read
Content level: Advanced
1

This article demonstrates how to use Python and AWS Lambda to automate the deletion of Amazon Q data integrations in Amazon Connect. Using Python's boto3 library to call Amazon Connect APIs, this solution efficiently manages the cleanup of knowledge bases, assistant associations, and data integration resources in your Connect instance.

Prerequisites

  • An AWS account with access to AWS Lambda, Amazon Connect, and Amazon Q services`
  • Basic understanding of AWS Lambda and Python
  • AWS CLI configured with appropriate permissions

Solution Overview

Key Components

  • AWS Lambda Function: Orchestrates the deletion process
  • Amazon Connect Instance: Contains the data integration configurations
  • App Integrations Service: Manages data integration resources
  • Amazon Q Connect Service: Handles knowledge bases and assistant associations
  • EventBridge: Manages associated rules and targets created by Amazon Q

Main Workflow Steps

  • Parameter validation and client initialization
  • Data integration association discovery
  • Knowledge base and assistant association cleanup
  • Final data integration removal

Required IAM Permissions

The Lambda execution role requires the following permission policies :

  • AWSLambdaBasicExecutionRole for CloudWatch logging
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:<REGION>:<ACCOUNT_ID>:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:<REGION>:<ACCOUNT_ID>:log-group:/aws/lambda/DeleteConnectDataIntegrations:*"
            ]
        }
    ]
}
  • Connect permissions for managing integrations and associations
{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "VisualEditor0",
        "Effect": "Allow",
        "Action": [
            "wisdom:ListAssistantAssociations",
            "app-integrations:DeleteDataIntegration",
            "wisdom:DeleteKnowledgeBase",
            "app-integrations:ListDataIntegrations",
            "app-integrations:ListDataIntegrationAssociations",
            "connect:ListIntegrationAssociations",
            "wisdom:DeleteAssistantAssociation",
            "connect:DeleteIntegrationAssociation",
            "wisdom:ListKnowledgeBases",
            "app-integrations:DeleteDataIntegrationAssociation"
        ],
        "Resource": [
            "arn:aws:connect:<REGION>:<ACCOUNT_ID>:instance/*",
            "arn:aws:connect:<REGION>:<ACCOUNT_ID>:instance/*/integration-association/*",
            "arn:aws:app-integrations:<REGION>:<ACCOUNT_ID>:data-integration/*",
            "arn:aws:wisdom:<REGION>:<ACCOUNT_ID>:association/*/*",
            "arn:aws:wisdom:<REGION>:<ACCOUNT_ID>:knowledge-base/*",
            "arn:aws:wisdom:<REGION>:<ACCOUNT_ID>:assistant/*",
            "arn:aws:app-integrations:<REGION>:<ACCOUNT_ID>:data-integration-association/*/*"
        ]
    }
]
}
  • EventBridge permissions for managing rules and targets created by Wisdom
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Action": [
                "events:ListTargetsByRule",
                "events:RemoveTargets",
                "events:DeleteRule"
            ],
            "Resource": [
                "arn:aws:events:<REGION>:<ACCOUNT_ID>:rule/*"
            ]
        }
    ]
}
  • IAM permissions for updates on Connect Service Linked Role
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:UpdateRole",
                "iam:PutRolePolicy",
                "iam:DeleteRolePolicy"
            ],
            "Resource": "arn:aws:iam::<ACCOUNT_ID>:role/aws-service-role/connect.amazonaws.com/*"
        }
    ]
}

Ensure to replace '<ACCOUNT_ID>' and '<REGION>' placeholders with your account and region specific values.

Implementation Steps

  1. Create a new Lambda function :
  • Navigate to AWS Lambda console in the same region as your Amazon Connect instance
  • Click "Create function" and select "Author from scratch"
  • Configure basic settings:
- Runtime: Python 3.13
- Architecture: x86_64 (default)
  • Configure execution role:
- Select "Create a new role with basic Lambda permissions"
- After creation, update the role with the permissions specified in the IAM Permissions section
  • Configure function settings:
- Memory: 128 MB (minimum recommended)
- Timeout: 10 seconds
  1. Configure Environment Variables (Optional) :
  • INSTANCE_ID: Your Amazon Connect instance ID
  • DATA_INTEGRATION_NAME: Name of the data integration to delete
  1. Deploy the Lambda Code :
  • Copy the below provided Python code into the Lambda function's code editor and deploy the function..
import boto3
import json
import os
import logging
import uuid

# Configure logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def generate_request_id():
    """Generate a unique request ID for tracking"""
    return str(uuid.uuid4())

def get_parameters(event, context):
    """Get parameters from both environment variables and event payload"""
    region = context.invoked_function_arn.split(":")[3]
    logger.info(f"Lambda is running in region: {region}")
    
    params = {
        'instance_id': event.get('instance_id') or os.environ.get('INSTANCE_ID') or os.environ.get('instance_id'),
        'data_integration_name': event.get('data_integration_name') or os.environ.get('DATA_INTEGRATION_NAME') or os.environ.get('data_integration_name'),
        'region': region
    }
    return params

def initialize_clients(region):
    """Initialize AWS service clients"""
    try:
        return {
            'appintegrations': boto3.client('appintegrations', region_name=region),
            'connect': boto3.client('connect', region_name=region),
            'qconnect': boto3.client('qconnect', region_name=region)
        }
    except Exception as e:
        logger.error(f"Error initializing AWS clients: {str(e)}")
        raise

def list_data_integrations(client):
    """List all available data integrations"""
    try:
        response = client.list_data_integrations()
        integration_names = [integration['Name'] for integration in response.get('DataIntegrations', [])]
        return integration_names
    except Exception as e:
        logger.error(f"Error listing data integrations: {str(e)}")
        raise

def delete_data_integration(clients, data_integration_name, instance_id, request_id):
    """Delete data integration and associated resources"""
    try:
        # Log request parameters with request_id
        logger.info(f"RequestId: {request_id} - Starting deletion process for Data Integration: {data_integration_name}")
        
        # Step 1: Check data integration associations
        data_integration_assocs = clients['appintegrations'].list_data_integration_associations(
            DataIntegrationIdentifier=data_integration_name
        )
        logger.info(f"Data integration associations check completed for: {data_integration_name}")

        # Step 2: List integration associations
        integration_assocs = clients['connect'].list_integration_associations(
            InstanceId=instance_id
        )
        
        # Step 3: Handle WISDOM_KNOWLEDGE_BASE association
        kb_association_id = None
        assistant_arn = None
        
        for assoc in integration_assocs.get('IntegrationAssociationSummaryList', []):
            if assoc['IntegrationType'] == 'WISDOM_KNOWLEDGE_BASE':
                kb_association_id = assoc['IntegrationAssociationId']
            elif assoc['IntegrationType'] == 'WISDOM_ASSISTANT':
                assistant_arn = assoc['IntegrationArn']

        if kb_association_id:
            logger.info(f"Deleting QCONNECT_KNOWLEDGE_BASE association: {kb_association_id}")
            clients['connect'].delete_integration_association(
                InstanceId=instance_id,
                IntegrationAssociationId=kb_association_id
            )

        # Step 4 & 5: Handle WISDOM_ASSISTANT associations
        if assistant_arn:
            assistant_id = assistant_arn.split('/')[-1]
            assistant_assocs = clients['qconnect'].list_assistant_associations(
                assistantId=assistant_id
            )
            
            for assoc in assistant_assocs.get('assistantAssociationSummaries', []):
                logger.info(f"Deleting qconnect assistant association: {assoc['assistantAssociationId']}")
                clients['qconnect'].delete_assistant_association(
                    assistantId=assistant_id,
                    assistantAssociationId=assoc['assistantAssociationId']
                )

        # Step 6 & 7: Handle WISDOM knowledge bases
        knowledge_bases = clients['qconnect'].list_knowledge_bases()
        
        for kb in knowledge_bases.get('knowledgeBaseSummaries', []):
            if kb['name'] == data_integration_name:
                logger.info(f"Deleting qconnect knowledge base: {kb['knowledgeBaseId']}")
                clients['qconnect'].delete_knowledge_base(
                    knowledgeBaseId=kb['knowledgeBaseId']
                )

        # Step 8 & 9: Final verification and deletion of data-integrations
        verify_assocs = clients['appintegrations'].list_data_integration_associations(
            DataIntegrationIdentifier=data_integration_name
        )
        
        if not verify_assocs.get('DataIntegrationAssociations'):
            clients['appintegrations'].delete_data_integration(
                DataIntegrationIdentifier=data_integration_name
            )
            logger.info(f"Successfully deleted data integration: {data_integration_name}")
            return True
        else:
            logger.warning("Data integration associations still exist")
            return False

    except Exception as e:
        logger.error(f"RequestId: {request_id} - Error during deletion process: {str(e)}")
        logger.error(f"RequestId: {request_id} - Failed operation details - Data Integration: {data_integration_name}")
        raise

def get_error_details(error):
    """Extract error type and operation from boto3 exception"""
    error_message = str(error)
    if hasattr(error, 'operation_name'):
        return f"An error occurred ({error.__class__.__name__}) when calling the {error.operation_name} operation"
    return error_message

def lambda_handler(event, context):
    request_id = generate_request_id()
    try:
        # Get parameters
        params = get_parameters(event, context)
        
        # Initialize clients
        clients = initialize_clients(params['region'])

        # Check if parameters are available
        if not params['instance_id'] or not params['data_integration_name']:
            integration_names = list_data_integrations(clients['appintegrations'])
            missing_params = []
            if not params['instance_id']:
                missing_params.append('instance_id')
            if not params['data_integration_name']:
                missing_params.append('data_integration_name')
                
            logger.error(f"RequestId: {request_id} - Missing required parameters: {missing_params}")
            return {
                'statusCode': 400,
                'body': json.dumps({
                    'error': 'Missing required parameters',
                    'message': 'Please provide all required parameters. Check logs for details.',
                    'requestId': request_id
                }, separators=(',', ':'))  # Remove extra whitespace
            }

        # Proceed with deletion
        success = delete_data_integration(
            clients,
            params['data_integration_name'],
            params['instance_id'],
            request_id
        )

        return {
            'statusCode': 200 if success else 400,
            'body': json.dumps({
                'success': success,
                'message': f"Data integration: {params['data_integration_name']} deleted successfully" if success else f"Failed to delete data integration: {params['data_integration_name']}",
                'requestId': request_id
            }, separators=(',', ':'))
        }

    except Exception as e:
        logger.error(f"RequestId: {request_id} - Lambda execution error: {str(e)}")
        error_message = get_error_details(e)
        
        return {
            'statusCode': 500,
            'body': json.dumps({
                'error': 'Internal server error',
                'message': f"{error_message}. Check logs for details.",
                'requestId': request_id
            }, separators=(',', ':'))
        }
  • The code handles the below
- Parameter validation
- AWS service client initialization
- Connect Data integration association cleanup
- QConnect knowledge base and assistant association removal
- Final data integration deletion
- Includes proper error handling and logging mechanisms to ensure reliable execution and troubleshooting capabilities.
  1. Execute the Function :

You can provide the input parameters in either of the two ways :

  • Environment variables (as mentioned in step 2) or
  • JSON event format:
{ 
"instance_id": "your-instance-id", 
"data_integration_name": "your-integration-name" 
} 

Testing and Validation

Sample Test Cases

Basic deletion test event:

{
    "instance_id": "your-instance-id",
    "data_integration_name": "test-integration"
}

Expected Outputs

{
    "statusCode": 200,
    "body": {
        "success": true,
        "message": "Data integration: test-integration deleted successfully",
        "requestId": "uuid-value"
    }
}

Common Error Scenario

Missing Parameters:

{
    "statusCode": 400,
    "body": {
        "error": "Missing required parameters",
        "message": "Please provide all required parameters. Check logs for details.",
        "requestId": "uuid-value"
    }
}

Best Practices

Security

  • Follow least privilege principle for IAM roles
  • Encrypt sensitive environment variables

Performance

  • Implement concurrent deletion where possible
  • Use proper timeout settings
  • Implement pagination for large resource lists

Limitations

  • Knowledge bases: hard limit of 10
  • Data integrations: soft limit of 10 (can be increased via Service Quotas console)
  • Region availability depends on Amazon Q service availability

Troubleshooting

Common Issues

  1. Permission errors
    • Solution: Verify IAM role permissions
  2. Resource not found
    • Solution: Check resource existence and region
  3. Deletion failures
    • Solution: Check cloudwatch logs and resource dependencies

Debug Procedures

  1. Enable DEBUG level logging if required.
  2. Check CloudWatch logs for detailed error messages.
  3. Verify AWS service health status.
  4. Confirm resource state before deletion.

Clean-up Steps

  1. Verify resource deletion using AWS CLI or AWS CloudShell
aws appintegrations list-data-integrations
aws qconnect list-knowledge-bases
  1. Delete Lambda function if no longer needed.
  2. Review and clean up IAM roles.

Reference Documentation