Complete a 3 Question Survey and Earn a re:Post Badge
Help improve AWS Support Official channel in re:Post and share your experience - complete a quick three-question survey to earn a re:Post badge!
How to Programmatically Delete Amazon Q Data Integrations in Amazon Connect using Python and AWS Lambda
This article demonstrates how to use Python and AWS Lambda to automate the deletion of Amazon Q data integrations in Amazon Connect. Using Python's boto3 library to call Amazon Connect APIs, this solution efficiently manages the cleanup of knowledge bases, assistant associations, and data integration resources in your Connect instance.
Prerequisites
- An AWS account with access to AWS Lambda, Amazon Connect, and Amazon Q services`
- Basic understanding of AWS Lambda and Python
- AWS CLI configured with appropriate permissions
Solution Overview
Key Components
- AWS Lambda Function: Orchestrates the deletion process
- Amazon Connect Instance: Contains the data integration configurations
- App Integrations Service: Manages data integration resources
- Amazon Q Connect Service: Handles knowledge bases and assistant associations
- EventBridge: Manages associated rules and targets created by Amazon Q
Main Workflow Steps
- Parameter validation and client initialization
- Data integration association discovery
- Knowledge base and assistant association cleanup
- Final data integration removal
Required IAM Permissions
The Lambda execution role requires the following permission policies :
- AWSLambdaBasicExecutionRole for CloudWatch logging
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "arn:aws:logs:<REGION>:<ACCOUNT_ID>:*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:<REGION>:<ACCOUNT_ID>:log-group:/aws/lambda/DeleteConnectDataIntegrations:*"
]
}
]
}
- Connect permissions for managing integrations and associations
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"wisdom:ListAssistantAssociations",
"app-integrations:DeleteDataIntegration",
"wisdom:DeleteKnowledgeBase",
"app-integrations:ListDataIntegrations",
"app-integrations:ListDataIntegrationAssociations",
"connect:ListIntegrationAssociations",
"wisdom:DeleteAssistantAssociation",
"connect:DeleteIntegrationAssociation",
"wisdom:ListKnowledgeBases",
"app-integrations:DeleteDataIntegrationAssociation"
],
"Resource": [
"arn:aws:connect:<REGION>:<ACCOUNT_ID>:instance/*",
"arn:aws:connect:<REGION>:<ACCOUNT_ID>:instance/*/integration-association/*",
"arn:aws:app-integrations:<REGION>:<ACCOUNT_ID>:data-integration/*",
"arn:aws:wisdom:<REGION>:<ACCOUNT_ID>:association/*/*",
"arn:aws:wisdom:<REGION>:<ACCOUNT_ID>:knowledge-base/*",
"arn:aws:wisdom:<REGION>:<ACCOUNT_ID>:assistant/*",
"arn:aws:app-integrations:<REGION>:<ACCOUNT_ID>:data-integration-association/*/*"
]
}
]
}
- EventBridge permissions for managing rules and targets created by Wisdom
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"events:ListTargetsByRule",
"events:RemoveTargets",
"events:DeleteRule"
],
"Resource": [
"arn:aws:events:<REGION>:<ACCOUNT_ID>:rule/*"
]
}
]
}
- IAM permissions for updates on Connect Service Linked Role
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:GetRole",
"iam:UpdateRole",
"iam:PutRolePolicy",
"iam:DeleteRolePolicy"
],
"Resource": "arn:aws:iam::<ACCOUNT_ID>:role/aws-service-role/connect.amazonaws.com/*"
}
]
}
Ensure to replace '<ACCOUNT_ID>' and '<REGION>' placeholders with your account and region specific values.
Implementation Steps
- Create a new Lambda function :
- Navigate to AWS Lambda console in the same region as your Amazon Connect instance
- Click "Create function" and select "Author from scratch"
- Configure basic settings:
- Runtime: Python 3.13
- Architecture: x86_64 (default)
- Configure execution role:
- Select "Create a new role with basic Lambda permissions"
- After creation, update the role with the permissions specified in the IAM Permissions section
- Configure function settings:
- Memory: 128 MB (minimum recommended)
- Timeout: 10 seconds
- Configure Environment Variables (Optional) :
- INSTANCE_ID: Your Amazon Connect instance ID
- DATA_INTEGRATION_NAME: Name of the data integration to delete
- Deploy the Lambda Code :
- Copy the below provided Python code into the Lambda function's code editor and deploy the function..
import boto3
import json
import os
import logging
import uuid
# Configure logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def generate_request_id():
"""Generate a unique request ID for tracking"""
return str(uuid.uuid4())
def get_parameters(event, context):
"""Get parameters from both environment variables and event payload"""
region = context.invoked_function_arn.split(":")[3]
logger.info(f"Lambda is running in region: {region}")
params = {
'instance_id': event.get('instance_id') or os.environ.get('INSTANCE_ID') or os.environ.get('instance_id'),
'data_integration_name': event.get('data_integration_name') or os.environ.get('DATA_INTEGRATION_NAME') or os.environ.get('data_integration_name'),
'region': region
}
return params
def initialize_clients(region):
"""Initialize AWS service clients"""
try:
return {
'appintegrations': boto3.client('appintegrations', region_name=region),
'connect': boto3.client('connect', region_name=region),
'qconnect': boto3.client('qconnect', region_name=region)
}
except Exception as e:
logger.error(f"Error initializing AWS clients: {str(e)}")
raise
def list_data_integrations(client):
"""List all available data integrations"""
try:
response = client.list_data_integrations()
integration_names = [integration['Name'] for integration in response.get('DataIntegrations', [])]
return integration_names
except Exception as e:
logger.error(f"Error listing data integrations: {str(e)}")
raise
def delete_data_integration(clients, data_integration_name, instance_id, request_id):
"""Delete data integration and associated resources"""
try:
# Log request parameters with request_id
logger.info(f"RequestId: {request_id} - Starting deletion process for Data Integration: {data_integration_name}")
# Step 1: Check data integration associations
data_integration_assocs = clients['appintegrations'].list_data_integration_associations(
DataIntegrationIdentifier=data_integration_name
)
logger.info(f"Data integration associations check completed for: {data_integration_name}")
# Step 2: List integration associations
integration_assocs = clients['connect'].list_integration_associations(
InstanceId=instance_id
)
# Step 3: Handle WISDOM_KNOWLEDGE_BASE association
kb_association_id = None
assistant_arn = None
for assoc in integration_assocs.get('IntegrationAssociationSummaryList', []):
if assoc['IntegrationType'] == 'WISDOM_KNOWLEDGE_BASE':
kb_association_id = assoc['IntegrationAssociationId']
elif assoc['IntegrationType'] == 'WISDOM_ASSISTANT':
assistant_arn = assoc['IntegrationArn']
if kb_association_id:
logger.info(f"Deleting QCONNECT_KNOWLEDGE_BASE association: {kb_association_id}")
clients['connect'].delete_integration_association(
InstanceId=instance_id,
IntegrationAssociationId=kb_association_id
)
# Step 4 & 5: Handle WISDOM_ASSISTANT associations
if assistant_arn:
assistant_id = assistant_arn.split('/')[-1]
assistant_assocs = clients['qconnect'].list_assistant_associations(
assistantId=assistant_id
)
for assoc in assistant_assocs.get('assistantAssociationSummaries', []):
logger.info(f"Deleting qconnect assistant association: {assoc['assistantAssociationId']}")
clients['qconnect'].delete_assistant_association(
assistantId=assistant_id,
assistantAssociationId=assoc['assistantAssociationId']
)
# Step 6 & 7: Handle WISDOM knowledge bases
knowledge_bases = clients['qconnect'].list_knowledge_bases()
for kb in knowledge_bases.get('knowledgeBaseSummaries', []):
if kb['name'] == data_integration_name:
logger.info(f"Deleting qconnect knowledge base: {kb['knowledgeBaseId']}")
clients['qconnect'].delete_knowledge_base(
knowledgeBaseId=kb['knowledgeBaseId']
)
# Step 8 & 9: Final verification and deletion of data-integrations
verify_assocs = clients['appintegrations'].list_data_integration_associations(
DataIntegrationIdentifier=data_integration_name
)
if not verify_assocs.get('DataIntegrationAssociations'):
clients['appintegrations'].delete_data_integration(
DataIntegrationIdentifier=data_integration_name
)
logger.info(f"Successfully deleted data integration: {data_integration_name}")
return True
else:
logger.warning("Data integration associations still exist")
return False
except Exception as e:
logger.error(f"RequestId: {request_id} - Error during deletion process: {str(e)}")
logger.error(f"RequestId: {request_id} - Failed operation details - Data Integration: {data_integration_name}")
raise
def get_error_details(error):
"""Extract error type and operation from boto3 exception"""
error_message = str(error)
if hasattr(error, 'operation_name'):
return f"An error occurred ({error.__class__.__name__}) when calling the {error.operation_name} operation"
return error_message
def lambda_handler(event, context):
request_id = generate_request_id()
try:
# Get parameters
params = get_parameters(event, context)
# Initialize clients
clients = initialize_clients(params['region'])
# Check if parameters are available
if not params['instance_id'] or not params['data_integration_name']:
integration_names = list_data_integrations(clients['appintegrations'])
missing_params = []
if not params['instance_id']:
missing_params.append('instance_id')
if not params['data_integration_name']:
missing_params.append('data_integration_name')
logger.error(f"RequestId: {request_id} - Missing required parameters: {missing_params}")
return {
'statusCode': 400,
'body': json.dumps({
'error': 'Missing required parameters',
'message': 'Please provide all required parameters. Check logs for details.',
'requestId': request_id
}, separators=(',', ':')) # Remove extra whitespace
}
# Proceed with deletion
success = delete_data_integration(
clients,
params['data_integration_name'],
params['instance_id'],
request_id
)
return {
'statusCode': 200 if success else 400,
'body': json.dumps({
'success': success,
'message': f"Data integration: {params['data_integration_name']} deleted successfully" if success else f"Failed to delete data integration: {params['data_integration_name']}",
'requestId': request_id
}, separators=(',', ':'))
}
except Exception as e:
logger.error(f"RequestId: {request_id} - Lambda execution error: {str(e)}")
error_message = get_error_details(e)
return {
'statusCode': 500,
'body': json.dumps({
'error': 'Internal server error',
'message': f"{error_message}. Check logs for details.",
'requestId': request_id
}, separators=(',', ':'))
}
- The code handles the below
- Parameter validation
- AWS service client initialization
- Connect Data integration association cleanup
- QConnect knowledge base and assistant association removal
- Final data integration deletion
- Includes proper error handling and logging mechanisms to ensure reliable execution and troubleshooting capabilities.
- Execute the Function :
You can provide the input parameters in either of the two ways :
- Environment variables (as mentioned in step 2) or
- JSON event format:
{ "instance_id": "your-instance-id", "data_integration_name": "your-integration-name" }
Testing and Validation
Sample Test Cases
Basic deletion test event:
{ "instance_id": "your-instance-id", "data_integration_name": "test-integration" }
Expected Outputs
{ "statusCode": 200, "body": { "success": true, "message": "Data integration: test-integration deleted successfully", "requestId": "uuid-value" } }
Common Error Scenario
Missing Parameters:
{ "statusCode": 400, "body": { "error": "Missing required parameters", "message": "Please provide all required parameters. Check logs for details.", "requestId": "uuid-value" } }
Best Practices
Security
- Follow least privilege principle for IAM roles
- Encrypt sensitive environment variables
Performance
- Implement concurrent deletion where possible
- Use proper timeout settings
- Implement pagination for large resource lists
Limitations
- Knowledge bases: hard limit of 10
- Data integrations: soft limit of 10 (can be increased via Service Quotas console)
- Region availability depends on Amazon Q service availability
Troubleshooting
Common Issues
- Permission errors
- Solution: Verify IAM role permissions
- Resource not found
- Solution: Check resource existence and region
- Deletion failures
- Solution: Check cloudwatch logs and resource dependencies
Debug Procedures
- Enable DEBUG level logging if required.
- Check CloudWatch logs for detailed error messages.
- Verify AWS service health status.
- Confirm resource state before deletion.
Clean-up Steps
- Verify resource deletion using AWS CLI or AWS CloudShell
aws appintegrations list-data-integrations aws qconnect list-knowledge-bases
- Delete Lambda function if no longer needed.
- Review and clean up IAM roles.
Reference Documentation
- AWS Lambda Developer Guide : https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html
- Amazon Connect Service API Reference : https://docs.aws.amazon.com/connect/latest/APIReference/API_Operations_Amazon_Connect_Service.html
- Amazon AppIntegrations Service API Reference : https://docs.aws.amazon.com/connect/latest/APIReference/API_Operations_Amazon_AppIntegrations_Service.html
- Amazon Q Connect API Reference : https://docs.aws.amazon.com/connect/latest/APIReference/API_Operations_Amazon_Q_Connect.html
- AWS CLI Installation Guide : https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-install.html
- Amazon Connect Appintegrations CLI reference : https://awscli.amazonaws.com/v2/documentation/api/latest/reference/appintegrations/index.html
- Amazon Q in Connect CLI reference : https://awscli.amazonaws.com/v2/documentation/api/latest/reference/qconnect/index.html
- AWS CloudShell reference : https://docs.aws.amazon.com/cloudshell/latest/userguide/getting-started.html
Relevant content
- AWS OFFICIALUpdated 11 days ago
- asked 2 years ago
- asked 5 months ago