Thank you for writing on re:Post
Kindly know that “FileNotFoundError” error could be caused due to the following reasons:
- The path of .SQL files in Nexus repository is not correct.
- It could also be due to a python module missing that the function is trying to connect.
Also, if you are using Boto3, Python SDK, please note that the "execute_statement" API, which helps to run SQL statement, is not yet supported with Boto3 for Redshift-Serverless. So, we would recommend connecting to Redshift Serverless cluster, from Lambda, using redshift-connector.
The Amazon Redshift Connector module in python can help in connecting to Redshift-Serverless cluster and it also easily integrates with pandas and numpy. The connector supports numerous Amazon Redshift specific features that helps you get the most out of your data.
Kindly find below the steps to make the connection to cluster using Redshift Connector:
- In order to connect to Redshift Serverless, from a Lambda Function, you would first require to add the Redshift connector and the AWSDataWrangler-Python39 layers for the AWS Lambda Function as the Redshift Connector module is not an in-built module.
a) Redshift Connector : This has to be downloaded in the form of a zip file and added as a layer. You can download the zip file from the following GitHub page: https://github.com/aws/amazon-redshift-python-driver
b) AWSDataWrangler-Python39 which is an AWS layer (a layer from a list of layers provided by AWS). Data Wrangler is built on top of other open-source projects like pandas, apache, arrow and boto3, it offers abstracted functions to execute usual ETL tasks like load/unload data from Data Lakes, Data Warehouse and Databases.
- Below is a sample code to connect to Redshift Serverless using a python script in AWS Lambda Service.
def lambda_handler(event, context):
conn = redshift_connector.connect ( host='HOST_NAME', database='DATABASE_NAME', user='DATABASE_USER', SecretArn: '', port=5439 )
cursor.execute("YOUR SQL STATEMENT")
result: tuple = cursor.fetchall()
Note: HOST_NAME in the format : workgroup-name.account-ID.region.redshift-serverless.amazonaws.com . Also, please note that this sample code has been shared on a best-effort basis and we would request that you test the code thoroughly before deploying to your production environment, if you decide to use this code snippet.
Kindly try the above suggestions. Hope they will improve your experience. However, if the issue persists and you would like us to investigate the issue further then please create a technical support case with Amazon RedShift team or AWS Lambda team.
In order to open a Technical Support Case, you will need to have a Developer, Business or Enterprise Support Plan. If you do not have a Support Plan in place, please kindly visit the link provided below on how to sign up for one: https://aws.amazon.com/premiumsupport/knowledge-center/sign-up-support/
Redshift data APIasked 7 days ago
Redshift serverless data api sql callsasked 15 days ago
Does the Redshift Data API support Redshift serverless?
Best AWS tool to move/transform data from redshift to an APIasked 7 months ago
Using redshift-data boto3 to make cross account redshift callsasked 10 months ago
Redshift Serverless query from Data API
RedShift Serverless timeout when connecting with pythonasked 3 months ago
Redshift serverless: error while trying to create an external tableasked 5 months ago
How to truncate a Redshift-serverless table by Temporary credentials?Accepted Answerasked 2 months ago
Using Redshift Serverless for DatasharingAccepted Answerasked a month ago