AWS Glue error when connecting AWS services

Hello Anushka, Thank you very much for your question. The error you are obtaining typically occurs when the AWS Glue job is unable to list the shards in the Amazon Kinesis Data Stream. This could be due to several reasons, such as insufficient permissions, incorrect stream name, or issues with the Kinesis Data Stream itself. In order to resolve the issue, here are some steps you can check:

• Check that you are in the right AWS region: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html
• Check IAM Permissions: Ensure that the IAM role associated with your AWS Glue job has the necessary permissions to access the Kinesis Data Stream. The role should have the kinesis:ListShards and kinesis:DescribeStream permissions for the specific Kinesis Data Stream. https://docs.aws.amazon.com/kinesis/latest/APIReference/API_ListShards.html
• Verify Kinesis Data Stream Name: Ensure that it matches the actual stream name in your AWS account. Kinesis Data Stream names are case-sensitive, so make sure you're using the correct case.

If these resources do not help troubleshoot your issue, make sure to check the error logs you are obtaining. You can do so by for example, checking the CloudWatch logs for your AWS Glue job. The logs may provide additional information about the error and help identify the root cause. If the issue persists after following these steps, you may need to contact AWS Support for further assistance.