By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Glue job error - Error while List shards

0

Getting error while connecting streaming data from kinesis to redshift with few transformations using visual ETL. (using amazon kinesis - glue data catalog table as source ). Schema is already defined prior, hence crawler is not used. Error Category: UNCLASSIFIED_ERROR; StreamingQueryException: Query [id = 30dc3a82xxx, runId = ba421exxxx] terminated with exception: Error while List shards

Topics
AnalyticsDatabaseInternet of Things (IoT)
Tags
AWS GlueExtract Transform & Load DataAmazon RedshiftAmazon Kinesis
Language
English
Vanishree
asked 13 days ago137 views
1 Answer
1
Accepted Answer

Check the driver error logs, that could be a permission issue, that the stream is in a different region or similar

profile pictureAWS
EXPERT
Gonzalo Herreros
answered 13 days ago
profile picture
EXPERT
Oleksii Bebych
reviewed 13 days ago
  • Vanishree
    13 days ago

    Checked region and permissions which seems correct. Tried another method to directly stream into redshift using external schema and materialized view , even then getting error - error: Unable to describe stream code: 42001 ,Timeout was reached Stream name: name query: -1[child_sequence:1] location: kinesis_client.

  • Gonzalo Herreros EXPERT
    13 days ago

    Timeout normally means you are running in a VPC without internet or a kinesis endpoint (could also be the security group but is less likely)

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content