AWS announces AWS Interconnect – multicloud (preview), providing simple, resilient, high-speed private connections to other cloud service providers. AWS Interconnect - multicloud is easy to configure and provides high-speed, resilient connectivity with dedicated bandwidth, enabling customers to interconnect AWS networking services such as AWS Transit Gateway, AWS Cloud WAN, and Amazon VPC to other cloud service providers with ease.
How to connect Kinesis Data Streams with EMR in EMR Studio?
0
I need to load data from Kinesis Data Streams to EMR via EMR Studio. I Follow this sample but doesn't work: https://github.com/awslabs/spark-sql-kinesis-connector
- Topics
- Analytics
- Language
- English
asked 2 years ago1.6K views
1 Answer
- Newest
- Most votes
- Most comments
Relevant content
- asked a year ago
- asked 2 years ago
- asked a year ago
Follow the error: An error was encountered: [STREAM_FAILED] Query [id = 8e197d37-61c3-48de-b388-b5b7e4b0078d, runId = 7a4bf9bb-97af-495e-8f61-edd54ef7d8dc] terminated with exception: Unable to fetch committed metadata from previous batch id 0. Some data may have been missed Traceback (most recent call last): File "/mnt1/yarn/usercache/livy/appcache/application_1709526661976_0019/container_1709526661976_0019_01_000001/pyspark.zip/pyspark/sql/streaming/query.py", line 221, in awaitTermination return self._jsq.awaitTermination() File "/mnt1/yarn/usercache/livy/appcache/application_1709526661976_0019/container_1709526661976_0019_01_000001/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1322, in call return_value = get_return_value( File "/mnt1/yarn/usercache/livy/appcache/application_1709526661976_0019/container_1709526661976_0019_01_000001/pyspark.zip/pyspark/errors/exceptions/captured.py", line 185, in deco raise converted from None pyspark.errors.exceptions.captured.StreamingQueryException: [STREAM_FAILED] Query [id = 8e197d37-61c3-48de-b388-b5b7e4b0078d, runId = 7a4bf9bb-97af-495e-8f61-edd54ef7d8dc] terminated with exception: Unable to fetch committed metadata from previous batch id 0. Some data may have been missed
also I follow this example: https://github.com/aws-samples/spark-kinesis-connector-emr-sample