AWS Glue Studio Data Preview Fails instantly (Py4JJavaError)

0

Hi, I'm using AWS Glue Studio and once I click "data preview" it fails with the following error. The flow consists of 2 actions - PosgtreSQL JDBC Data source and Select Field action. The error is thrown instantly once the "Data Preview" button is clicked. The overall flow run successfully if i click "RUN" button - there are no errors and I get the outcome if I add the Target Source to dump the results back to PosgtreSQL table. It's just a Data Preview functionality that fails. Any idea what could be wrong and how to troubleshoot it?

Py4JJavaError: An error occurred while calling o538.getSampleDynamicFrame. : java.lang.UnsupportedOperationException: empty.reduceLeft at scala.collection.TraversableOnce$class.reduceLeft(TraversableOnce.scala:180) at scala.collection.AbstractTraversable.reduceLeft(Traversable.scala:104) at scala.collection.TraversableOnce$class.reduce(TraversableOnce.scala:208) at scala.collection.AbstractTraversable.reduce(Traversable.scala:104) at com.amazonaws.services.glue.SparkSQLDataSource.com$amazonaws$services$glue$SparkSQLDataSource$$getPaths(DataSource.scala:724) at com.amazonaws.services.glue.SparkSQLDataSource$$anonfun$getSampleDynamicFrame$7.apply(DataSource.scala:799) at com.amazonaws.services.glue.SparkSQLDataSource$$anonfun$getSampleDynamicFrame$7.apply(DataSource.scala:793) at com.amazonaws.services.glue.util.FileSchemeWrapper$$anonfun$executeWithQualifiedScheme$1.apply(FileSchemeWrapper.scala:89) at com.amazonaws.services.glue.util.FileSchemeWrapper$$anonfun$executeWithQualifiedScheme$1.apply(FileSchemeWrapper.scala:89) at com.amazonaws.services.glue.util.FileSchemeWrapper.executeWith(FileSchemeWrapper.scala:82) at com.amazonaws.services.glue.util.FileSchemeWrapper.executeWithQualifiedScheme(FileSchemeWrapper.scala:89) at com.amazonaws.services.glue.SparkSQLDataSource.getSampleDynamicFrame(DataSource.scala:792) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748)

Michael
asked 2 years ago703 views
1 Answer
1

It could be due to the IAM role permission that you have granted for the glue job. Check if the job has relevant permissions needed.

AWS
snathan
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions