Please investigate as a matter of urgency
With regard to the hadoop client classes, there is an incompatibility in the jars loaded from the /opt/flink/lib directory. To reproduce the bug, create a new notebook and attempt to run:
%flink
import org.apache.hadoop.fs.s3a.S3ARetryPolicy
import org.apache.hadoop.conf.Configuration
new S3ARetryPolicy(new Configuration())
You will see the following error message:
java.lang.NoSuchMethodError: 'long org.apache.hadoop.conf.Configuration.getTimeDuration(java.lang.String, java.lang.String, java.util.concurrent.TimeUnit)'
at org.apache.hadoop.fs.s3a.S3ARetryPolicy.<init>(S3ARetryPolicy.java:113)
... 80 elided
The classes have been resolved from:
| Class | Jar Location |
|---|
| S3ARetryPolicy | /opt/flink/lib/flink-s3-fs-hadoop-1.13.2.jar |
| Configuration | /opt/flink/lib/hadoop-common-2.6.5.jar |
