1回答
- 新しい順
- 投票が多い順
- コメントが多い順
3
Hello,
Seems like you are getting UnsupportedOperationException when reading the parquet data. There might be two cases as far as I aware. Either the underlying parquet file/files might be corrupted or the schema/datatype reference interpreted incorrectly. If you have partitioned data in s3 data source, try reading different data and see if you are getting the same issue when specifically reading particular partitioned data. If the files are not corrupted on the other hand, check if any column is in different type for an example, it may also throw this kind of exception. Refer this Jira - https://issues.apache.org/jira/browse/SPARK-24828
関連するコンテンツ
AWS公式更新しました 2年前- AWS公式更新しました 2年前
Thank you!!. I got the issue when querying particular partition, not sure though but I recreated that partition and the issue is resolved.