1 Risposta
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
3
Hello,
Seems like you are getting UnsupportedOperationException when reading the parquet data. There might be two cases as far as I aware. Either the underlying parquet file/files might be corrupted or the schema/datatype reference interpreted incorrectly. If you have partitioned data in s3 data source, try reading different data and see if you are getting the same issue when specifically reading particular partitioned data. If the files are not corrupted on the other hand, check if any column is in different type for an example, it may also throw this kind of exception. Refer this Jira - https://issues.apache.org/jira/browse/SPARK-24828
Contenuto pertinente
- Come posso risolvere gli errori di connessione di Marketplace AWS nei miei processi ETL in AWS Glue?AWS UFFICIALEAggiornata 3 mesi fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
Thank you!!. I got the issue when querying particular partition, not sure though but I recreated that partition and the issue is resolved.