- 新しい順
- 投票が多い順
- コメントが多い順
Yes, there are a couple of ways to handle this issue:
Modify the Glue schema: You can modify the Glue schema to handle the NULL values explicitly. You can set the Null type to true for the columns that can have NULL values. This will ensure that the conversion to Parquet format does not fail when there are NULL values in the data.
Use a Lambda function: You can use a Lambda function in between Kinesis Firehose and Glue to modify the data before it is written to Glue. The Lambda function can check for NULL values and replace them with a default value or remove the entire record if necessary. This will ensure that only valid data is written to Glue and the conversion to Parquet format does not fail.
Use a data processing framework like Apache Spark: You can use a data processing framework like Apache Spark to handle NULL values in the data. Apache Spark provides a rich set of functions to handle NULL values and can convert data from JSON to Parquet format without any issues.
Overall, the best approach depends on your specific use case and requirements.
関連するコンテンツ
- 質問済み 9ヶ月前
AWS公式更新しました 3年前- AWS公式更新しました 1年前
- AWS公式更新しました 2年前
- AWS公式更新しました 3年前
Thank you.. Could you point me to some AWS documentation for 1 please: Modify the Glue schema: You can modify the Glue schema to handle the NULL values explicitly. You can set the Null type to true for the columns that can have NULL values. This will ensure that the conversion to Parquet format does not fail when there are NULL values in the data.