Glue 3.0 是否支持 Glue Spark shuffle manager？

0

【以下的问题经过翻译处理】 Glue 3.0 是否支持 Spark shuffle manager？我正在尝试在Glue 3.0 作业中设置 --write-shuffle-files-to-s3 标志，但它似乎不起作用，没有任何文件写入。

如果目前不支持——该特性最终会支持吗？

Topics

Analytics Database

Tags

AWS Glue Extract Transform & Load Data

Language

中文 (简体)

EXPERT

rePost Polyglot

asked 5 months ago27 views

1 Answer

Newest
Most votes
Most comments

Are these answers helpful? Upvote the correct answer to help the community benefit from your knowledge.

0

【以下的回答经过翻译处理】目前AWS Glue 3.0 版本已经支持AWS Glue Spark Shuffle Manager，可以参考如下链接：

Managing ETL jobs with AWS Glue Studio - https://docs.aws.amazon.com/glue/latest/ug/managing-jobs-chapter.html

EXPERT

rePost Polyglot

answered 5 months ago

Relevant content

怎样通过CloudFormation来给DocumentDB创建一些表和索引(how to create some table and index in DocumentDB by CloudFormation）
Accepted Answer
Dgk
asked 10 months ago
Sites on Amplify become very slow recently. Amplify站点最近突然变得很慢。
Zhu
asked 10 months ago
我通过Elastic Beanstalk创建环境之后，用code pipeline关联github中的springcloud项目deploy失败了，下面是具体报错原因
rePost-User-1988868
asked a year ago
sagemaker美国人口普查数据在中国宁夏区下载提示拒绝访问
LiuChuang
asked a year ago
How can I troubleshoot problems with viewing the Spark UI for AWS Glue ETL jobs?
AWS OFFICIALUpdated 3 years ago
How do I resolve the "No space left on device" error in an AWS Glue ETL job?
AWS OFFICIALUpdated a year ago
How do I resolve the "java.lang.OutOfMemoryError: Java heap space" error in an AWS Glue Spark job?
AWS OFFICIALUpdated 2 years ago
How can I use Apache Iceberg with a cross-account AWS Glue Data Catalog in Spark?
AWS OFFICIALUpdated a year ago
AWS Glue Dynamic Frame – JDBC Performance Tuning Configuration
EXPERT
AWSRB
published a year ago
AWS Glue Python Shell – Oracle DB Connection Solution
EXPERT
AWSRB
published a year ago