使用 AWS re:Post 即表示您同意 AWS re:Post 使用條款
AWS re:Postre:Post

Glue Jupyter Job , Execution Class

0

How can one set up an Execution Class = FLEX on a Jupyter Job Run , im using the %magic on my %%configure cell like below and also setting the input arguments with --execution_class = FLEX But still the jobs are quicking as STANDARD

%%configure { "region": "us-east-1", "idle_timeout": "480", "glue_version": "3.0", "number_of_workers": 10, "execution_class": "FLEX", "worker_type": "G.1X" }

Enter image description here

主題
分析資料庫
標籤
分析AWS Glue提取轉換與載入資料
語言
English
Jorge Vidinha
已提問 1 年前檢視次數 560 次
2 個答案
1

Sorry,me too...

Although it is a rather peculiar means, I was able to change the execution class to FLEX as follows.

  1. Push Glue to CodeCommit.
  2. On CodeCommit, change the value of "executionClass" to "FLEX" in [Job name].json Enter image description here
  3. Pull from CodeCommit.

With the above operation, the setting of FLEX has been changed. Enter image description here Enter image description here

profile picture
ARBR
已回答 1 年前
  • Jorge Vidinha
    1 年前

    Thanks ARBR , Im not using Codecommit at this point so really searching for a solution from console jupyter job runs

  • ARBR
    1 年前

    By comparing before and after, we found the following:

    Under "jobConfig", the following two settings were added.

    • "flexExecution": true
    • "minFlexWorkers": null

    Adding these to the file downloaded with [Action]-[Download] and uploading with [Action]-[Upload] enabled "Flex".

    However, the statement below did not work.

    %%configure
{
    "flex_execution" : "true",
    "min_flex_workers" : "null"
}
1

I haven't tried it yet, but the URL below seems to be helpful.

https://aws.amazon.com/jp/blogs/big-data/introducing-aws-glue-flex-jobs-cost-savings-on-etl-workloads/

According to the information on the site, the name of the parameter seems to be "execution-class" (instead of "execution_class").

profile picture
ARBR
已回答 1 年前
  • Jorge Vidinha
    1 年前

    Thanks for the help ...

    On interactive Jupyter Jobs I have set up the the execution class to FLEX tried the 2 variants below: A "execution_class": "FLEX" B "execution-class": "FLEX" And also added execution-class = FLEX to JOB Run with job parameters at no success, it still shows Execution Class Standard .

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋，同時有助於提問者的專業成長。

回答問題指南

相關內容