AWS re:Post을(를) 사용하면 다음에 동의하게 됩니다. AWS re:Post 이용 약관
AWS re:Postre:Post

Glue Jupyter Job , Execution Class

0

How can one set up an Execution Class = FLEX on a Jupyter Job Run , im using the %magic on my %%configure cell like below and also setting the input arguments with --execution_class = FLEX But still the jobs are quicking as STANDARD

%%configure { "region": "us-east-1", "idle_timeout": "480", "glue_version": "3.0", "number_of_workers": 10, "execution_class": "FLEX", "worker_type": "G.1X" }

Enter image description here

주제
분석데이터베이스
태그
분석AWS Glue데이터 추출, 변환 및 로드
언어
English
Jorge Vidinha
질문됨 일 년 전560회 조회
2개 답변
1

Sorry,me too...

Although it is a rather peculiar means, I was able to change the execution class to FLEX as follows.

  1. Push Glue to CodeCommit.
  2. On CodeCommit, change the value of "executionClass" to "FLEX" in [Job name].json Enter image description here
  3. Pull from CodeCommit.

With the above operation, the setting of FLEX has been changed. Enter image description here Enter image description here

profile picture
ARBR
답변함 일 년 전
  • Jorge Vidinha
    일 년 전

    Thanks ARBR , Im not using Codecommit at this point so really searching for a solution from console jupyter job runs

  • ARBR
    일 년 전

    By comparing before and after, we found the following:

    Under "jobConfig", the following two settings were added.

    • "flexExecution": true
    • "minFlexWorkers": null

    Adding these to the file downloaded with [Action]-[Download] and uploading with [Action]-[Upload] enabled "Flex".

    However, the statement below did not work.

    %%configure
{
    "flex_execution" : "true",
    "min_flex_workers" : "null"
}
1

I haven't tried it yet, but the URL below seems to be helpful.

https://aws.amazon.com/jp/blogs/big-data/introducing-aws-glue-flex-jobs-cost-savings-on-etl-workloads/

According to the information on the site, the name of the parameter seems to be "execution-class" (instead of "execution_class").

profile picture
ARBR
답변함 일 년 전
  • Jorge Vidinha
    일 년 전

    Thanks for the help ...

    On interactive Jupyter Jobs I have set up the the execution class to FLEX tried the 2 variants below: A "execution_class": "FLEX" B "execution-class": "FLEX" And also added execution-class = FLEX to JOB Run with job parameters at no success, it still shows Execution Class Standard .

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠