AWS Glue 2.0/3.0 are not available in Dev endpoints

0

I am trying to create a Dev endpoint in AWS Glue 2.0 or higher and use a Notebook to attach to this env. However, the only 3 options I see are for Glue <= 1.0. Please note, the higher versions are indeed available in Glue Jobs. Any idea why this is the case and how do I resolve this ?

posta 2 anni fa482 visualizzazioni
2 Risposte
1
Risposta accettata

Dev Endpoints only support Glue 1.0, they will not be upgraded to support Glue 2.0 or later.

For interactive development with Glue 2.0 or later , please review and test either Glue studio notebooks or Glue Interactive Sessions.

Please, note these feature are in public preview (available in selected regions us-east-1,us-west-2,eu-west-1,ap-northeast-1) , so the full feature set will be available at General Availability , and there still might be some rough edges.

For more information on these new feature look at this documentation section.

Specific info on Interactive Sessions are here; while details on Glue studio notebooks are here.

AWS
ESPERTO
con risposta 2 anni fa
  • Thanks much for the response! Can you please let me know what is 'GA' & how do I select either of the two mechanisms ? I don't see the Glue Studio Notebook available to be used, I suppose it needs to be activated somehow. Any pointers regarding this is very much appreciated.

  • @AWS-User-1825611 see the edited answer, preview is limited to few regions.

  • Thanks much for your help!

  • Hello. For clarification, it sounds like you're saying that AWS will never update the Glue version supported for Glue development endpoints. When Glue 1.0 gets deprecated does that mean that Glue development endpoint will go away as well? We should switch to Glue Studio for interactive development going forward?

0

From the docs:

Note Development endpoints are not supported for use with AWS Glue version 2.0 jobs. For more information, see Running Spark ETL Jobs with Reduced Startup Times.

profile pictureAWS
ESPERTO
Roi
con risposta 2 anni fa
  • Please, add a link to the documentation section you are referring to enhance the user experience.

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande