EMR Serverless install packages Go Dependecies (bootstrap)

0

Hi, I need to install Go packages that interact with my Spark script. Is it possible to do such things ?

mgtdi
質問済み 1年前1119ビュー
2回答
0

If I'm reading this sample correctly it should work; if so you'll have to stuff those libraries into a custom package and source it from there.

profile picture
回答済み 1年前
  • Thanks for your comment. Yes, The problem is not with PIP/Python packages. The problem is to use a Go app like SOPS. Which is used to decrypt secrets. In EMR we apply it using the bootstrap but in EMR Serverless there's no bootstrap available.

  • Interesting - are you shelling out to the Go binary from your Spark script? You can certainly include the binaries in the tarbell and it will get unpacked into an environment folder...I'm not sure if there's anything preventing you from executing other binaries, but you could give it a shot.

0

yes, maybe adding the Go binary app (SOPS) to the ZIP file might work. thanks

mgtdi
回答済み 1年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ