EMR Serverless install packages Go Dependecies (bootstrap)

0

Hi, I need to install Go packages that interact with my Spark script. Is it possible to do such things ?

mgtdi
질문됨 일 년 전1118회 조회
2개 답변
0

If I'm reading this sample correctly it should work; if so you'll have to stuff those libraries into a custom package and source it from there.

profile picture
답변함 일 년 전
  • Thanks for your comment. Yes, The problem is not with PIP/Python packages. The problem is to use a Go app like SOPS. Which is used to decrypt secrets. In EMR we apply it using the bootstrap but in EMR Serverless there's no bootstrap available.

  • Interesting - are you shelling out to the Go binary from your Spark script? You can certainly include the binaries in the tarbell and it will get unpacked into an environment folder...I'm not sure if there's anything preventing you from executing other binaries, but you could give it a shot.

0

yes, maybe adding the Go binary app (SOPS) to the ZIP file might work. thanks

mgtdi
답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠