I have set up an EMR Serverless application to run a simple Hello World spark job. I've configured everything properly and the job execution is successful.
However, once I've updated the application to use the custom image and submitted the same Spark job it fails. No logs are available and no clue about the error cause.
If you're building on an M1 or M2 Mac, make sure you've added --platform=linux/amd64 to the FROM line in your Dockerfile or you've chosen arm64 architecture when you create your application.
If you're building on an M1 or M2 Mac, make sure you've added
--platform=linux/amd64
to theFROM
line in your Dockerfile or you've chosen arm64 architecture when you create your application.