I'm running into an issue where I have certain properties, such as EMR secret resolution and package resolution that are not working when I define them as spark defaults in the job configuration file but are working properly when I define them as spark parameters. Additionally, it also seems inconsistent, especially secret resolution. I've had jobs where it was unable to look up a secret and failed but a clone of the job was able to process fine.
Has anybody else experienced similar issues?
AWS 官方已更新 2 年前