Instead of using the CLI to submit your job, have you tried using the boto3 Python library? https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr-serverless.html . All of the configuration parameters you've shared can be passed in EMRServerless boto3.
You can use below method to submit job for EMR serverless.
=>Running jobs from the EMR Studio console
=>Running jobs from the AWS CLI
Submittion of EMR serverless jobs from SageMaker notebook is not supported yet.
How to publish Spark (EMR serverless) jobs logs to cloudwatchAccepted Answerasked 3 months ago
Distributed processing in AWSasked 9 months ago
emr serverless custom imageasked 2 months ago
Do I have to redownload dataset to training job every time I run a Sagemaker Estimator training job?asked a year ago
Data sharing between EMR Serverless Jobsasked a month ago
EMR on EKS local filesasked 4 months ago
EMR serverless spark properties fileasked 4 months ago
Submit EMR serverless jobs from SageMaker notebookasked 2 months ago
Is it possible to run a JDK11 application on EMR serverless?Accepted Answerasked 5 months ago
Cost related Tagging for EMR Serverlessasked 7 months ago