Hi, Thanks for writing to re:Post.
As I understand you are facing an issue with getting logs into Cloudwatch.
I would like to inform you that, EMR Serverless integration with Cloudwatch is not available at the moment. They are currently under development and expected to be available in late Q3 2022. The best way to track the updates from customers side would be to keep an eye on the EMR Serverless documentation page .
Also, please note that given the volume of logs generated by Spark and Hive, Cloudwatch logs is not always cost-effective at that scale. Hence we have managed storage where EMR stores the logs for customer at no additional cost for 30 days. Customers can also choose to store logs in S3.
Thanks and stay safe!
How to publish Spark (EMR serverless) jobs logs to cloudwatchAccepted Answerasked 3 months ago
Spark with scala 2.12.x on EMRasked 4 years ago
emr serverless custom imageasked 2 months ago
log4j2 with AWS EMR and Sparkasked 3 years ago
EMR serverless spark properties fileasked 4 months ago
How do I access the Spark UI?asked 3 years ago
Run spark code serverlessAccepted AnswerMODERATORasked 2 years ago
How to optimize a batch of Spark Jobs on EMR to reduce overall processing time by 4-5x?Accepted Answerasked 2 years ago
Reading Aurora Postgress Table with Spark SQL on EMRAccepted Answerasked 4 years ago
Environment variables not found in Spark env with EMR 6.xasked 2 years ago