application history shows empty

Problem resolved and I can use ssh port forwarding to see spark UI and jobs in yarn UI. Thanks.

Hi Leon,

Glad to hear that you addressed the issue.

Furthermore, as you are using EMR 5.29.0 you can consider exploring the One-Click Access to Persistent Spark History Server UI which doesn't require SSH tunneling.

https://aws.amazon.com/about-aws/whats-new/2019/11/amazon-emr-announces-one-click-access-to-persistent-spark-history-server/
https://docs.aws.amazon.com/emr/latest/ManagementGuide/app-history-spark-UI.html

However, if you have reconfigured the default Spark events directory, you won't be able to view the applications using one-click access and would have to use the SSH tunneling method for Spark UI.

I hope this helps.
Do post back on this thread, if you have any further questions or concerns.

Regards,
Almas