Is there any way that EMR developers could provide a roadmap for planned features in upcoming EMR releases?
For example: I am a data scientist / MLOps engineer responsible for coordinating my team's development cycles, and we are all very excited about the Pandas on Spark API provided in Spark 3.2.0+. It would be very helpful to know if the EMR developers plan to upgrade to that version in release EMR-6.6.0. Otherwise, we might have to find a workaround or another platform.
A vague idea of the EMR release schedule would also be helpful. I understand plans are flexible, but it seems like the EMR team cuts versions every 4-5 (ish) months based on previous releases?
After opening a ticket the support engineer had no advice other than to monitor the EMR latest releases webpage.