Can't Connect Elastic Beanstalk to RDS with Flask API

I created a basic Flask API and was able to connect it to AWS RDS. Just as a test to make sure it was working, I also made a basic form and was able to send a POST request and update the database.

I start running into issues when I deploy my application using Elastic Beanstalk. I don't want to do anything fancy. I just need a basic configuration and want to drop a zip file into the "Upload and Deploy" feature and let it run. I set the environment's configuration variables (includes database credentials) and still kept running into issues.

The main error I keep seeing in the logs involves mysqlclient. Specifically:

OSError: mysql_config not found

I seem to have all the required files needed for deployment (Procfile, requirements.txt). I'm not sure if I need mysqlclient at all. Do I need to set mysql_config in another location?