I would say that given your explanation, the way to do this is to have a EC2 instance running SSM and create a tunnel that will point localhost:3306 to your container|host:3306. If you are on ECS you could look at using execute-command but that'd be mostly to execute admin tasks remotely.
You could also consider using a web based admin tool, i.e. phpMyAdmin or stuff like that so that you don't require tunneling.
But for the long run, I'd recommend to not rely on that and not to run DB containers unless you really know what you're doing.
Instead, use RDS / RDS Aurora. If costs is an issue, consider Aurora Serverless. I have an instance running for 1y+ and never paid for it (it's there for API integrations, not for actual DB work). If need be you can still tunnel to it or use other ways to manage it.
Can I SSH into my container running on AppRunner?asked 4 months ago
Segmentation fault when running wine on AWS Lambda containerasked 3 months ago
Same container image, behaving differently on separate EKS clustersasked 3 years ago
Deploying a Machine Learning Project with django and laravel as a backend.asked 3 months ago
Accessing files and CLI inside of container deployment in lightsailAccepted Answerasked 3 months ago
Process hangs when running on batchasked 2 years ago
Recommended IAM structure to link a machine user to a policy? Running on-prem.asked 2 months ago
Accessing Running Container on AWS to my local machineasked a month ago
How to clone a Bitbucket repo from a Springboot microservice running on AWS EKS?asked 5 months ago
Migrating Multi-container Docker running on Amazon Linux to ECS on Amazon Linux 2asked 4 months ago