Why Might Cross-Availability Zone (AZ) charges might apply??

When you provisioned the RDS database did you select the option to Connect to an EC2 compute resource Set up a connection to an EC2 compute resource for this database https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/option2-task2-create-rds-database.html

If you selected this at the start then the rest of the wizard would have greyed-out a lot of options around VPC, DB subnet group, and Availability Zone and more-or-less forced the database to be created in the same AZ as your existing EC2 instance.

If this wasn't selected at the start then the database will have been provisioned randomly into a subnet within the default DB subnet group, which basically comprises a subnet in each AZ for the VPC.

If the chosen subnet is in a different AZ to the subnet your EC2 is in, then if you try and make a connection between the two later, any data transferred will be charged.

Take a look at the section Workload components in same AWS Region of https://aws.amazon.com/blogs/architecture/overview-of-data-transfer-costs-for-common-architectures/ for a fuller explanation of what's in-scope of being charged, and what isn't, in various scenarios.

@Riku's suggestion of enabling multi-AZ, failing over to a subnet in the desired AZ, and the shrinking the config back to single-AZ would work and would be a good learning exercise to undertake, but if you are trying to stay within free tier it would be outside the scope of that. If this is just a freshly-created empty database with nothing of value in it then consider deleting and recreating in a subnet within the appropriate AZ.