Is AWS Managed Microsoft AD (Hybrid Edition) compatible with Microsoft Entra ID Domain Services?

No, I do not believe this use case is support because of a limitation with Microsoft Entra ID Domain Services. Microsoft would have release their own Hybrid version of their service before this would be possible.

From Microsoft's Entra Domain Services FAQ:

Can I add domain controllers to a Microsoft Entra Domain Services managed domain?
No. The domain provided by Microsoft Entra Domain Services is a managed domain. You don't need to provision, configure, or otherwise manage domain controllers for this domain. These management activities are provided as a service by Microsoft. Therefore, you can't add more domain controllers (read-write or read-only) for the managed domain.

https://learn.microsoft.com/en-us/entra/identity/domain-services/faqs#can-i-add-domain-controllers-to-a-microsoft-entra-domain-services-managed-domain-

Based on the available information, AWS Managed Microsoft AD (Hybrid Edition) is designed to connect with self-managed Microsoft Active Directory environments, not specifically with Microsoft Entra ID Domain Services.

For integrating with Microsoft Entra ID (formerly Azure AD), AWS offers different solutions. AD Connector can be used with Entra ID Domain Services for certain use cases like SFTP transfers through AWS Transfer Family. However, AD Connector doesn't support RDS SQL Server as you noted.

For your specific requirements of authenticating to EC2 Windows instances and RDS SQL Server using Microsoft Entra ID credentials, you might need to consider a combination of services. AWS Managed Microsoft AD (Standard or Enterprise Edition) could be set up with appropriate trust relationships to your identity environment.

Another approach would be using AWS IAM Identity Center in conjunction with Microsoft Entra Connect and Active Directory Federation Services (AD FS). This setup allows synchronization of users from your directory to Microsoft Entra ID and enables federation for accessing AWS services using your existing credentials.

For your specific scenario involving Microsoft Entra ID Domain Services rather than traditional on-premises AD, you should contact AWS Support for the most current compatibility information, as the documentation doesn't explicitly confirm compatibility between AWS Managed Microsoft AD (Hybrid Edition) and Microsoft Entra ID Domain Services.
Sources
AWS integrate with AD\AAD | AWS re:Post
Using AWS Directory Service for Entra ID Domain Services - AWS Transfer Family
Understanding AWS Managed Microsoft AD (Hybrid Edition) - AWS Directory Service

Entra ID is not the former Azure AD DS. Entra ID is the former Azure AD. The naming convention is quite confusing here.

The short answer is no. Entra ID (formerly Azure AD) is not a fully features Active Directory, it is more like a user pool for authenticating web applications using SAML. You can sync users from AWS Managed AD (including Hybrid) to an Entra ID tenant by using the Microsoft Entra Connect Sync but you cannot write-back passwords to AWS Managed AD. So you can only partially achieve you goal.

A different service is Entra Domain Services (formerly called Azure AD DS which stands for Azure Active Directory Domain Services). This is a fully featured Active Directory running on Azure cloud. Assuming there is an Entra Domain Services and network connectivity to an AWS Managed AD (for example over a VPN) you may be able to setup a trust relationship between both and then achieve your goal.