How can I implement 'login with aws' for a web application?

We are building a web application that asks users to login using their aws account and uses the auth token generated to access specific resources from the user's aws account.

This is similar to the support in Azure and GCP: https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app https://developers.google.com/identity/sign-in/web/sign-in

What is the correct way of implementing this for aws?

Investigation done so far:

• Aws Cognito - We explored aws cognito and it seems like it is a solution to manage our own user pool. Users will need to sign up and signed-in users can sign in. We are looking to access the aws user pool, instead of managing our own user pool.
• Aws STS - STS has APIs to generate temp credentials for an IAM role, but the STS SDK itself needs to be initialized using our aws credentials. So this scenario is not feasible for web applications.

We are looking for a way which uses the oAuth2 protocol to authenticate the user and return an access token to the web application.