SFTP Connector says password is required using with AWS Transfer Family and AWS Glue

The issue you're facing is related to the configuration of the SFTP connector in AWS Glue. The SFTP connector expects both the username and password to be provided, even though the instructions state that they are optional.

To overcome this issue, you can try the following:

Use the AWS Glue SFTP Data Source connector instead of the SFTP connector from the Marketplace:
    The AWS Glue SFTP Data Source connector is natively available in AWS Glue and is designed to work with the AWS Transfer Family service.
    This connector does not require a password, as it uses the private key file for authentication.
    In your AWS Glue job, you can use the s3_target and sftp_source parameters to configure the SFTP connection.


Use the SFTP connector from the Marketplace and provide the password:
    If you still want to use the SFTP connector from the Marketplace, you'll need to provide the password, even though the documentation states it's optional.
    You can do this by setting the fs.sftp.password.<id>.server.transfer.eu-west-2.amazonaws.com.<user> property in your AWS Glue job.
    Replace <id>, <server>, and <user> with the appropriate values for your setup.

You can also checkout the SFTP connector offering from AWS Transfer Family - https://docs.aws.amazon.com/transfer/latest/userguide/creating-connectors.html

This connector can be used to copy files between any remote SFTP server (including AWS Transfer Family SFTP endpoints) and your Amazon S3 buckets. Once files are in S3, you can trigger further downstream processing on those files using Glue.