Need help with the Salesforce connector in glue, testing or creating always results in an internal error

I understand you're experiencing difficulties with the Salesforce connector in AWS Glue, specifically encountering an internal error during testing or creation of the connection. This can be frustrating, but there are several steps we can take to troubleshoot and potentially resolve the issue.

First, let's ensure all the basic configuration steps are correct:

1. Verify that the Salesforce instance URL you're using is correct and accessible.
2. Double-check that the IAM role associated with your AWS Glue job has the necessary permissions to read the secret in AWS Secrets Manager.
3. Confirm that your Salesforce account has API access and the ability to install connected apps.

Since you're using the Authorization Code OAuth grant type with the AWS managed client application, and you can see it's getting the grant and updating the secret values, it seems the initial OAuth flow is working correctly. However, the error suggests there's an issue with parsing the response from the connector application.

To get more detailed error information, you can try the following:

1. Check the CloudWatch logs associated with your AWS Glue connection test. These logs might provide more detailed error messages that could help identify the root cause.

2. Verify that your VPC and security group settings allow outbound access to Salesforce. Even though you're using the AWS managed client application, network connectivity issues could cause this type of error.

3. Try creating a new connection from scratch, ensuring all parameters are correctly entered. Sometimes, recreating the connection can resolve unexpected issues.

4. If possible, try using the JWT_TOKEN grant type instead of the Authorization Code. This requires creating a connected app in your Salesforce instance, but it might help isolate whether the issue is specific to the Authorization Code flow.

5. Check if there are any Salesforce API limits that you might be hitting. Although unlikely during a connection test, it's worth verifying.

6. Ensure that your Salesforce instance is not undergoing any maintenance or experiencing any known issues at the time of your connection attempt.

If the issue persists after trying these steps, it would be advisable to contact AWS Support. They can provide more specific guidance and potentially investigate the internal error you're encountering.

Remember, when working with AWS Glue connections, it's always a best practice to test the connection before using it in an ETL job. While this is causing an issue now, it's good that you're following this practice as it helps identify problems early in the process.
Sources
Configuring Salesforce connections - AWS Glue
Configuring Salesforce - AWS Glue
Testing an AWS Glue connection - AWS Glue
Troubleshooting connection issues in AWS Glue - AWS Glue