Why do S3 CLI uploads sometimes hang mid-transfer with no error?

When AWS CLI uploads to S3 hang mid-transfer without errors, several factors could be causing this issue:

1. Multipart upload interruptions: For large files (over 100MB), the AWS CLI automatically uses multipart uploads. If a part fails to upload but doesn't generate an error, the process might appear to hang. The CLI is designed to be synchronous, meaning commands like aws s3 cp should only complete after all files are transferred.

2. Network connectivity issues: Intermittent network problems can cause uploads to stall without generating errors.

3. Resource constraints: The AWS CLI S3 transfer commands are multithreaded by default, with 10 concurrent requests. If your system is resource-constrained, this could cause timeouts or stalled transfers.

4. Incomplete multipart uploads: If a multipart upload gets interrupted, it might leave incomplete upload parts in S3 without properly failing.

You could try these solutions:

• Adjust the max_concurrent_requests configuration value (decrease it if your system is overwhelmed, or increase it if you want faster transfers with more bandwidth)
• Use the --debug flag to get more detailed information about what's happening
• For very large files (over 5GB), ensure you're using multipart uploads correctly
• If an upload appears stuck, you might need to manually abort the multipart upload using the abort-multipart-upload command

The AWS CLI performs checksum calculations for uploads to ensure data integrity, which is another process that could potentially cause issues if there are problems with the file being uploaded.
Sources
Upload large files to S3 via CLI | AWS re:Post
Is AWS CLI really synchronious? | AWS re:Post
AWS CLI S3 FAQ — AWS CLI 2.27.32 Command Reference
Amazon S3 examples using AWS CLI - AWS SDK Code Examples
AWS CLI S3 Configuration — AWS CLI 2.27.32 Command Reference