S3 Transfer Rates Capped at 2.8mb/s ??? How Can I Speed This Up?

S3 can send and receive your data at practically any speed you can send or receive it at, as long as you're using multiple parallel connections to different servers in S3's frontend server fleet. Which tool are you using? Are you using the AWS CLI, with commands like aws s3 cp or aws s3 sync executed on the command line? Or are you uploading the files via the graphical management console in your browser?

If you're using the CLI commands, they'll handle the parallelisation and all other complexities transparently. Try running the following commands to set the CLI to use 16 parallel threads to upload and to maintain a queue of up to 10,000 files/segments, ensuring that the next upload can begin as soon as an earlier one has completed:

aws configure set default.s3.max_concurrent_requests 16
aws configure set default.s3.max_queue_size 10000
aws configure set default.s3.multipart_threshold 64MB
aws configure set default.s3.multipart_chunksize 64MB
aws configure set default.s3.payload_signing_enabled true

If the speed still isn't adequate and your machine isn't overwhelmed, try increasing max_concurrent_requests to 64, for example, and see if there's a difference. If performance still remains unchanged, the bottleneck is most likely your internet connection's upstream bandwidth. If throughput does notably increase but your CPU starts to get overwhelmed, reduce the max_concurrent_requests setting to a lower value where the CPU doesn't stall. You can also disable payload_signing_enabled setting to avoid the overhead from calculating a SHA-256 hash of the data blocks being uploaded.

The slow transfer rates you're experiencing are not typical for Amazon S3, and there are several ways to improve your upload speeds. S3 is designed to handle high-volume data transfers efficiently, and it's not AWS's practice to artificially throttle speeds to upsell services.

Here are some suggestions to speed up your transfers:

1. Use Amazon S3 Transfer Acceleration: This feature is specifically designed to optimize long-distance transfers between clients and S3 buckets. It uses AWS edge locations to route data more efficiently, which can significantly improve upload speeds, especially for large files and long-distance transfers.

2. Implement multipart uploads: For large files, using multipart uploads can dramatically increase transfer speeds. This method allows you to upload parts of a file in parallel, which can better utilize your available bandwidth.

3. Optimize your network connection: Ensure you're using a fast, stable internet connection. Wired connections are generally more reliable than wireless for large data transfers.

4. Consider your geographic location: If you're far from the S3 region you're uploading to, you might experience slower speeds. Using S3 Transfer Acceleration or choosing a closer region can help mitigate this.

5. Use AWS CLI or SDK: These tools are optimized for AWS services and can offer better performance than third-party tools or web interfaces.

6. Check for client-side limitations: Ensure your client machine isn't the bottleneck. Factors like CPU usage, disk I/O, or other running processes could limit your upload speed.

7. Parallel uploads: If you're transferring many small files, consider uploading multiple files simultaneously to increase overall throughput.

To determine the potential benefit of using S3 Transfer Acceleration for your specific use case, you can use the Amazon S3 Transfer Acceleration Speed Comparison tool. This will help you compare accelerated and non-accelerated upload speeds across different S3 regions.

Remember, S3 is capable of handling very high transfer rates, often reaching hundreds of megabits per second or even multiple gigabits per second in optimal conditions. If you're still experiencing issues after trying these solutions, it would be advisable to contact AWS support for further assistance.

Sources
Amazon Simple Storage Service (S3) — Cloud Storage — AWS
Using Amazon S3 Transfer Acceleration to Accelerate Geographically Disparate Data Transfers - Best Practices Design Patterns: Optimizing Amazon S3 Performance
Use Amazon S3 Transfer Acceleration to Minimize Latency Caused by Distance - Best Practices Design Patterns: Optimizing Amazon S3 Performance

Where are you uploading from? If it’s from home over WiFi your limited to the slowest link.

Bear in mind your network connectivity too.