Redshift can query data from S3 by using Redshift Spectrum, but it's likely to hit the same issue when transferring the result to the client because of the network overhead.
It might be worth reconsidering if you really need such a large dataset in the frontend. Transferring a large dataset to the frontend may lead to lower user experience because of longer page response and more memory consumption in the client. Also, in most cases users do not need to visualize that much rows at once.
So, one option is to aggregate or extract the specific data which gives the insight to the user.
Another option is to introduce incremental rendering like some BI applications and websites do. The frontend will fetch and display minimal data at the first render. As a user scrolls the screen, the frontend will incrementally fetch and render the paginated result. This will minimize the initial data transfer and page response. If the user needs the full dataset, it can be implemented as a data export button.
- asked 9 months ago
- Accepted Answerasked 2 years ago
- asked 10 months ago
- AWS OFFICIALUpdated 6 months ago
- How can I resolve errors I receive while uploading data to or downloading data from Amazon Aurora to Amazon S3?AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago