AWS announces AWS Interconnect – multicloud (preview), providing simple, resilient, high-speed private connections to other cloud service providers. AWS Interconnect - multicloud is easy to configure and provides high-speed, resilient connectivity with dedicated bandwidth, enabling customers to interconnect AWS networking services such as AWS Transit Gateway, AWS Cloud WAN, and Amazon VPC to other cloud service providers with ease.
Real Time Data Ingestion
IHAC working on enabling real time Data ingestion on their S3 based Datalake with the objective of showing near-realtime data on QuickSight Dashboards. As of now, they are heavily relying on QuickSight SPICE engine for better Dashboard performance since direct Athena queries from QuickSight are not so performant. With Realtime data flowing into the Datalake, they need to refresh QuickSight SPICE datasets very frequently. However, they are hitting a limitation of 32 calls per 24 hours on CreateIngestion API to refresh Quicksight SPICE dataset. This is a serious blocker for them. Could you please suggest how can they overcome this limitation and move forward.. An error occurred (LimitExceededException) when calling the CreateIngestion operation (reached max retries: 2): Rate exceeded for CreateIngestion on this dataset. Please retry after 1304 minutes
- Language
- English
- Newest
- Most votes
- Most comments
Hi,
Since you are allowed to 32 full refreshes per day, you are probably using the Enterprise edition of QuickSight.
In that case, you can also use the incremental refresh (i.e. up to every 15 min) in order to refresh your SPICE data. Did you consider it?
Best,
Didier
Relevant content
- AWS OFFICIALUpdated a year ago
Thanks for the recommendation Didier. Cx is looking for near real time data refresh. Incremental refresh may run with out limit exceptions but there is a 15 minute delay which is long time for this requirement.