Amazon Quicksight SPICE Data set partition

0

I have been searching for a documentation on how SPICE partitions data set but found none. How does the SPICE engine partition data set from S3 via a Manifest File?

AWS
Bims_D
asked 5 years ago1052 views
1 Answer
0
Accepted Answer

SPICE does not partition the data. It simply loads the data from all the files listed in the manifest into a single table. This is why there is a requirement that the data types for every field have to be the same in all files listed in the manifest.

You can read more about this here https://docs.aws.amazon.com/quicksight/latest/user/data-sets-based-on-multiple-s3-files.html

If you want to leverage partitions on S3, you can use Athena to query the data set on S3 before it gets imported into SPICE. Using Athena with QuickSight allows you to load a complete table from Athena or write a custom SQL query to load. With a custom SQL query, you can leverage your Athena partitions at query time, prior to having the result set loaded into SPICE.

https://docs.aws.amazon.com/quicksight/latest/user/create-a-data-set-athena.html

answered 5 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions