Redshift not loading sorted data as sorted

Question

Following this doc:
https://docs.aws.amazon.com/redshift/latest/dg/vacuum-load-in-sort-key-order.html

It seems that we're loading things according to these guidelines but our data is still defaulting to loading as unsorted. If we're using a time based sort key and there is overlap in this way:

Existing sort keys in table:

1
2
3

New sort keys in COPY:

3
4
5

Would the overlap on the 3 value explain the behavior we're seeing, even if the data could technically be considered sorted?

Fabrizio@AWS · Answer

Yes, I think that is the issue ; the documentation explicitly mentions that new rows are higher:

*All the* **new rows are higher in sort order than the existing rows** *, including rows marked for deletion. In this instance, Amazon Redshift uses the first eight bytes of the sort key to determine sort order.*

hope this helps

Redshift not loading sorted data as sorted

Relevant content