How does Aurora reconstruct VCL from a 3/6 read quorum after writer failure?

Question

Hello,
I’m reading the SIGMOD 2018 paper Amazon Aurora: On Avoiding Distributed Consensus for I/Os, Commits, and Membership Changes, particularly Sections 2.3 and 2.4.
My understanding is:
- Each protection group contains six storage copies.
- A storage node advances its SCL only through a gap-free segment log chain.
- During normal operation, the database instance advances PGCL after observing four SCLs covering the relevant writes.
- VCL is the volume-wide point through which all preceding writes have met quorum.
- PGCL and VCL are transient state maintained by the database instance.
- After a writer failure, Aurora contacts at least a 3/6 read quorum for each protection group, reconstructs PGCL/VCL from storage-node state, and truncates the ragged edge beyond the reconstructed consistency point.
The quorum-intersection argument clearly guarantees that any write previously completed on four nodes appears on at least one member of every three-node read quorum.
What I don’t understand is how recovery handles a structurally complete write that had not reached write quorum before the writer failed.
For example:

```
Storage nodes: A B C D E F

Previous durable history: complete through LSN 104

LSN 105:
- Has valid backlinks
- Completes an MTR/consistency boundary
- Reaches only node A before the writer fails
- Never receives four SCL acknowledgements
- Is never acknowledged as committed to the client

Recovery quorum: {A, D, F}
````

Recovery discovers LSN 105 on A. From those three responses alone, its presence appears indistinguishable from a record that previously reached four nodes but now survives on only one member of the recovery quorum.
My questions are:
1. Can reconstructed VCL/VDL include LSN 105 and then repair/write it to additional storage nodes?
2. Or must LSN 105 be truncated because it never achieved the original 4/6 write quorum?
3. If recovery can distinguish these cases, what durable metadata proves that four SCLs previously covered the record, given that PGCL/VCL were transient writer state?
4. Does recovery normally query all available storage copies, with 3/6 representing only the minimum availability requirement rather than exactly three selected copies?
5. Is the outcome intentionally unspecified because a transaction that was not acknowledged to the client is allowed either to survive or be rolled back?
I’m mainly trying to distinguish:
- Recovering every previously acknowledged commit, which quorum intersection clearly guarantees.
- Determining the outcome of a complete but non-quorum, unacknowledged log tail.
Thank you for any clarification or references to a more detailed recovery description.

re:Post Agent · Answer

Your analysis demonstrates a sophisticated understanding of Aurora's quorum-based replication. Let me address your questions about recovery and VCL reconstruction.

The key insight is that Aurora's recovery process is designed to be conservative and deterministic. When reconstructing VCL after a writer failure, Aurora must establish a consistent point that reflects only writes that previously achieved write quorum (4 of 6 acknowledgements).

For your specific scenario with LSN 105:

LSN 105 must be truncated during recovery. Even though it appears structurally complete on node A, it never achieved the required 4/6 write quorum before the writer failed. The recovery process cannot and will not promote it to VCL or repair it to additional nodes.

Here's why: Aurora's recovery algorithm reconstructs VCL by finding the highest LSN that appears on at least a read quorum (3 of 6) of storage nodes for each protection group. However, this is a necessary but not sufficient condition. The quorum intersection property guarantees that any write that previously completed on four nodes will appear on at least one member of every possible three-node read quorum. The converse, however, is not true—finding a record on one node in a read quorum does not prove it previously reached four nodes.

Aurora resolves this ambiguity conservatively: VCL is reconstructed as the highest point where the recovery process can verify, through the pattern of responses across the read quorum, that write quorum was previously achieved. Records that appear on fewer nodes than would be expected from a completed write quorum are treated as part of the "ragged edge" and truncated.

Regarding your question about querying all available storage copies: In practice, Aurora's recovery process contacts all reachable storage nodes, not just exactly three. The 3/6 read quorum represents the minimum availability requirement—recovery can proceed as long as at least three nodes per protection group are reachable. Querying all available nodes provides more information for accurately reconstructing VCL and minimizes unnecessary truncation.

The outcome is not intentionally unspecified. Aurora's design principle is clear: only writes that achieved write quorum and were acknowledged to the client are guaranteed durable. Unacknowledged writes that didn't reach quorum may be lost during recovery. This is consistent with standard database semantics—a transaction that hasn't been acknowledged as committed to the client has no durability guarantee.

The recovery process effectively distinguishes these cases through the distribution pattern of log records across storage nodes. A write that previously achieved quorum will have a characteristic footprint across the storage fleet that differs from a write that only reached one or two nodes before failure.

Your understanding of the quorum intersection guarantee is correct: it ensures that every previously acknowledged commit (which by definition achieved write quorum) will be discovered during recovery and included in the reconstructed VCL. The conservative truncation of the ragged edge ensures consistency while preserving all durable, acknowledged transactions.  
**Sources**  
[Amazon Aurora reliability - Amazon Aurora](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Overview.Reliability.html)  
[Resilience in Amazon Aurora DSQL - Amazon Aurora DSQL](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/disaster-recovery-resiliency.html)

How does Aurora reconstruct VCL from a 3/6 read quorum after writer failure?

Add your answer

Relevant content