Can a CloudFront distribution have a chain of failover origins?

I'm using CloudFormation to create a CloudFront distribution. CloudFront doesn't seem to let me set OriginId of an origin group member to an origin group ID, even though the docs seem to suggest that origin IDs and origin group IDs are interchangable. I'm trying to avoid using a Lambda just to achieve multiple failovers for a resource path.

This is the error I get back from CloudFormation:

Invalid request provided: The parameter Members has one or more origins [OriginGroupB] for origin group OriginGroupA that aren't valid. Origins that you specify for an origin group must be associated with the distribution.

How can I get my distribution to failover more than once?

In other words, if my secondary origin responds with 404, I want a third origin to be tried.