How to add merging steps after a ConditionStep to a Sagemaker Pipeline?

Question

We want to create a Sagemaker pipeline with two branches based on a ConditionStep.  After either branch, we want to execute multiple common steps.  However, Sagemaker doesn't allow us to create downstream steps depend on the ConditionStep.  Is there any workaround to this except for duplicating these common steps (E and F below)?
Here is the topology of our pipeline,
1. Execute step A followed by a ConditionStep B.
2. If condition B is true, then execute steps C, E and F.
2. If condition B is false, then execute steps D, E and F.

Answer

Hi there,

Thank you for reaching out to us.

Unfortunately this is something not supported yet, this is a sought after feature by a couple of our customers (see Feature Request [1.] below) and it has been added to our backlog.

A per your understanding at present the workaround would be to duplicate the complete DAG after the conditional step.

Thanks again for your enquiry.

[1.]  https://github.com/aws/sagemaker-python-sdk/issues/3565

How to add merging steps after a ConditionStep to a Sagemaker Pipeline?

Contenido relevante