How to keep TaskDefinitions in sync with the CloudFormation template that created them when CodeDeploy is used?

I was able to set up a full ECS Service stack with Blue/Green deployments using CloudFormation. The stack includes the load balancer (with listeners and target groups), an ECS Cluster, an ECS Service, a Task Definition, a Deployment Group, and CodeDeploy Application. Everything works great, but now that I have my TaskDefinition, I can't update it via CloudFormation without getting the following error:

Unable to update task definition on services with a CODE_DEPLOY deployment controller.  Use AWS CodeDeploy to trigger a new deployment.

That's straight-forward enough, I can update the TaskDefinition manually with a new revision that contains the changes I need (I simply added an environment file to a container definition) and do a new deploy using the new revision. However, now my CloudFormation template is out of sync with my resources, and I am unable to use it to make further modifications. Even if I make the same changes to the template that I made manually to the TaskDefinition, CloudFormation is still confused and thinks it needs to do something.

One workaround I tried was editing my template to completely remove the TaskDefinition, ECS Service, and Deployment Group. Run that as an update, and then put them all back in (with the modified version of the TaskDefinition) and run a second update. That seemed to work ok, but now I am getting false positives on my Drift Detection results regarding the changes I made to the TaskDefinition. (The Drift Detection says the "expected" value is that the container should contain the environment file - which is the correct "expected" value - but the "actual" value is reporting as missing. However, when I look at the actual TaskDefinition it is pointing to, it clearly has the environment file that Drift Detection says is missing.)