Elastic Beanstalk solution stack upgrades malfunction

Elastic Beanstalk solution stack upgrades malfunction due to old environment instance(s) termination failure:
-Auto Scaling group update progress: Terminating instance(s) [i-04c274d9f6fb09012]; replacing with 1 new instance(s).
-Auto Scaling group update progress: Successfully terminated instance(s) [i-04c274d9f6fb09012] (Progress 100%). <--FALSE - Old instance stays running
As of now, both instances stay running and the old one needs to be terminated manually or via a user script
Repro. instructions:
Create an EB environment using "64bit Amazon Linux 2 v3.5.0 running Docker" solution stack, with 1 instance, use default AWS EB web app. Not sure if autoscaling makes a difference, but I have it enabled to autoscale up to 1 instance. Try to update the solution stack to v3.5.1 and you will see that the old instance doesn't automatically get terminated and you'll see the EB event stating that it did get terminated, which is false.

I believe that the issue started when New EC2 Experience started to get forced upon users, which wasn't too long ago.