Resolving “Remediate ESXi Host(s) to be compliant with Cluster’s Image” Error when adding a new ESXi Host in Amazon EVS (Amazon EVS Custom Add-on version mismatch)

Background

This issue occurs if you initially deployed Amazon EVS with VMware Cloud Foundation (VCF) version 5.2.1.x and have since upgraded to VCF version 5.2.2.x. The newly added host using the ESXi build 24859861 AMI contains a newer version of the Amazon EVS Custom Add-on than the version defined in the existing vSphere cluster image. vSphere Lifecycle Manager enforces image consistency across all ESXi hosts in a cluster—ensuring each host runs the same ESXi version, vendor add-ons, and any additional components (such as non‑standard VIBs). If you attempt to add a host where software versions do not match the cluster’s desired image, the operation will fail.

The Amazon EVS Custom Add-on provides the required drivers and host configuration that enables ESXi to run on EC2 metal instance types. This add-on is updated periodically, and the latest version is included with each new ESXi AMI.

Situation

You have run through the add Host procedure within SDDC Manager, as covered in this re:Post article or by following relevant Broadcom documentation.

During the “Applying the vSphere Cluster Image” phase, the task fails with the error message: "Remediate ESXi Host(s) to be Compliant with Cluster's Image".

The Host remains in the cluster and is left in maintenance mode.

Tasks

To resolve, complete the following tasks:

1: Create a new, temporary vSphere cluster image by importing the image from the new Host

2: Update the existing cluster’s image to use the newer Amazon EVS Custom Add-on version and restart the failed task in SDDC Manager

Task 1: Create a temporary vSphere cluster image by importing the image from the new Host

1. Log into the Amazon EVS environment vSphere console.
2. From the Hosts and Clusters view, enter the context menu of the Datacenter object and select New Cluster
3. Enter a name for the temporary cluster, then choose Import image from an existing host in the Center inventory, leave all other settings at default then select Next

Figure 1. Example adding a new cluster, using the option to Import image from an existing host in the vCenter inventory option.

4. Select the new Host (it should be in maintenance mode) and select Next

Figure 2. Selecting the new Host in mainteance mode to import the image from

5. Review your selections and then select Finish

Note: This process will automatically extract and import the entire ESXi image of the new Host into vSphere Lifecycle Manager.

When the process is complete, vSphere Lifecycle Manager will list the newly discovered version of the Amazon EVS Custom Add-on, making it available for inclusion in your cluster image.

You can confirm this by going to the vSphere Lifecycle Manager within the vSphere console and selecting Vendor Add-ons

Figure 3. Example of Vendor Addons section within vSphere Lifecycle Manager showing 2 versions of the Amazon EVS Custom Addon.

Task 2: Update the existing cluster’s image to use the newer Amazon EVS Custom Add-on version and remediate all Hosts within the Cluster

Now that you’ve extracted the ESXi image along with the updated Amazon EVS Custom Add-on, the next step is to edit the cluster image and update its configuration to use the new add-on version.

1. In the Amazon EVS environment vSphere console, select Inventory and then select the vSphere cluster that you want to add the new host to.

2. From the right-hand pane, select Updates > Image > Edit, to modify the cluster image.

Figure 4. Example of the cluster image settings and where to edit the current image.

3. Select the pencil icon next to Amazon EVS Custom Addon 803.1.0.2-1 and then use the drop-down box to select the new version of the add-on e.g. 803.1.0.3-1, then choose Select.

Figure 5. Select the drop-down arrow under the Version heading to select the latest version to match the new Host being added.

4. Select Save.

5. Once complete, return to the Amazon EVS SDDC Manager and select Restart Task on the failed task for adding the new Host.

Important: Updating the cluster image will trigger a cluster‑wide compliance check and remediation cycle. All ESXi hosts will be evaluated against the newly defined version of the Amazon EVS Custom Add-on. Any host running an older version will automatically enter maintenance mode, be updated, and then rebooted.

Once all hosts have been successfully remediated, the newly added host will exit maintenance mode and will be ready for service.

Summary

You have successfully extracted the latest version of the Amazon EVS Custom Add-on from the newly added host, incorporated that version into your vSphere cluster image, and remediated all hosts in the cluster to ensure they are now compliant and running the updated add-on.