By using AWS re:Post, you agree to the AWS re:Post Terms of Use

Switching from EC2-Launch v1 to EC2-Launch v2 has broken my website.

0

I've automated the updating of an IIS/DotNet website, thus:

  • zip file of latest version of website is uploaded to S3
  • S3 notification triggers lambda
  • lambda launches a "staging instance" from a "base ami"
  • staging instance UserData does everything needed to make the website work -- installs prerequisites, unzips the zip file, configures IIS...
  • Last thing the staging instance does is run the appropriate (version-dependent) EC2-Launch command to get the instance ready to be imaged, then invokes another lambda that...
  • images the staging instance
  • Does a CloudFormation update, changing the parameter that says what image a load balancer should launch.

For v1, the "appropriate command" is "InitializeInstance.ps1 --Schedule". For v2, it's "EC2Launch.exe sysprep".

The only thing that has changed on the "base image" is the version of EC2 launch. The only thing that has changed in the "staging instance's" UserData is how EC2Launch is used.

One more maybe relevant datum. I need to set the computer name, because the website needs to determine which environment it's running in -- and therefore which database server to connect to. It does this based on the computer name, which contains the environment as a substring. With V1, I was just doing it with the PowerShell commandlet "Rename-Computer". This wasn't working with v2. So now I'm putting the following in the config file before running sysprep:

      - task: setHostName
        inputs:
          reboot: true
          hostName:myNewName

That does seem to get the hostname changed. But...

With V2, attempts to connect to the website fail. Connection Reset.

Looking around on the server instance, there is very little obviously amiss. IIS is running. The website itself is running and configured right. The app pool is running. I tried to get a clue by looking at \inetpub\logs, but the really curious thing was that there were no logs. Really? There were always logs before -- even when things were broken. I guess it's breaking now before it even gets around to writing logs. A clue, maybe, but not much of one.

The only other thing I could find was that, every time I try to get to the broken website from a client, the Windows Security Logs show one of more logs of failed event id 5061. (These events don't happen on a working system.) I've looked into event id 5061 enough to find out that it has something to do with an encryption key. But I sure am having trouble figuring out why the change of EC2Launch version would have such a result, or what to do about it. Any ideas?

Security Log with event id 5061

  • What happens when you try and connect to the website? What errors? What happens when going direct vs cloudfront?

asked 23 days ago50 views
1 Answer
0

The issues you're experiencing after switching from EC2Launch v1 to EC2Launch v2 are likely related to the differences in how these versions handle instance configuration and sysprep processes. Here are some potential causes and solutions to consider:

  1. Configuration migration: When migrating from EC2Launch v1 to v2, not all configurations are automatically transferred. The migration tool doesn't detect scheduled tasks linked to EC2Launch v1 scripts, so you may need to manually configure these in EC2Launch v2. Review your agent-config.yml file and ensure all necessary tasks are properly set up.

  2. Sysprep process: The sysprep process in EC2Launch v2 might be handling some aspects differently than v1. This could affect how your instance is prepared for imaging. Make sure you're using the correct sysprep command for v2: "EC2Launch.exe sysprep".

  3. Computer name setting: Your approach to setting the computer name has changed. While using the setHostName task in the config file is correct for EC2Launch v2, ensure that this change isn't causing any unexpected side effects. The reboot after setting the hostname might be interfering with other setup processes.

  4. User data execution timing: In EC2Launch v2, the timing of user data execution has changed. It now runs during the PostReady stage, and the exact timing depends on the format of your user data (YAML 1.0, YAML 1.1, or XML). Ensure your user data is in the correct format and running at the appropriate time.

  5. IIS configuration: The lack of IIS logs suggests that IIS might not be fully initialized or configured correctly after the sysprep process. Review your IIS setup process in the user data script to ensure it's compatible with the EC2Launch v2 workflow.

  6. Encryption key issues: The Event ID 5061 relates to cryptographic operations, specifically the sealing of keys. This could indicate that there's an issue with how system or user keys are being handled after the sysprep process. This might be related to how EC2Launch v2 prepares the instance for imaging.

To troubleshoot:

  1. Review and update your agent-config.yml file to ensure all necessary tasks are properly configured for EC2Launch v2.
  2. Check the EC2Launch v2 logs (located in C:\ProgramData\Amazon\EC2Launch\log) for any error messages or unexpected behavior.
  3. Verify that your user data script is running at the correct time and completing all necessary setup steps.
  4. Consider temporarily disabling the setHostName task to see if it's causing any conflicts.
  5. Ensure that your IIS setup process is compatible with the EC2Launch v2 workflow.
  6. Investigate the cryptographic issues indicated by Event ID 5061, possibly by reviewing system security policies and how they're applied during the sysprep process.

If these steps don't resolve the issue, you may need to engage with AWS support for more in-depth troubleshooting, as there could be complex interactions between EC2Launch v2, your custom setup process, and Windows system configurations.
Sources
Migrate to EC2Launch v2 for Windows instances - Amazon Elastic Compute Cloud
Troubleshoot issues with the EC2Launch v2 agent - Amazon Elastic Compute Cloud
Use the EC2Launch v2 agent to perform tasks during EC2 Windows instance launch - Amazon Elastic Compute Cloud
Windows Service administration for EC2Launch v2 and EC2Config agents - Amazon Elastic Compute Cloud

profile picture
answered 23 days ago
  • I indicated in my original post that I wasn't sure whether or not changing the machine name had anything to do with it. Now I am sure. It doesn't.

    I tried removing my edit of the config file, so the machine name would not be changed. And it's failing in exactly the same way. Still getting the failed event id 5061's in the Security log. Still getting no inetpub\Logs. It doesn't matter that the website code can't figure out which DB to connect to based on the system name -- it looks like the website code just isn't running.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions