EC2 cost estimate when migrate Windows Server 2008 to AWS

While the operating system isn't factored into the cost because you are maybe preferring to using Window instance (Linux is cheaper), it's important to remember that every time you run an EC2 instance, you're charged for the usage of that instance. Therefore, it's crucial to incorporate this into your cost estimation. However, Amazon EC2 does offer various savings plans that can help reduce your expenses.

whether the cost of Windows Server needs to be included

That depends on how the EC2 instances are to be licensed. Choosing an AMI with Windows Server essentially means paying AWS for the infrastructure that the instance runs on, and on top of that a licensing cost for Microsoft software.

There is a bring-your-own-licence (BYOL) option, which may be more economical if you have already purchased Microsoft software and the associated licensing. There is much more detailed information in the FAQ at https://aws.amazon.com/windows/faq/#licensing

This blog post is also worth reading https://aws.amazon.com/blogs/aws/a-new-simplified-bring-your-own-license-experience-for-microsoft-windows-server/

Away from the licensing costs, if the long-term aim is to run many EC2 instances continuously then look at EC2 Reserved Instances https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-reserved-instances.html

You can always start off with a standard instance (i.e. pay-as-you-go) and convert to RI later once you're confident the instance is correctly sized.