When should I use IoT Core?

AWS IoT Core is a fully managed service, so you won't have to manage the underlying components. If you built similar functionality using AWS Lambda, API Gateway, etc you would have to configure it and write it. Here is a good blog with an overview of IoT Core.

IoT Core isn't limited to physical devices. Take a look at this reference architecture for a game backend.