Setting up content Security policy Header for base 64 file in website

Since you are hosting site on EC2, You can set any headers you want.

But if you are asking, can you setup custom headers on CloudFront response? the answer is yes you can setup a static header "Content-Security-Policy". But if you want to change the headers dynamically you would need to use your web server or application on EC2 to generate that.

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/understanding-response-headers-policies.html#understanding-response-headers-policies-custom