Is it possible to view the cloudformation of managed Response Headers Policies?

You will need to create your own custom policy. You can describe the managed policy in the CLI to get the JSON output that you can use in your custom policy in CloudFormation

See:

• https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudfront-responseheaderspolicy.html
• https://docs.aws.amazon.com/cli/latest/reference/cloudfront/get-response-headers-policy.html

aws cloudfront get-response-headers-policy --id 67f7725c-6f97-4210-82d7-5512b31e9d03

{
    "ETag": "E23ZP02F085DFQ",
    "ResponseHeadersPolicy": {
        "Id": "67f7725c-6f97-4210-82d7-5512b31e9d03",
        "LastModifiedTime": "1970-01-01T00:00:00+00:00",
        "ResponseHeadersPolicyConfig": {
            "Comment": "Adds a set of security headers to every response",
            "Name": "Managed-SecurityHeadersPolicy",
            "SecurityHeadersConfig": {
                "XSSProtection": {
                    "Override": false,
                    "Protection": true,
                    "ModeBlock": true
                },
                "FrameOptions": {
                    "Override": false,
                    "FrameOption": "SAMEORIGIN"
                },
                "ReferrerPolicy": {
                    "Override": false,
                    "ReferrerPolicy": "strict-origin-when-cross-origin"
                },
                "ContentSecurityPolicy": {},
                "ContentTypeOptions": {
                    "Override": true
                },
                "StrictTransportSecurity": {
                    "Override": false,
                    "AccessControlMaxAgeSec": 31536000
                }
            }
        }
    }
}