MSK Custom Configuration using Cloudformation

Question

Hi AWS Users,

I  am trying to spin up a MSK cluster with a custom MSK configuration using my serverless app. I wrote the cloudformation template for the generation of the MSK Cluster and was able to successfully bring it up. I recently saw that AWS added cloudformation template of `AWS::MSK::Configuration`. [1] I was trying that out to create a custom configuration. The Configuration requires a `ServerProperties`key that is usually a PlainText in AWS console.

An example of Server Properties:
```
auto.create.topics.enable=true
default.replication.factor=2
min.insync.replicas=2
num.io.threads=8
num.network.threads=5
num.partitions=10
num.replica.fetchers=2
replica.lag.time.max.ms=30000
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
unclean.leader.election.enable=true
zookeeper.session.timeout.ms=18000
``` 
`AWS::MSK::Configuration` accepts base64 (api functionality) and I have been trying to implement this. I am using the cloudformation `Fn::Base64` functionality. 
e.g: 
```
Resources:
  ServerlessMSKConfiguration:
    Type: AWS::MSK::Configuration
    Properties:
      ServerProperties:
        Fn::Base64: auto.create.topics.enable=true
```
This gives me back a 400 error during deploy. 
```
Resource handler returned message: "[ClientRequestToken: xxxxx] Invalid request body (Service: Kafka, Status Code: 400, Request ID: 1139d840-c02d-4fdb-b68c-cee93673d89d, Extended Request ID: null)" (RequestToken: xxxx HandlerErrorCode: InvalidRequest)
```

Can someone please help me format this ServerProperties properly, not sure how to give the proper base64 string in the template. Any help is much appreciated.

[1] - [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html](MSK::Configuration)

Answer

Here's one that works for me. It took some trial and error to figure out that Name and Description only accept some limited special characters, and the KafakVersionsList couldn't be a variable. And this did not require !Base64 to create the configuration.

```
  MskConfiguration:
    Type: "AWS::MSK::Configuration"
    Properties:
      Description: !Sub "${EnvironmentName}-${EnvironmentShard}-msk-cluster-configuration"
      Name: !Sub "${EnvironmentName}-${EnvironmentShard}-msk-cluster-configuration"
      KafkaVersionsList: [2.8.1]
      ServerProperties: |
        auto.create.topics.enable=false
        default.replication.factor=2
```

MSK Custom Configuration using Cloudformation

Relevanter Inhalt