Hi AWS Users,
I am trying to spin up a MSK cluster with a custom MSK configuration using my serverless app. I wrote the cloudformation template for the generation of the MSK Cluster and was able to successfully bring it up. I recently saw that AWS added cloudformation template of AWS::MSK::Configuration
. [1] I was trying that out to create a custom configuration. The Configuration requires a ServerProperties
key that is usually a PlainText in AWS console.
An example of Server Properties:
auto.create.topics.enable=true
default.replication.factor=2
min.insync.replicas=2
num.io.threads=8
num.network.threads=5
num.partitions=10
num.replica.fetchers=2
replica.lag.time.max.ms=30000
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
unclean.leader.election.enable=true
zookeeper.session.timeout.ms=18000
AWS::MSK::Configuration
accepts base64 (api functionality) and I have been trying to implement this. I am using the cloudformation Fn::Base64
functionality.
e.g:
Resources:
ServerlessMSKConfiguration:
Type: AWS::MSK::Configuration
Properties:
ServerProperties:
Fn::Base64: auto.create.topics.enable=true
This gives me back a 400 error during deploy.
Resource handler returned message: "[ClientRequestToken: xxxxx] Invalid request body (Service: Kafka, Status Code: 400, Request ID: 1139d840-c02d-4fdb-b68c-cee93673d89d, Extended Request ID: null)" (RequestToken: xxxx HandlerErrorCode: InvalidRequest)
Can someone please help me format this ServerProperties properly, not sure how to give the proper base64 string in the template. Any help is much appreciated.
[1] - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-configuration.html