@cfn-modules/msk-cluster

AWS MSK (Kafka) cluster using two or three availability zones with public and private subnets. Cloudwatch logging enabled by default.

Usage no npm install needed!

<script type="module">
  import cfnModulesMskCluster from 'https://cdn.skypack.dev/@cfn-modules/msk-cluster';
</script>

README

cfn-modules: AWS MSK cluster

AWS MSK (Kafka Cluster) using two or three availability zones with public and private subnets. Cloudwatch logging enabled by default.

Install

Install Node.js and npm first!

npm i @cfn-modules/msk-cluster

Usage

---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'cfn-modules example'
Resources:
  Key:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        VpcModule: !GetAtt 'Vpc.Outputs.StackName' # required
        ClientSgModule: !GetAtt 'ClientSg.Outputs.StackName' # required
        KmsKeyModule: !GetAtt 'Key.Outputs.StackName' # optional
        BastionModule: !GetAtt 'Bastion.Outputs.StackName' # optional
        AlertingModule: !GetAtt 'Alerting.Outputs.StackName' # optional
        NumberOfBrokerNodes: !GetAtt 'Vpc.Outputs.NumberOfAvailabilityZones' # required
        KafkaVersion: '2.2.1' # optional
        InstanceType: 'kafka.t3.small' # optional
        MSKConfigurationArn: '' # optional
        MSKConfigurationNumber: '0' # optional
        EBSVolumeSize: '1' # optional
      TemplateURL: './node_modules/@cfn-modules/msk-cluster/module.yml'

Examples

none

Related modules

none

Parameters

Name Description Default Required? Allowed values
VpcModule Stack name of vpc module yes
ClientSgModule Stack name of client-sg module where traffic is allowed from on port 9092, 9095 and 2181 to the cluster yes
KmsKeyModule Stack name of kms-key module (only works in combination with Access := [Private, PublicRead]) no
BastionModule Stack name of module implementing Bastion no
AlertingModule Stack name of alerting module no
NumberOfBrokerNodes The number of broker nodes you want in the Amazon MSK cluster. You can submit an update to increase the number of broker nodes in a cluster. yes Has to be a multiple of the private subnets in your VPC.
KafkaVersion The version of Apache Kafka. 2.2.1 no
InstanceType The type of Amazon EC2 instances to use for brokers. kafka.t3.small no
MSKConfigurationArn Amazon Resource Name (ARN) of the MSK configuration to use. no
MSKConfigurationNumber Revision of the Amazon MSK configuration to use (required if MSKConfigurationArn is set). no
EBSVolumeSize The size in GiB of the EBS volume for the data drive on each broker node. 1 no
ClientBrokerEncryption Indicates the encryption setting for data in transit between clients and brokers. TLS no [TLS, PLAINTEXT, TLS_PLAINTEXT]
LogsRetentionInDays Specifies the number of days you want to retain log events in the specified log group 14 no [1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653]

Outputs

Name Interface Description Exported?
ModuleId global Id of the module no
ModuleVersion global Version of the module no
StackName global Name of the stack (used to pass module references) no
Arn ExposeArn Cluster ARN yes

Limitations

  • Scalable: Auto scaling is not provided by MSK service. You have to increase the number of broker nodes to scale manually.
  • Secure: Does not backup/snapshot the data.