Ways to host a MongoDB cluster on Kubernetes

MongoDB is one of the most used database programs amongst developers. It is an open-source, general purpose, document-based, distributed NoSQL database server that is especially popular with JavaScript projects. It is instrumental in managing vast databases. In this blog, we will explain two ways of hosting a MongoDB cluster on Kubernetes.

Containerization provides developers flexibility, versatility, and support for many deployment environments. MongoDB helps Kubernetes in the automation of various critical aspects within containerized applications. Now, there are multiple ways to host a MongoDB cluster on K8s. However, we will discuss two of the easiest options:

• Using Community Kubernetes Operator.
• Using a custom Docker Image and Deployments.

Let’s see how we can use these options in detail.

1. Using Community Kubernetes Operator

Kubernetes Operator provides an interface to manage third-party applications just like Kubernetes-native objects. MongoDB Kubernetes Operator helps in creating, configuring, and managing MongoDB StatefulSet. The MongoDB Operators are of two types viz., MongoDB Community Operator and MongoDB Enterprise Kubernetes Operator. Both possess different sets of features and requirements. Let’s look at the steps to set up a cluster using the Community Kubernetes Operator.

• Install the Community Kubernetes Operator

Clone this repository:

git clone https://github.com/mongodb/mongodb-kubernetes-operator.git

Run the following command to create cluster-wide roles and role-bindings in the default namespace:

kubectl apply -f deploy/clusterwide

For each namespace that we want the Operator to watch, run the following commands to deploy a Role, RoleBinding, and ServiceAccount in that namespace:

kubectl apply -k config/rbac --namespace

Install the Custom Resource Definitions.

- Invoke the following command:

kubectl apply -f config/crd/bases/mongodbcommunity.mongodb.com_mongodbcommunity.yam

- Verify that the Custom Resource Definitions were installed successfully:

kubectl get crd/mongodbcommunity.mongodbcommunity.mongodb.com

Install the Operator.

- Invoke the following kubectl command to install the Operator in the specified namespace:

kubectl create -f config/manager/manager.yaml --namespace <my-namespace>

- Verify that the Operator installed successfully:

kubectl get pods --namespace <my-namespace>

Generate Self-signed CA Certificate

- Generate the CA key

openssl genrsa -out rootca.key 4096

Configure rootca.cnf

# For the CA policy 
[ policy_match ] 
countryName = match 
stateOrProvinceName = match 
organizationName = match 
organizationalUnitName = optional 
commonName = supplied 
emailAddress = optional 

[ req ] 
default_bits = 4096 
default_keyfile = rootca.pem    ## The default private key file name. 
default_md = sha256                           ## Use SHA-256 for Signatures 
distinguished_name = req_dn 
req_extensions = v3_req 
x509_extensions = v3_ca # The extentions to add to the self-signed cert 

[ v3_req ] 
subjectKeyIdentifier  = hash 
basicConstraints = CA:FALSE 
keyUsage = critical, digitalSignature, keyEncipherment 
nsComment = "OpenSSL Generated Certificate for TESTING only.  NOT FOR PRODUCTION        USE." 
extendedKeyUsage  = serverAuth, clientAuth 

[ req_dn ] 
countryName = Country Name (2 letter code) 
countryName_default = IN 

countryName_min = 2 
countryName_max = 2 

stateOrProvinceName = State or Province Name (full name) 
stateOrProvinceName_default = Pune 
stateOrProvinceName_max = 64 

localityName = Locality Name (eg, city) 
localityName_default = Pune 
localityName_max = 64 

organizationName = Organization Name (eg, company) 
organizationName_default = TestComp 
organizationName_max = 64 

organizationalUnitName = Organizational Unit Name (eg, section) 
organizationalUnitName_default = TestComp 
organizationalUnitName_max = 64 

commonName = Common Name (eg, YOUR name) 
commonName_max = 64 

[ v3_ca ] 
# Extensions for a typical CA 

subjectKeyIdentifier=hash 
basicConstraints = critical,CA:true 
authorityKeyIdentifier=keyid:always,issuer:always

Generate CA Certificate

openssl req -new -x509 -days 36500 -key rootca.key -out rootca.crt -config rootca.cnf

Upload CA Certificate to Kubernetes Cluster

kubectl create configmap ca-config-map --from-file=”~/ca.crt” --namespace <your-namespace> 
kubectl create secret tls ca-key-pair  --cert=”~/ca.crt”  --key=”~/ca.key” --namespace <your-namespace>

• Create a MongoDB replica set in Kubernetes

Install Cert Manager.

Create the Cert Manager issuer, this will create the certificates required by the MongoDB replica set.

cat <<EOF | kubectl apply -f - 
apiVersion: cert-manager.io/v1 
kind: Issuer 
metadata: 
name: ca-issuer-mongo 
  namespace: mongodb 
spec: 
  ca: 
    secretName: ca-key-pair 
--- 
apiVersion: cert-manager.io/v1 
kind: Certificate 
metadata: 
  name: cert-manager-certificate 
  namespace: mongodb 
spec: 
  secretName: mongodb-tls 
  issuerRef: 
    name: ca-issuer-mongo 
    kind: Issuer 
  commonName: "*.mongo-replicaset-svc.mongodb.svc.cluster.local" 
  dnsNames: 
  - "*.mongo-replicaset-svc.mongodb.svc.cluster.local" 
  - mongo-replicaset-0.com 
  - mongo-replicaset-1.com 
  - mongo-replicaset-2.com 
EOF

Use the Operator to create the MongoDB Replica set.

cat <<EOF | kubectl apply -f - 
apiVersion: mongodbcommunity.mongodb.com/v1 
kind: MongoDBCommunity 
metadata: 
  name: mongo-replicaset 
  namespace: mongodb 
spec: 
  members: 3 
  type: ReplicaSet 
  version: "5.0.2" 
  replicaSetHorizons: 
  - horizon: mongo-replicaset-0.com:27017 
  - horizon: mongo-replicaset-1.com:27017 
  - horizon: mongo-replicaset-2.com:27017 
  security: 
    tls: 
      enabled: true 
      certificateKeySecretRef: 
        name: mongodb-tls 
      caConfigMapRef: 
        name: ca-config-map 

    authentication: 
      modes: ["SCRAM"] 
  users: 
    - name: admin 
      db: admin 
      passwordSecretRef: # a reference to the secret that will be used to generate the user's password 
        name: admin-password 
      roles: 
        - name: clusterAdmin 
          db: admin 
        - name: userAdminAnyDatabase 
          db: admin 
        - name: root 
          db: admin 
      scramCredentialsSecretName: admin-scram 
    - name: dumpUser 
      db: admin 
      passwordSecretRef: # a reference to the secret that will be used to generate the user's password 
        name: dumpuser-password 
      roles: 
        - name: readWriteAnyDatabase 
          db: admin 
      scramCredentialsSecretName: dumpuser-scram 
  additionalMongodConfig: 
    storage.wiredTiger.engineConfig.journalCompressor: zlib 
  statefulSet: 
    spec: 
      volumeClaimTemplates: 
        - metadata: 
            name: data-volume 
          spec: 
            storageClassName: mongodb-ssd-storage 
            accessModes: [ "ReadWriteOnce" ] 
            resources: 
              requests: 
                storage: 512Gi 
--- 
apiVersion: v1 
kind: Secret 
metadata: 
  name: admin-password 
  namespace: mongodb 
type: Opaque 
stringData: 
  password: password 
--- 
apiVersion: v1 
kind: Secret 
metadata: 
  name: dumpuser-password 
  namespace: mongodb 
type: Opaque 
stringData: 
  password: password 
EOF

Expose the MongoDB replica set to the internet by creating Load Balancer service.

cat <<EOF | kubectl apply -f - 
apiVersion: v1 
kind: Service 
metadata: 
  name: mongo-replicaset-0 
  namespace: mongodb 
spec: 
  ports: 
  - port: 27017 
    protocol: TCP 
    targetPort: 27017 
  selector: 
    statefulset.kubernetes.io/pod-name: mongo-replicaset-0 
  type: LoadBalancer 
--- 
apiVersion: v1 
kind: Service 
metadata: 
  name: mongo-replicaset-1 
  namespace: mongodb 
spec: 
  ports: 
  - port: 27017 
    protocol: TCP 
    targetPort: 27017 
  selector: 
    statefulset.kubernetes.io/pod-name: mongo-replicaset-1 
  type: LoadBalancer 
--- 
apiVersion: v1 
kind: Service 
metadata: 
  name: mongo-replicaset-2 
  namespace: mongodb 
spec: 
  ports: 
  - port: 27017 
    protocol: TCP 
    targetPort: 27017 
  selector: 
    statefulset.kubernetes.io/pod-name: mongo-replicaset-2 
  type: LoadBalancer 
EOF

The IP address generated from the Load balancer SVC should be bound to the domain names mentioned in the replicaSetHorizons, for example: mongo-replicaset-0.com, mongo-replicaset-1.com, mongo-replicaset-2.com.

2. Using a custom Docker Image and Deployments

While using an operator to deploy a MongoDB cluster sure makes the cluster creation simpler, we lose a lot of control over the MongoDB cluster setup. For instance, to expand the Persistent Volume (PV) size, we won’t be able to do it without downtime. We need to use self-signed certificates since no CA provides certificates for the local domain… read more