Ways to host a MongoDB cluster on Kubernetes

Opcito Technologies
5 min readApr 27, 2022

--

MongoDB from Opcito

MongoDB is one of the most used database programs amongst developers. It is an open-source, general purpose, document-based, distributed NoSQL database server that is especially popular with JavaScript projects. It is instrumental in managing vast databases. In this blog, we will explain two ways of hosting a MongoDB cluster on Kubernetes.

Containerization provides developers flexibility, versatility, and support for many deployment environments. MongoDB helps Kubernetes in the automation of various critical aspects within containerized applications. Now, there are multiple ways to host a MongoDB cluster on K8s. However, we will discuss two of the easiest options:

  • Using Community Kubernetes Operator.
  • Using a custom Docker Image and Deployments.

Let’s see how we can use these options in detail.

1. Using Community Kubernetes Operator

Kubernetes Operator provides an interface to manage third-party applications just like Kubernetes-native objects. MongoDB Kubernetes Operator helps in creating, configuring, and managing MongoDB StatefulSet. The MongoDB Operators are of two types viz., MongoDB Community Operator and MongoDB Enterprise Kubernetes Operator. Both possess different sets of features and requirements. Let’s look at the steps to set up a cluster using the Community Kubernetes Operator.

  • Install the Community Kubernetes Operator

Clone this repository:

git clone https://github.com/mongodb/mongodb-kubernetes-operator.git

Run the following command to create cluster-wide roles and role-bindings in the default namespace:

kubectl apply -f deploy/clusterwide

For each namespace that we want the Operator to watch, run the following commands to deploy a Role, RoleBinding, and ServiceAccount in that namespace:

kubectl apply -k config/rbac --namespace

Install the Custom Resource Definitions.

- Invoke the following command:

kubectl apply -f config/crd/bases/mongodbcommunity.mongodb.com_mongodbcommunity.yam

- Verify that the Custom Resource Definitions were installed successfully:

kubectl get crd/mongodbcommunity.mongodbcommunity.mongodb.com

Install the Operator.

- Invoke the following kubectl command to install the Operator in the specified namespace:

kubectl create -f config/manager/manager.yaml --namespace <my-namespace>

- Verify that the Operator installed successfully:

kubectl get pods --namespace <my-namespace>

Generate Self-signed CA Certificate

- Generate the CA key

openssl genrsa -out rootca.key 4096

Configure rootca.cnf

# For the CA policy 
[ policy_match ]
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional

[ req ]
default_bits = 4096
default_keyfile = rootca.pem ## The default private key file name.
default_md = sha256 ## Use SHA-256 for Signatures
distinguished_name = req_dn
req_extensions = v3_req
x509_extensions = v3_ca # The extentions to add to the self-signed cert

[ v3_req ]
subjectKeyIdentifier = hash
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
nsComment = "OpenSSL Generated Certificate for TESTING only. NOT FOR PRODUCTION USE."
extendedKeyUsage = serverAuth, clientAuth

[ req_dn ]
countryName = Country Name (2 letter code)
countryName_default = IN

countryName_min = 2
countryName_max = 2

stateOrProvinceName = State or Province Name (full name)
stateOrProvinceName_default = Pune
stateOrProvinceName_max = 64

localityName = Locality Name (eg, city)
localityName_default = Pune
localityName_max = 64

organizationName = Organization Name (eg, company)
organizationName_default = TestComp
organizationName_max = 64

organizationalUnitName = Organizational Unit Name (eg, section)
organizationalUnitName_default = TestComp
organizationalUnitName_max = 64

commonName = Common Name (eg, YOUR name)
commonName_max = 64

[ v3_ca ]
# Extensions for a typical CA

subjectKeyIdentifier=hash
basicConstraints = critical,CA:true
authorityKeyIdentifier=keyid:always,issuer:always

Generate CA Certificate

openssl req -new -x509 -days 36500 -key rootca.key -out rootca.crt -config rootca.cnf

Upload CA Certificate to Kubernetes Cluster

kubectl create configmap ca-config-map --from-file=”~/ca.crt” --namespace <your-namespace> 
kubectl create secret tls ca-key-pair --cert=”~/ca.crt” --key=”~/ca.key” --namespace <your-namespace>
  • Create a MongoDB replica set in Kubernetes

Install Cert Manager.

Create the Cert Manager issuer, this will create the certificates required by the MongoDB replica set.

cat <<EOF | kubectl apply -f - 
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: ca-issuer-mongo
namespace: mongodb
spec:
ca:
secretName: ca-key-pair
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: cert-manager-certificate
namespace: mongodb
spec:
secretName: mongodb-tls
issuerRef:
name: ca-issuer-mongo
kind: Issuer
commonName: "*.mongo-replicaset-svc.mongodb.svc.cluster.local"
dnsNames:
- "*.mongo-replicaset-svc.mongodb.svc.cluster.local"
- mongo-replicaset-0.com
- mongo-replicaset-1.com
- mongo-replicaset-2.com
EOF

Use the Operator to create the MongoDB Replica set.

cat <<EOF | kubectl apply -f - 
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: mongo-replicaset
namespace: mongodb
spec:
members: 3
type: ReplicaSet
version: "5.0.2"
replicaSetHorizons:
- horizon: mongo-replicaset-0.com:27017
- horizon: mongo-replicaset-1.com:27017
- horizon: mongo-replicaset-2.com:27017
security:
tls:
enabled: true
certificateKeySecretRef:
name: mongodb-tls
caConfigMapRef:
name: ca-config-map

authentication:
modes: ["SCRAM"]
users:
- name: admin
db: admin
passwordSecretRef: # a reference to the secret that will be used to generate the user's password
name: admin-password
roles:
- name: clusterAdmin
db: admin
- name: userAdminAnyDatabase
db: admin
- name: root
db: admin
scramCredentialsSecretName: admin-scram
- name: dumpUser
db: admin
passwordSecretRef: # a reference to the secret that will be used to generate the user's password
name: dumpuser-password
roles:
- name: readWriteAnyDatabase
db: admin
scramCredentialsSecretName: dumpuser-scram
additionalMongodConfig:
storage.wiredTiger.engineConfig.journalCompressor: zlib
statefulSet:
spec:
volumeClaimTemplates:
- metadata:
name: data-volume
spec:
storageClassName: mongodb-ssd-storage
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 512Gi
---
apiVersion: v1
kind: Secret
metadata:
name: admin-password
namespace: mongodb
type: Opaque
stringData:
password: password
---
apiVersion: v1
kind: Secret
metadata:
name: dumpuser-password
namespace: mongodb
type: Opaque
stringData:
password: password
EOF

Expose the MongoDB replica set to the internet by creating Load Balancer service.

cat <<EOF | kubectl apply -f - 
apiVersion: v1
kind: Service
metadata:
name: mongo-replicaset-0
namespace: mongodb
spec:
ports:
- port: 27017
protocol: TCP
targetPort: 27017
selector:
statefulset.kubernetes.io/pod-name: mongo-replicaset-0
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: mongo-replicaset-1
namespace: mongodb
spec:
ports:
- port: 27017
protocol: TCP
targetPort: 27017
selector:
statefulset.kubernetes.io/pod-name: mongo-replicaset-1
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: mongo-replicaset-2
namespace: mongodb
spec:
ports:
- port: 27017
protocol: TCP
targetPort: 27017
selector:
statefulset.kubernetes.io/pod-name: mongo-replicaset-2
type: LoadBalancer
EOF

The IP address generated from the Load balancer SVC should be bound to the domain names mentioned in the replicaSetHorizons, for example: mongo-replicaset-0.com, mongo-replicaset-1.com, mongo-replicaset-2.com.

2. Using a custom Docker Image and Deployments

While using an operator to deploy a MongoDB cluster sure makes the cluster creation simpler, we lose a lot of control over the MongoDB cluster setup. For instance, to expand the Persistent Volume (PV) size, we won’t be able to do it without downtime. We need to use self-signed certificates since no CA provides certificates for the local domain… read more

--

--

Opcito Technologies

Product engineering experts specializing in DevOps, Containers, Cloud, Automation, Blockchain, Test Engineering, & Open Source Tech