Introduction to Persistent Volumes and mounting GCP Buckets on Containers

The global application container market has been on the up and is expected to go beyond $8 billion by 2025. Almost all major players are trying their best to capture this market by introducing feature-rich services. Google Cloud Platform (GCP) is one such player that provides services for everything from development to production, from container builder, registry, optimized OS to Kubernetes engine for orchestration. Google Kubernetes Engine (GKE) is the best tool to manage containerized applications using Google infrastructure that provides a managed environment for deploying, managing, and scaling. GKE works on the Kubernetes cluster and when you try to run services on Kubernetes it is advised to preserve the data of those services that are required at the time of a catastrophe or important data that can be needed in later stages. To store the important data Kubernetes uses persistent volumes.

So, what is a Persistent Volume (PV)?

A persistent volume is a cluster-wide resource used to store data. The method used to store data is such that it continues beyond the lifetime of a pod as well. On a worker node, the Persistent Volume doesn’t get backed by locally-attached storage. But, it gets backed by a networked storage system like EBS, NFS or a distributed file system like Ceph. Persistent Volumes run independently which means data stored within persistent volumes continue to exist even if pods are deleted or recreated. Persistent Volumes are provisioned by Persistent Volume Claims (PVC) and it is a request for the platform when you want to create a PV. You can also attach PVs to your pods through a PVC. Pods use PVCs as Volumes. The cluster inspects the PVC to figure out the bound volume. It also mounts the volume for the pod.

Persistent Volumes support the following access modes:

• ReadWriteOnce: Volume can be mounted as read-write by a single node.
• ReadOnlyMany: Volume can be mounted as read-only by many nodes.
• ReadWriteMany: Volume can be mounted as read-write by multiple nodes.

What are the challenges with Persistent Volumes on GKE for simultaneous ReadWrite on multiple nodes?

Persistent Volume (PV) is a cluster-wide resource and is used to store data that can be retained even if the associated pod is not there. In a typical business application, there could be complex data pipelines that continuously process raw data to produce business outputs. To process this data parallelly, there are multiple pods involved in the…read more