Ceph Deployment with Rook: Part 1

We will cover a Ceph deployment on a Kubernetes cluster using Rook.

Previously, we deploy a Ceph cluster on the Linux, specifically on Ubuntu 20.04 using Docker container runtime. In a series of articles, we shows various aspects of a Ceph cluster: as block storages, as distributed file systems, as object storages.

This time, a Ceph cluster will be deployed on a Kubernetes cluster using Rook operator. Before we start explaining how to do it, we first list all the versions of software packages that will be used.

OS : Ubuntu 22.04
Container Runtime : Docker 20.10.21
Kubernetes Deployment : rke 1.3.14
Kubernetes Version : v1.24.4 (kubectl as well)
helm : v3.11.3

Machine Setup

We will use four nodes.

• 1 x Deploy node:
• 1 x Kubernetes master node
• 2 x Kubernetes worker node with a 256GB disk on each

All the nodes has Ubuntu 22.04. A passwordless-ssh is configured on the deploy node to all three Kubernetes nodes. The container runtime package (docker.io) is installed on all three k8s nodes.

On the deploy node, we get rke, kubectl and helm executables and make them available by updating the PATH environment variable.

sudo apt install -y docker.io
wget https://github.com/rancher/rke/releases/download/v1.3.14/rke_linux-amd64
wget https://get.helm.sh/helm-v3.11.3-linux-amd64.tar.gz
curl -LO https://dl.k8s.io/release/v1.24.4/bin/linux/amd64/kubectl

These are commands that I used to install/obtain four packages.

Kubernetes Deployment

We will deploy a Kubernetes cluster using rke from the deploy node. First, we need to have cluster.yml file to describe the cluster. A diagram of the cluster is shown below.

Kubernetes nodes

All the nodes should have docker installed and a chosen user (throughout this article, we will use cluster-admin as the chosen user) should have a permission to run docker command. To give the permission, use the following commands.

sudo usermod -aG docker cluster-admin
sudo systemctl restart docker.service