Ingress for Ceph RGW service

Some Kubernetes common applications use object storage. Gitlab and Loki are such examples. Their default installation comes with minio deployment. We sometimes replace this with Ceph RGW service. In order to provide HA (high available) object storage, we use an ingress service on top of Ceph RGW service.

Previously, we covers how to deploy Ceph RGW (Rados Gateway) service in the two previous articles.

• Ceph Object Storage
• Ceph Object Storage with https

In order to use ingress with https (ssl_cert), we had to change https rgw service back to http rgw service.

At the end of this process, we will have the following architecture.

Let’s start with deploying rgw service with http.

sudo ceph orch apply -i rgw-k8s-default-http.yaml

The yaml file is shown below.

service_type: rgw
service_id: k8s-default
placement:
  label: rgw
  count: 2
networks:
- 172.25.0.0/17
spec:
  ssl: false

This will deply two rgw daemons on hosts with rgw label without https. Specific values can be changed accordingly including the service id and networks.

The ports can also be changed by specifying rgw_frontend_port: [port] in the spec section.