K3S Persistent Storage

In this tutorial I will show how to install Longhorn (Archived) to manage persistent storage requirements of the kubernetes cluster.

As first step be sure to have the program iscsiadm installed (on Fedora this is provided by the package iscsi-initiator-utils) and start iscsid and enable it.

systemctl {start,enable} iscsid

Be sure to check the official requirements (Archived) for the full Longhorn installation.

Then we download the deploy for longhorn-system

wget -O 03a-longhorn-system.yaml
https://raw.githubusercontent.com/longhorn/longhorn/v1.2.4/deploy/longhorn.yaml

and with the editor search for kind: StorageClass and change parameters.numberOfReplicas: 1 (otherwise volumes will fail to show up) and reclaimPolicy: Retain. You also cannot modify storage classes after creation so you better do it now. In case it gives you errors in deploying be sure to remove all comment lines that could create some problems in YAML interpretation.

Now we also proceed to deploy an ingress with basic-auth to access the longhorn UI. We use the following commands to generate the authorization (taken from this post (Archived)). Be sure to substitute USERNAME and the typed PASSWORD.

printf "USERNAME:`openssl passwd -apr1`\n" >> auth
kubectl create secret generic basic-auth --from-file auth -n longhorn-system 

Now you can check that all pods are ready

kubectl get pods -n longhorn-system

and we finish by deploying the ingress with basic auth middleware for the longhorn UI. Since we already installed the cert-issuer we should have no problem in waiting for a certificate.

# 03b-longhorn-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: basic-auth-middleware
  namespace: longhorn-system
spec:
  basicAuth:
    removeHeader: true
    secret: basic-auth
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    kubernetes.io/ingress.class: traefik
    # The most tricky part: NAMESPACE-MIDDLEWARE@kubernetescrd
    traefik.ingress.kubernetes.io/router.middlewares: longhorn-system-basic-auth-middleware@kubernetescrd
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  tls:
  - hosts:
    - longhorn.${MASTER_FQDN}
    secretName: secret-letsencrypt
  rules:
  - host: longhorn.${MASTER_FQDN}
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: longhorn-frontend
            port:
              number: 80

Now remember when issuing a PersistentVolumeClaim to add storageClassName: longhorn.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: longhorn-pvc
  namespace: test
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: longhorn
  resources:
    requests:
      storage: 1Gi