How to create custom Helm charts

Here we create a custom helm chart for OpenProject with data persistence and an external database.

How to create custom Helm charts

TL;DR

Make sure you have Helm installed

  1. helm create [chart name]
  2. Edit files to change to a Docker image that you want to deploy
  3. Customize with any K8s features within the templates folder
  4. Deploy with helm install [release name] [chart folder]

Grab the custom chart created in this article for the community version of the project management software OpenProject, on Github.

Helm 3 overview

Helm 3 is a complete departure from the previous versions of helm. There is no server side component required. Whereas before you would need to add an application called Tiller to your cluster. Dependencies are handled directly within the chart.yaml file. No more requirements.yaml file needed. Helm 3 now also supports RBAC.

Even with all these changes under the hood, and some changes to the CLI as well, Helm 2 charts are still largely compatible with Helm 3. There are API versions that must be changed and there may be other chages required, depening on the complexity of the chart, but everything on Helm Hub should be compatible.

Helm create

First we will use the helm create command to create our custom chart. I will be using OpenProject as my application of choice for this article. You can use whichever application you like, the catch is that it needs to have a Docker image hosted on a registry.

Be warned, every application has it's own nuances and requirements. If you choose a different application please make sure to read their documentation on deployment requirements.

We will be creating a helm chart for OpenProject a project management software. Let's start by using the helm create command to create the baseline of our project.

helm create openproject-helm-chart

We will see the following files are generated by the helm CLI.

├── charts
├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml

All kubernetes spec files are contained within the templates folder. They are all templated using the Go Template Engine. In this article we will mainly focus on chart.yaml, deployment.yaml, and values.yaml files.

First we will need to modify the values.yaml file to update the Docker image being used for this chart. Here we will also change the service.type to LoadBalancer so we can have external access to our OpenProject instance.

Side node: It is recommended that you keep the service.type as ClusterIP and use ingress to direct traffic to your internal instances. But that is outside that scope of this article. I will do an article in the future to deploy this properly behind an ingress.

Your values.yaml file should look like this:

# Default values for openproject-helm-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

image:
  repository: openproject/community
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "10.6.5"

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

podAnnotations: {}

podSecurityContext: {}
  # fsGroup: 2000

securityContext: {}
  # capabilities:
  #   drop:
  #   - ALL
  # readOnlyRootFilesystem: true
  # runAsNonRoot: true
  # runAsUser: 1000

service:
  type: LoadBalancer
  port: 8080

ingress:
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  hosts:
    - host: chart-example.local
      paths: []
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

resources: {}
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80
  # targetMemoryUtilizationPercentage: 80

nodeSelector: {}

tolerations: []

affinity: {}

And your chart.yaml file should look like this:

apiVersion: v2
name: openproject-helm-chart
description: A Helm chart for Kubernetes

# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: 10.6.5

Shameless plug

If you like content on this blog you can support us by using the DO referral link to sign up and get free credit. And you'll support us as well.

Referral link: https://m.do.co/c/590c0c82c1fc

Also you can subscribe directly signing up for free, or support us with a small monthly or yearly fee. Simply click the button at the bottom right corner.


Adding persistent storage

Now let's add some persistent storage to our chart so we don't lose our data when we update. To do this we will need to update two and add a new one. So first we will add pvc.yaml file in the templates folder. This will define the template for a Persistent Volume Claim. It should look like this:

{{- if and .Values.persistence.enabled (not .Values.persistence.existingClaim) }}
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: {{ template "openproject-helm-chart.fullname" . }}
  labels: {{- include "openproject-helm-chart.labels" . | nindent 4 }}
spec:
  accessModes:
    - {{ .Values.persistence.accessMode | quote }}
  resources:
    requests:
      storage: {{ .Values.persistence.size | quote }}
{{- end }}

Now we will update deployment.yaml and values.yaml files to make use of this new template.

deployment.yaml: Notice the volumes and volumeMounts sections

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "openproject-helm-chart.fullname" . }}
  labels:
    {{- include "openproject-helm-chart.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
{{- end }}
  selector:
    matchLabels:
      {{- include "openproject-helm-chart.selectorLabels" . | nindent 6 }}
  template:
    metadata:
    {{- with .Values.podAnnotations }}
      annotations:
        {{- toYaml . | nindent 8 }}
    {{- end }}
      labels:
        {{- include "openproject-helm-chart.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      serviceAccountName: {{ include "openproject-helm-chart.serviceAccountName" . }}
      securityContext:
        {{- toYaml .Values.podSecurityContext | nindent 8 }}
      containers:
        - name: {{ .Chart.Name }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /
              port: http
          readinessProbe:
            httpGet:
              path: /
              port: http
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
          volumeMounts:
            - mountPath: /var/openproject
              name: openproject-data
      volumes:
        - name: openproject-data
        {{- if .Values.persistence.enabled }}
          persistentVolumeClaim:
            claimName: {{ .Values.persistence.existingClaim | default (include "openproject-helm-chart.fullname" .) }}
        {{- else }}
          emptyDir: {}
        {{ end }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}

values.yaml: Added the persistence section

# Default values for openproject-helm-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

image:
  repository: openproject/community
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "10.6.5"

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

podAnnotations: {}

podSecurityContext: {}
  # fsGroup: 2000

securityContext: {}
  # capabilities:
  #   drop:
  #   - ALL
  # readOnlyRootFilesystem: true
  # runAsNonRoot: true
  # runAsUser: 1000

service:
  type: LoadBalancer
  port: 8080

ingress:
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  hosts:
    - host: chart-example.local
      paths: []
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

resources: {}
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80
  # targetMemoryUtilizationPercentage: 80

nodeSelector: {}

tolerations: []

affinity: {}

persistence:
  enabled: true
  accessMode: ReadWriteOnce
  size: 48Gi

This will add a 48 GB volume to our OpenProject instance. It will be mounted on /var/openproject where all data is stored by OpenProject.

Adding external database

To add an external database to our OpenProject instance we will need to add a dependency to our chart. Plus update the _helpers.tpl with a helper function, update the deployment.yaml and values.yaml files to make use of the external database. First we will update chart.yaml file to add PostgreSQL as a dependency. We will be using the excellent Bitnami chart for PostgreSQL. Your chart.yaml file should look like this:

apiVersion: v2
name: openproject-helm-chart
description: A Helm chart for Kubernetes

# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: 10.6.5

dependencies:
- name: postgresql
  version: "9.1.1"
  repository: "https://charts.bitnami.com/bitnami"

Now we have to update the _helpers.tpl file to add the following section to the file.

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
*/}}
{{- define "openproject-helm-chart.postgresql.fullname" -}}
{{- printf "%s-%s" .Release.Name "postgresql" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

Now we will update the values.yaml file to customize the some of the values provided by the PostgreSQL chart file.

# Default values for openproject-helm-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

image:
  repository: openproject/community
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "10.6.5"

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

podAnnotations: {}

podSecurityContext: {}
  # fsGroup: 2000

securityContext: {}
  # capabilities:
  #   drop:
  #   - ALL
  # readOnlyRootFilesystem: true
  # runAsNonRoot: true
  # runAsUser: 1000

service:
  type: LoadBalancer
  port: 8080

ingress:
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  hosts:
    - host: chart-example.local
      paths: []
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

resources: {}
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80
  # targetMemoryUtilizationPercentage: 80

nodeSelector: {}

tolerations: []

affinity: {}

persistence:
  enabled: true
  accessMode: ReadWriteOnce
  size: 48Gi

postgresql:
  persistence:
    enabled: true
    size: 15G
  postgresqlPassword: superSecretPassword
  postgresqlDatabase: openproject

Finally it's time to update the deployment.yaml file to to make use of these values in an environment variable. We will also take this opportunity to add initialDelaySeconds and periodSeconds to our livenessProbe and readinessProbe. This will allow the application to startup before kubernetes thinks it's failed and restarts it.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "openproject-helm-chart.fullname" . }}
  labels:
    {{- include "openproject-helm-chart.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
{{- end }}
  selector:
    matchLabels:
      {{- include "openproject-helm-chart.selectorLabels" . | nindent 6 }}
  template:
    metadata:
    {{- with .Values.podAnnotations }}
      annotations:
        {{- toYaml . | nindent 8 }}
    {{- end }}
      labels:
        {{- include "openproject-helm-chart.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      serviceAccountName: {{ include "openproject-helm-chart.serviceAccountName" . }}
      securityContext:
        {{- toYaml .Values.podSecurityContext | nindent 8 }}
      containers:
        - name: {{ .Chart.Name }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          env:
          - name: DATABASE_URL
            value: "postgresql://postgres:{{ .Values.postgresql.postgresqlPassword }}@{{ include "openproject-helm-chart.postgresql.fullname" . }}:5432/{{ .Values.postgresql.postgresqlDatabase }}"
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          livenessProbe:
            initialDelaySeconds: 120
            periodSeconds: 120
            httpGet:
              path: /
              port: http
          readinessProbe:
            initialDelaySeconds: 120
            periodSeconds: 120
            httpGet:
              path: /
              port: http
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
          volumeMounts:
            - mountPath: /var/openproject
              name: openproject-data
      volumes:
        - name: openproject-data
        {{- if .Values.persistence.enabled }}
          persistentVolumeClaim:
            claimName: {{ .Values.persistence.existingClaim | default (include "openproject-helm-chart.fullname" .) }}
        {{- else }}
          emptyDir: {}
        {{ end }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}

With the above changes we now a database that is external to our application.

Deploy

With the above changes, we have a working chart for OpenProject. We can deploy the chart with a few simple steps, assuming you have your kubernetes cluster all setup:

  1. helm dependency update
  2. helm install openproject ./openproject-helm-chart

Give it a few minutes and you should see the final product on the IP of the your load balancer. Make sure specify port 8080

OpenProject Home screen