Deploying Github actions runners to Kubernetes

Updated 1 year ago Published 1 year ago

Managed runners for Github actions can be quite expensive as such for most folk and organisations it's likely cheaper to host the runners of your own infrastructure and given this is 2023 your probably running kubernetes somewhere.

This guide is up-to-date as of November 4th 2023.

This guide will cover how you can deploy Github runners to a Kubernetes cluster at the organisational level.

To start you will need to generate a new token, any user that has been granted admin rights to your organisation should have the permissions required to create a new token.

There are still a few issues with the new Fine grained tokens for Organisation level permissions so we will create a classic PAT token.

Settings > Developer Settings > Personal access tokens > Tokens (classic)

Now you will need to generate a new token

Generate new token > Generate new token (classic)

When you generate your new token you will need to ensure that all of the required scopes are added.

The required scopes are

repo
admin:org

You will also need to select an expiration for you new token, I would recommend something long lived as you will probably forget to rotate Kubernertes secrets.

Once you have created your new token you can simply install the runners via the helm chart listed below.

Installing with Helm

Creating Namespace for ARC

kubectl create ns actions-runner-system

Creating secret for ARC, so that it can register runners on Github, You will replace {TOKEN} with the token you have just generated.

kubectl create secret generic controller-manager -n actions-runner-system --from-literal=github_token={TOKEN}

Adding Helm Repo of ARC

helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller

Updating Helm Repo

helm repo update

Any now you can install ARC

helm upgrade --install --namespace actions-runner-system \

--create-namespace --wait actions-runner-controller \

actions-runner-controller/actions-runner-controller \

--set syncPeriod=1m

Now you will need to deploy a new RunnerDeployment resource, This resource defines the repositories and organisations that can access your runners.

Given the focus of this tutorial is how to deploy the runners for an organisation the below configuration will allow any repository within an organisation to bind to the runners

apiVersion: actions.summerwind.dev/v1alpha1

kind: RunnerDeployment

metadata:

  name: k8s-action-runner

  namespace: actions-runner-system

spec:

  replicas: 1

  template:

    spec:

      labels:

        - actions_runner_dev

        - actions_runner

        - default

        - self-hosted

      organization: {ORG_NAME}

And once you run all of the above command you should check that all of the resources deployed to the new namespace have been properly deployed.

kubectl --namespace actions-runner-system get all

NAME                                             READY   STATUS    RESTARTS   AGE
pod/actions-runner-controller-78c95847b9-dngl5   2/2     Running   0          36m
pod/k8s-action-runner-kjxw9-ddvgz                2/2     Running   0          30s
pod/k8s-action-runner-kjxw9-hppg9                2/2     Running   0          31s
pod/k8s-action-runner-kjxw9-4l85r                2/2     Running   0          32s
pod/k8s-action-runner-kjxw9-dxhps                2/2     Running   0          31s
pod/k8s-action-runner-kjxw9-rkbkm                2/2     Running   0          32s
pod/k8s-action-runner-kjxw9-hwljg                2/2     Running   0          30s
pod/k8s-action-runner-kjxw9-chcwb                2/2     Running   0          30s
pod/k8s-action-runner-kjxw9-jr5pk                2/2     Running   0          30s
pod/k8s-action-runner-kjxw9-m55gg                2/2     Running   0          32s
pod/k8s-action-runner-kjxw9-cl485                2/2     Running   0          32s

NAME                                                TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/actions-runner-controller-metrics-service   ClusterIP   10.43.230.96   <none>        8443/TCP   54m
service/actions-runner-controller-webhook           ClusterIP   10.43.245.32   <none>        443/TCP    54m

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/actions-runner-controller   1/1     1            1           54m

NAME                                                   DESIRED   CURRENT   READY   AGE
replicaset.apps/actions-runner-controller-8c9d89578    0         0         0       54m
replicaset.apps/actions-runner-controller-55f6876bc    0         0         0       40m
replicaset.apps/actions-runner-controller-78c95847b9   1         1         1       36m
replicaset.apps/actions-runner-controller-8657fcd84    0         0         0       38m