Reset a node

Kairos has a recovery mechanism built-in which can be leveraged to restore the system to a known point. At installation time, the recovery partition is created from the installation medium and can be used to restore the system from scratch, leaving configuration intact and cleaning any persistent data accumulated by usage in the host (e.g. Kubernetes images, persistent volumes, etc. ).

The reset action will regenerate the bootloader configuration and the images in the state partition (labeled COS_STATE) by using the recovery image generated at install time, cleaning up the host.

The configuration files in /oem are kept intact, the node on the next reboot after a reset will perform the same boot sequence (again) of a first-boot installation.

How to

The reset action can be accessed via the Boot menu, remotely, triggered via Kubernetes or manually. In each scenario the machine will reboot into reset mode, perform the cleanup, and reboot automatically afterwards.

From the boot menu

It is possible to reset the state of a node by either booting into the “Reset” mode into the boot menu, which automatically will reset the node:


Remotely, via command line

On a Kairos booted system, logged as root:

$ grub2-editenv /oem/grubenv set next_entry=statereset
$ reboot

From Kubernetes

system-upgrade-controller can be used to apply a plan to the nodes to use Kubernetes to schedule the reset on the nodes itself, similarly on how upgrades are applied.

Consider the following example which resets a machine by changing the config file used during installation:

apiVersion: v1
kind: Secret
  name: custom-script
  namespace: system-upgrade
type: Opaque
  config.yaml: |
    hostname: testcluster-{{ trunc 4 .MachineID }}
      enabled: true
    - name: kairos
      passwd: kairos
      - github:mudler |
    set -e
    if diff /host/run/system-upgrade/secrets/custom-script/config.yaml /host/oem/90_custom.yaml >/dev/null; then
        echo config present
        exit 0
    # we can't cp, that's a symlink!
    cat /host/run/system-upgrade/secrets/custom-script/config.yaml > /host/oem/90_custom.yaml
    grub2-editenv /host/oem/grubenv set next_entry=statereset

    mount --rbind /host/dev /dev
    mount --rbind /host/run /run
    nsenter -i -m -t 1 -- reboot
    exit 1    
kind: Plan
  name: reset-and-reconfig
  namespace: system-upgrade
  concurrency: 2
  # This is the version (tag) of the image.
  version: "opensuse-leap-15.5-standard-amd64-generic-v2.4.3-v1.27.6-k3s1"
      - { key:, operator: Exists }
  serviceAccountName: system-upgrade
  cordon: false
    # Here goes the image which is tied to the flavor being used.
    # Currently can pick between opensuse and alpine
      - "/bin/bash"
      - "-c"
      - bash /host/run/system-upgrade/secrets/custom-script/
    - name: custom-script
      path: /host/run/system-upgrade/secrets/custom-script

Manual reset

It is possible to trigger the reset manually by logging into the recovery from the boot menu and running kairos reset from the console.

Cleaning up state directories

An alternative way and manual of resetting your system is possible by deleting the state paths. You can achieve this by deleting the contents of the /usr/local directory. It’s recommended that you do this while in recovery mode with all services turned off.

Please note that within /usr/local, there are two important folders to keep in mind. The first is /usr/local/.kairos, which contains sentinel files that will trigger a complete deployment from scratch when deleted. However, your data will be preserved. The second folder is /usr/local/.state, which contains the bind-mounted data for the system. By deleting these two folders, you can achieve a pristine environment while leaving all other contents of /usr/local untouched.

Last modified November 5, 2023: refactor (200b0a0)