Booting Kairos on Nvidia Jetson ARM

This page contains a reference on how to run Kairos on Nvidia Jetson ARM

This page is a development reference in order to boot Kairos in Nvidia Jetson devices. Nvidia Jetson images by default ship extlinux as bootloader, without EFI boot. This guide explains how to get instead u-boot to chainload to grub2, which can be used to boot and load Kairos.

Note that currently there are no official Kairos core images for Jetson images, this page will refer to Jetson Nano eMMC version as the current reference, but the steps should be similar, as outline how to use the Nvidia SDK to flash the OS onboard in the eMMC of the device.

The steps involved are:

  • Prepare the Kernel (if you have one, compatible with EFISTUB, you can skip this part)
  • Flash u-boot (If the U-boot version support booting efi shells, you might skip this part too)
  • Prepare the Kairos partitions
  • Flash the image to the board


You need the Nvidia SDK and few other dependencies in the system. Note that for the Jetson Nano you can’t use the latest SDK version as it is not anymore supporting it. The latest version available with support for Jetson Nano is r32.7.3:

# Build dependencies
apt update && apt install -y git-core build-essential bc wget xxd kmod flex libelf-dev bison libssl-dev

mkdir build
cd build

# Get Jetson SDK compatible with Jetson NANO

wget -O Jetson-210_Linux_R32.7.3_aarch64.tbz2
tar xvf Jetson-210_Linux_R32.7.3_aarch64.tbz2

Prepare the Kernel

The only requirement of the kernel in order to this to work is that has to have CONFIG_EFI_STUB and CONFIG_EFI enabled.

The default kernel with the Nvidia Jetson Nano is 4.9 and it turns out to not have those enabled.

Build from official Nvidia sources

If your kernel is not compiled to boot as EFI stub you can refer to the steps below to compile the official Nvidia kernel with EFISTUB:

cd build
wget -O public_sources.tbz2
tar xvf
# gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/....
export CROSS_COMPILE_AARCH64_PATH=$PWD/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/

cd Linux_for_Tegra/source/public
tar xvf kernel_src.bz2
mkdir kernel_out
echo "CONFIG_EFI_STUB=y" >> ./kernel/kernel-4.9/arch/arm64/configs/tegra_defconfig
echo "CONFIG_EFI=y" >> ./kernel/kernel-4.9/arch/arm64/configs/tegra_defconfig

sed -i '86s/.*/ O_OPT=(O="${KERNEL_OUT_DIR}")/'
## See workaround for DTB errors in Troubleshooting (edit Kconfig.include..)
./ -o $PWD/kernel_out

Note that, with the Jetson NANO, the kernel will fail to boot allocating the memory during the EFI stub boot phase.

Build from official linux kernel

Seems the kernel 5.15 boots fine on the Jetson Nano, however, it fails to load eMMC drivers to detect eMMC partitions. A configuration reference can be found here.

cd build

# Clone the kernel
git clone --branch v5.15 --depth 1 kernel-4.9

wget -O public_sources.tbz2
tar xvf public_sources.tbz2
tar xvf l4t-gcc-7-3-1-toolchain-64-bit

# Replace the kernel in the SDK
pushd Linux_for_Tegra/source/public && tar xvf kernel_src.tbz2 && rm -rf kernel/kernel-4.9 && mv $build_dir/kernel-4.9 ./kernel/ && popd

# Use the tegra config, patch
mkdir kernel_out && \
wget -O ./kernel/kernel-4.9/arch/arm64/configs/defconfig && \
wget -O && chmod +x

# gcc 12 patches
pushd Linux_for_Tegra/source/public/kernel/kernel-4.9 && curl -L | patch -p1 && popd

# Build the kernel
pushd Linux_for_Tegra/source/public && \
   CROSS_COMPILE_AARCH64_PATH=$build_dir/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/ ./ -o $PWD/kernel_out

Prepare container image (Kairos)

Now we need a container image with the OS image. The image need to contain the kernel and the initramfs generated with dracut.

For instance, given that the kernel is available at /boot/Image, and the modules at /lib/modules:

FROM ....

RUN ln -sf Image /boot/vmlinuz
RUN kernel=$(ls /lib/modules | head -n1) && \
    dracut -f "/boot/initrd-${kernel}" "${kernel}" && \
    ln -sf "initrd-${kernel}" /boot/initrd && \
    depmod -a "${kernel}"


In order to flash to the eMMC we need the Nvidia SDK.

mkdir work
cd work
tar xvf Jetson-210_Linux_R32.7.3_aarch64.tbz2

Replace U-boot (optional)

If the version of u-boot is old and doesn’t support EFI booting, you can replace the u-boot binary like so:

mkdir u-boot
cd u-boot
rpm2cpio ../u-boot-p3450-0000-2023.01-2.1.aarch64.rpm | cpio -idmv
cd ..
cd Linux_for_Tegra
# "p3450-0000" Depends on your board
cp -rfv ../u-boot/boot/u-boot.bin bootloader/t210ref/p3450-0000/u-boot.bin

Disable Extlinux

We need to disable extlinux, in order for u-boot to scan for EFI shells:

# Drop extlinux
echo "" > ./bootloader/extlinux.conf

Prepare Partitions

We need to prepare the partitions from the container image we want to boot, in order to achieve this, we can use osbuilder, which will prepare the img files ready to be flashed for the SDK:

cd Linux_for_Tegra
docker run --privileged -e container_image=$IMAGE -v $PWD/bootloader:/bootloader --entrypoint / -ti --rm

This command should create efi.img, oem.img, persistent.img, recovery_partition.img, state_partition.img in the bootloader directory

Configure the SDK

In order to flash the partitions to the eMMC of the board, we need to configure the SDK to write the partitions to the board via its configuration files.

For the Jetson Nano, the configuration file for the partitions is located at bootloader/t210ref/cfg/flash_l4t_t210_emmc_p3448.xml, where we replace the partition name=APP with:

 <partition name="esp" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
	    <size> 20971520 </size>
	    <file_system_attribute> 0 </file_system_attribute>
	    <partition_type_guid> C12A7328-F81F-11D2-BA4B-00A0C93EC93B </partition_type_guid>
	    <allocation_attribute> 0x8 </allocation_attribute>
	    <percent_reserved> 0 </percent_reserved>
            <filename> efi.img </filename>
            <description> **Required.** Contains a redundant copy of CBoot. </description>
       <partition name="COS_RECOVERY" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 2298478592 </size>
            <allocation_attribute>  0x8 </allocation_attribute>
            <filename> recovery_partition.img </filename>
            <description>  </description>
        <partition name="COS_STATE" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 5234491392 </size>
            <allocation_attribute>  0x8 </allocation_attribute>
            <filename> state_partition.img </filename>
            <description>  </description>
        <partition name="COS_OEM" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 67108864 </size>
            <allocation_attribute>  0x8 </allocation_attribute>
            <filename> oem.img </filename>
            <description>  </description>
        <partition name="COS_PERSISTENT" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 2147483648 </size>
            <allocation_attribute>  0x8 </allocation_attribute>
            <filename> persistent.img </filename>
            <description>  </description>

Note: The order matters here. We want to replace the default “APP” partition with our set of partitions.

If you didn’t changed the default size of the images you should be fine, however, you should check the <size></size> of each of the blocks if corresponds to the files generated from your container image:

stat -c %s bootloader/efi.img
stat -c %s bootloader/recovery_partition.img
stat -c %s bootloader/state_partition.img
stat -c %s bootloader/oem.img
stat -c %s bootloader/persistent.img


Turn the board in recovery mode, depending on the model this process might differ:

  • Turn off the board
  • Jump the FCC REC pin to ground
  • Plug the USB cable
  • Power on the board

If you see the board ready to be flashed, you should see the following:

$ lsusb
Bus 003 Device 092: ID 0955:7f21 NVIDIA Corp. APX

To flash the configuration to the board, run:

./ -r jetson-nano-devkit-emmc mmcblk0p1

Troubleshooting notes

You can use picom to see the serial console:

picocom -b 115200 /dev/ttyUSB0


Last modified February 23, 2024: Reduce sizes and remove warnings (0e183ae)