You are viewing the development docs which are in progress. There is no guarantee that the development documentation will be accurate, including instructions, links, and other information. For the latest stable documentation, click here.

Nvidia Orin NX

Install Kairos on Nvidia Orin NX

This page describes how to install Kairos on an Nvidia Orin NX on the NVMe.

Prerequisites

  • An Nvidia Orin NX board
  • A USB type-C cable
  • A Linux host used to flash the Nvidia Orin NX board
  • Jetson linux SDK download

Flashing

We are going to write the partitions to the NVMe. In order to do this we will use the Nvidia SDK configured with a custom partitioning layout.

The partitions are:

  • OEM for storing cloud config files (/oem)
  • COS_STATE for storing the active/passive images to boot the system
  • EFI for storing the efi shell and grub to boot the system
  • RECOVERY - to store the recovery system
  • PERSISTENT - this is an optional partition to store the persistent data of the system. you can either write this in the NVMe or, for instance, to an external storage. It is enough to create a partition and label it as COS_PERSISTENT. There can be only one partition with such label, the first that matches wins.

Prepare the SDK

The Jetson Linux SDK is used to perform the flashing process.

Download the Jetson Linux SDK. Change the URL according to your desired Linux for Tegra (L4T) version. For L4T 36.4.4 (JetPack 6.2.1):

wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v4.4/release/Jetson_Linux_r36.4.4_aarch64.tbz2 -O tegra.bz2
tar xvf tegra.bz2

Now, we are going to prepare the rootfs and the bootloader. The Jetson Linux SDK requires the rootfs to generate the system.img file and continue the flashing process however, we will not use the image generated by the SDK as we will use a different set of images (see below). Here we also disable extlinux as Kairos uses GRUB:

cd Linux_for_Tegra
# Drop extlinux
echo "" > ./bootloader/extlinux.conf
# This is needed so the SDK doesn't complain of missing files (not really used in the flash process)
IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-orin-nx-master
docker run -ti --rm -v $PWD/rootfs:/rootfs quay.io/luet/base util unpack "$IMAGE" /rootfs
# workaround needed (SDK writes to the symlink)
rm rootfs/boot/initrd
# Extlinux is required by the SDK - so we fake it in our root (it will not be there eventually)
mkdir -p rootfs/boot/extlinux/
echo "" > rootfs/boot/extlinux/extlinux.conf

Prepare the images

You can find Kairos core ubuntu images based on Ubuntu 22.04 here (search for nvidia in the tags)

If you are customizing the image, or either modifying the default partition sizes you can build the images by running:

IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-orin-nx-master
docker run --privileged --platform=linux/arm64 \
        -e container_image=$IMAGE \
        -e STATE_SIZE="25500" \
        -e RECOVERY_SIZE="21000" \
        -e DEFAULT_ACTIVE_SIZE="7000" \
        -v $PWD/bootloader:/bootloader --entrypoint /prepare_nvidia_orin_images.sh -ti --rm quay.io/kairos/auroraboot:v0.13.0

If you have instead the rootfs as a directory, you can create the required partitions with:

ROOTFS=/rootfs/path
docker run --privileged --platform=linux/arm64 \
        -e directory=/rootfs \
        -e STATE_SIZE="25500" \
        -e RECOVERY_SIZE="21000" \
        -e DEFAULT_ACTIVE_SIZE="7000" \
	-v $ROOTFS:/rootfs \
        -v $PWD/bootloader:/bootloader --entrypoint /prepare_nvidia_orin_images.sh -ti --rm quay.io/kairos/auroraboot:v0.13.0

After running any of the commands above, the generated images files required for flashing will be inside the bootloader directory (bootloader/efi.img, bootloader/recovery_partition.img, bootloader/state_partition.img, bootloader/oem.img, bootloader/persistent.img ).

Edit the parition layout

We are going now to modify the partition layout in bootloader/generic/cfg/flash_t234_qspi_nvme.xml which corresponds to the partitioning of the Orin NX board. An example config file can be found in here. Note that the file might change across Nvidia jetson releases, so if flashing fails, use this file as baseline.

wget 'https://kairos.io/examples/board-configs/flash_t234_qspi_nvme.xml' -O ./bootloader/generic/cfg/flash_t234_qspi_nvme.xml

If you are editing the partition sizes and generating the images manually, use the example config file as a baseline and edit the size accordingly to the corresponding partitions (find the respective filename and compare the file size, see the notes below).

Flash

To flash the images to the Orin board

  1. Put the board in recovery mode
  2. Inside the Linux_for_Tegra directory find the Jetson board configuration (file ending with .conf). If it is not there, consult the board provider. An example for the Orin NX can be found here. Set the filename without the extension as an env variable, for example:
BOARD_CONFIG=brla4n-orin-nx
  1. Run:
sudo ./tools/l4t_flash_prerequisites.sh # Install missing dependencies and fix file permissions 

# This step should be performed only if there is no esp.img.
# If this is done without being in recovery-mode the flash process fails later on with a USB timeout error.
sudo ./flash.sh --no-flash --no-systemimg $BOARD_CONFIG mmcblk0p1

# Flash images using NVIDIA's tools for the NVMe option and the custom partition layout and board configuration
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme_kairos.xml -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" --network usb0 $BOARD_CONFIG internal

Booting

The Orin board now should boot. If you are connected over the serial you can login with: kairos/kairos, similarly if you have plugged it to the network you should be able to SSH in as well.

Notes

USB Timeout error

It is possible that during flashing on certain kernel versions to see an error message:

[ 0.3623 ] tegrarcm_v2 --new_session --chip 0x23 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[ 0.3630 ] BR_CID: 0x80012344705DD25D1C00000019028240
[ 0.3932 ] Sending bct_br
[ 0.4409 ] ERROR: might be timeout in USB write.
[ 5.5325 ]

See also the relevant Nvidia discussions in the forum:

The solution here is trying with a different kernel version, as suggested in the Nvidia threads.

Default configuration

To customize the default cloud config of the board, generate the images mounting the cloud config you want in the images in /defaults.yaml:

IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-orin-nx-master
CLOUD_CONFIG=/cloud/config.yaml
docker run -v $CLOUD_CONFIG:/defaults.yaml --privileged \
        -e container_image=$IMAGE \
        -e STATE_SIZE="25500" \
        -e RECOVERY_SIZE="21000" \
        -e DEFAULT_ACTIVE_SIZE="7000" \
        -v $PWD/bootloader:/bootloader --entrypoint /prepare_nvidia_orin_images.sh -ti --rm quay.io/kairos/auroraboot:v0.13.0

Debugging

Use the micro USB as debug serial port with minicom to debug any booting issues.

sudo minicom -D /dev/ttyACM0 -8 -b 115200

Flashing port

In order to flash the Nvidia AGX Orin you will need to use the USB Type-C ports. The Micro USB port is reserved only for debugging over the serial console.