Nvidia AGX Orin

Warning

Nvidia AGX Orin currently only works with Ubuntu 22.04-based images.

This page describes how to install Kairos on Nvidia AGX Orin in the eMMC.

Building your own AGX Orin image

The image tags referenced below are examples. To build a fresh Kairos image for AGX Orin from Ubuntu 22.04, use kairos-init --model nvidia-jetson-agx-orin. That guide replaces the old Dockerfile.nvidia.

Prerequisites

• Nvidia AGX Orin
• An USB type-C cable
• A Linux host used to flash the Nvidia AGX Orin board
• Jetson linux SDK download

You can find debugging information here: https://developer.ridgerun.com/wiki/index.php/NVIDIA_Jetson_Orin/In_Board/Getting_in_Board/Serial_Console

Flashing

We are going to write the partitions in the eMMC. In order to do this we will use the Nvidia SDK configured with a custom partitioning layout.

The partitions are:

• OEM for storing cloud config files (/oem)
• COS_STATE for storing the active/passive images to boot the system
• EFI for storing the efi shell and grub to boot the system
• RECOVERY - to store the recovery system
• PERSISTENT - this is an optional partition to store the persistent data of the system. you can either write this in the eMMC or, for instance, to an external storage. It is enough to create a partition and label it as COS_PERSISTENT. There can be only one partition with such label, the first that matches wins.

Prepare the SDK

The Jetson Linux SDK is used to perform the flashing process.

Download the Jetson Linux SDK:

wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v4.3/release/Jetson_Linux_r36.4.3_aarch64.tbz2 -O tegra.bz2
tar xvf tegra.bz2

Now, we are going to prepare the rootfs and the bootloader. The Jetson Linux SDK requires the rootfs to generate the system.img file and continue the flashing process however, we will not use the image generated by the SDK as we will use a different set of images (see below). Here we also disable extlinux as Kairos uses GRUB:

cd Linux_for_Tegra
# Drop extlinux
echo "" > ./bootloader/extlinux.conf
# This is needed so the SDK doesn't complain of missing files (not really used in the flash process)
IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-agx-orin-v4.1.2
docker run -ti --rm -v $PWD/rootfs:/rootfs quay.io/luet/base util unpack "$IMAGE" /rootfs --platform linux/arm64
# workaround needed (SDK writes to the symlink)
rm rootfs/boot/initrd
# Extlinux is required by the SDK - so we fake it in our root (it will not be there eventually)
mkdir -p rootfs/boot/extlinux/
echo "" > rootfs/boot/extlinux/extlinux.conf

Prepare the images

Legacy flavor example

The Ubuntu image tags in this page are valid examples for AGX Orin workflows, but these non-Hadron flavor repositories are not actively updated by the Kairos release pipeline anymore. Build and publish your own image with BYOI and Kairos Factory.

You can find Kairos core ubuntu images based on Ubuntu 22.04 here: https://quay.io/repository/kairos/ubuntu (search for nvidia in the tags)

AuroraBoot can emit the EFI, OEM, and recovery files directly. The following command uses the same raw-disk path as other Kairos targets, but stops before merging the partitions into a single disk:

IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-agx-orin-v4.1.2
docker run --rm --privileged --platform=linux/arm64 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$PWD/bootloader:/output" \
  quay.io/kairos/auroraboot:v0.24.0 \
  --set "arch=arm64" \
  --set "container_image=$IMAGE" \
  --set "state_dir=/output" \
  --set "disable_http_server=true" \
  --set "disable_netboot=true" \
  --set "disk.partitions=true"

The generated files are bootloader/efi.img, bootloader/oem.img, and bootloader/recovery_partition.img. On the first boot, Kairos creates COS_STATE and COS_PERSISTENT from the remaining disk space through the standard reset flow. This is the tested AGX Orin flow and avoids flashing large, mostly empty state and persistent images.

Filesystem choice

The reset flow creates the Kairos data partitions as ext2. This is intentional: the immutable system does not benefit from journaling, and ext2 keeps the existing, tested partition expansion behavior.

Edit the partition layout

We are going now to modify the partition layout in bootloader/generic/cfg/flash_t234_qspi_sdmmc.xml which corresponds to the partitioning of the AGX Orin board. An example config file can be found in here. Note that the file might change across Nvidia jetson releases, so if flashing fails, use this file as baseline.

wget 'https://kairos.io/examples/board-configs/flash_t234_qspi_sdmmc.xml' -O ./bootloader/generic/cfg/flash_t234_qspi_sdmmc.xml

If you are editing the partition sizes and generating the images manually, use the example config file as a baseline and edit the size accordingly to the corresponding partitions (find the respective filename and compare the file size, see the notes below).

Note on editing the partition layout manually

If you want to use the original file, identify the sdmmc_user section ( e.g. <device type="sdmmc_user" instance="3" sector_size="512" num_sectors="INT_NUM_SECTORS" > ), inside there is an "APP" partition ( <partition name="APP" id="1" type="data"> ), remove it , and add the following instead:

       <partition name="COS_RECOVERY" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 10485760000 </size>
            <allocation_attribute>  0x8 </allocation_attribute>
            <filename> recovery_partition.img </filename>
            <description>  </description>
        </partition>
        <partition name="COS_OEM" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 67108864 </size>
            <allocation_attribute>  0x8 </allocation_attribute>
            <filename> oem.img </filename>
            <description>  </description>
        </partition>

Be mindful also to change the esp partition or add it if required:

     <partition name="esp" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
	    <size> 20971520 </size>
	    <file_system_attribute> 0 </file_system_attribute>
	    <partition_type_guid> C12A7328-F81F-11D2-BA4B-00A0C93EC93B </partition_type_guid>
	    <allocation_attribute> 0x8 </allocation_attribute>
	    <percent_reserved> 0 </percent_reserved>
            <filename> efi.img </filename>
            <description> **Required.** Contains a redundant copy of CBoot. </description>
        </partition>

You can also remove the other partitions under sdmmc_user as not effectively used by Kairos during boot.

Note

If modifying the partition sizes, replace the size inside the <size></size> tags of each partition in the XML:

stat -c %s bootloader/efi.img
stat -c %s bootloader/recovery_partition.img
stat -c %s bootloader/oem.img

Flash

To flash the images to the Orin board

1. Put the board in recovery mode
2. Run:

sudo ./tools/l4t_flash_prerequisites.sh # Install missing dependencies and fix file permissions 
sudo ./flash.sh jetson-agx-orin-devkit internal

Booting

The Orin board now should boot. If you are connected over the serial you can login with: kairos/kairos, similarly if you have plugged it to the network you should be able to SSH in as well.

Notes

USB Timeout error

It is possible that during flashing on certain kernel versions to see an error message:

[ 0.3623 ] tegrarcm_v2 --new_session --chip 0x23 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[ 0.3630 ] BR_CID: 0x80012344705DD25D1C00000019028240
[ 0.3932 ] Sending bct_br
[ 0.4409 ] ERROR: might be timeout in USB write.
[ 5.5325 ]

See also the relevant Nvidia discussions in the forum:

• https://forums.developer.nvidia.com/t/usb-timeout-when-flashing-agx-orin/235600
• https://forums.developer.nvidia.com/t/cannot-flash-jetson-os-image-to-jetson-agx-orin-devkit-via-sdk-manager/219489/2

The solution here is trying with a different kernel version, as suggested in the Nvidia threads.

Default configuration

To customize the default cloud config of the board, generate the images mounting the cloud config you want in the images in /defaults.yaml:

IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-agx-orin-v4.1.2
CLOUD_CONFIG=/cloud/config.yaml
docker run --rm --privileged --platform=linux/arm64 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$CLOUD_CONFIG:/config.yaml:ro" \
  -v "$PWD/bootloader:/output" \
  quay.io/kairos/auroraboot:v0.24.0 \
  --set "arch=arm64" \
  --set "container_image=$IMAGE" \
  --set "state_dir=/output" \
  --set "disable_http_server=true" \
  --set "disable_netboot=true" \
  --set "disk.partitions=true" \
  --cloud-config /config.yaml

Debugging

Use the micro USB as debug serial port with minicom to debug any booting issues.

sudo minicom -D /dev/ttyACM0 -8 -b 115200

Flashing port

In order to flash the Nvidia AGX Orin you will need to use the USB Type-C ports. The Micro USB port is reserved only for debugging over the serial console.