You are viewing the development docs which are in progress. There is no guarantee that the development documentation will be accurate, including instructions, links, and other information. For the latest stable documentation, click here.
Nvidia Orin NX
Warning
The Ubuntu versions supported on the Orin NX depend on the JetPack release. Check the compatibility matrix here.This page describes how to install Kairos on an Nvidia Orin NX on the NVMe.
Prerequisites
- An Nvidia Orin NX board
- A USB type-C cable
- A Linux host used to flash the Nvidia Orin NX board
- Jetson linux SDK download
Debugging
You can find debugging information hereFlashing
We are going to write the partitions to the NVMe. In order to do this we will use the Nvidia SDK configured with a custom partitioning layout.
The partitions are:
- OEM for storing cloud config files (
/oem) - COS_STATE for storing the active/passive images to boot the system
- EFI for storing the efi shell and grub to boot the system
- RECOVERY - to store the recovery system
- PERSISTENT - this is an optional partition to store the persistent data of the system. you can either write this in the NVMe or, for instance, to an external storage. It is enough to create a partition and label it as
COS_PERSISTENT. There can be only one partition with such label, the first that matches wins.
Prepare the SDK
The Jetson Linux SDK is used to perform the flashing process.
Download the Jetson Linux SDK. Change the URL according to your desired Linux for Tegra (L4T) version. For L4T 36.4.4 (JetPack 6.2.1):
wget https://developer.nvidia.com/downloads/embedded/l4t/r36_release_v4.4/release/Jetson_Linux_r36.4.4_aarch64.tbz2 -O tegra.bz2
tar xvf tegra.bz2
Now, we are going to prepare the rootfs and the bootloader. The Jetson Linux SDK requires the rootfs to generate the system.img file and continue the flashing process however, we will not use the image generated by the SDK as we will use a different set of images (see below). Here we also disable extlinux as Kairos uses GRUB:
cd Linux_for_Tegra
# Drop extlinux
echo "" > ./bootloader/extlinux.conf
# This is needed so the SDK doesn't complain of missing files (not really used in the flash process)
IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-orin-nx-master
docker run -ti --rm -v $PWD/rootfs:/rootfs quay.io/luet/base util unpack "$IMAGE" /rootfs
# workaround needed (SDK writes to the symlink)
rm rootfs/boot/initrd
# Extlinux is required by the SDK - so we fake it in our root (it will not be there eventually)
mkdir -p rootfs/boot/extlinux/
echo "" > rootfs/boot/extlinux/extlinux.conf
Prepare the images
You can find Kairos core ubuntu images based on Ubuntu 22.04 here
(search for nvidia in the tags)
If you are customizing the image, or either modifying the default partition sizes you can build the images by running:
IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-orin-nx-master
docker run --privileged --platform=linux/arm64 \
-e container_image=$IMAGE \
-e STATE_SIZE="25500" \
-e RECOVERY_SIZE="21000" \
-e DEFAULT_ACTIVE_SIZE="7000" \
-v $PWD/bootloader:/bootloader --entrypoint /prepare_nvidia_orin_images.sh -ti --rm quay.io/kairos/auroraboot:v0.13.0
If you have instead the rootfs as a directory, you can create the required partitions with:
ROOTFS=/rootfs/path
docker run --privileged --platform=linux/arm64 \
-e directory=/rootfs \
-e STATE_SIZE="25500" \
-e RECOVERY_SIZE="21000" \
-e DEFAULT_ACTIVE_SIZE="7000" \
-v $ROOTFS:/rootfs \
-v $PWD/bootloader:/bootloader --entrypoint /prepare_nvidia_orin_images.sh -ti --rm quay.io/kairos/auroraboot:v0.13.0
After running any of the commands above, the generated images files required for flashing will be inside the bootloader directory (bootloader/efi.img, bootloader/recovery_partition.img, bootloader/state_partition.img, bootloader/oem.img, bootloader/persistent.img ).
Note
The persistent image is optional, as you can store the system persistent data rather in an SD card or an NVME disk. The default persistent.img is of 2GB size. To create a persistent image manually of the size you prefer instead you can run:
# Create a 2GB filesystem for COS_PERSISTENT volume
truncate -s $((2048*1024*1024)) bootloader/persistent.img
mkfs.ext2 -L "COS_PERSISTENT" bootloader/persistent.img
Note that the size of the partitions you modify should be duly reported in the partition layout (see below).
Edit the parition layout
We are going now to modify the partition layout in bootloader/generic/cfg/flash_t234_qspi_nvme.xml which corresponds to the partitioning of the Orin NX board. An example config file can be found in here. Note that the file might change across Nvidia jetson releases, so if flashing fails, use this file as baseline.
wget 'https://kairos.io/examples/board-configs/flash_t234_qspi_nvme.xml' -O ./bootloader/generic/cfg/flash_t234_qspi_nvme.xml
If you are editing the partition sizes and generating the images manually, use the example config file as a baseline and edit the size accordingly to the corresponding partitions (find the respective filename and compare the file size, see the notes below).
Note on editing the partition layout manually
If you want to use the original file, identify the nvme section ( e.g. <device type="nvme" instance="4" sector_size="512" num_sectors="INT_NUM_SECTORS" > ), inside there is an “APP” partition ( <partition name="APP" id="1" type="data"> ), remove it , and add the following instead:
<partition name="COS_RECOVERY" type="data">
<allocation_policy> sequential </allocation_policy>
<filesystem_type> basic </filesystem_type>
<size> @RECOVERY_SIZE@ </size>
<allocation_attribute> 0x8 </allocation_attribute>
<filename> recovery_partition.img </filename>
<description> </description>
</partition>
<partition name="COS_STATE" type="data">
<allocation_policy> sequential </allocation_policy>
<filesystem_type> basic </filesystem_type>
<size> @STATE_SIZE@ </size>
<allocation_attribute> 0x8 </allocation_attribute>
<filename> state_partition.img </filename>
<description> </description>
</partition>
<partition name="COS_OEM" type="data">
<allocation_policy> sequential </allocation_policy>
<filesystem_type> basic </filesystem_type>
<size> @OEM_SIZE@ </size>
<allocation_attribute> 0x8 </allocation_attribute>
<filename> oem.img </filename>
<description> </description>
</partition>
<partition name="COS_PERSISTENT" type="data">
<allocation_policy> sequential </allocation_policy>
<filesystem_type> basic </filesystem_type>
<size> @PERSISTENT_SIZE@ </size>
<allocation_attribute> 0x8 </allocation_attribute>
<filename> persistent.img </filename>
<description> </description>
</partition>
Be mindful also to change the esp partition or add it if required:
<partition name="esp" type="data">
<allocation_policy> sequential </allocation_policy>
<filesystem_type> basic </filesystem_type>
<size> @ESP_SIZE@ </size>
<file_system_attribute> 0 </file_system_attribute>
<partition_type_guid> C12A7328-F81F-11D2-BA4B-00A0C93EC93B </partition_type_guid>
<allocation_attribute> 0x8 </allocation_attribute>
<percent_reserved> 0 </percent_reserved>
<filename> efi.img </filename>
<description> **Required.** Contains a redundant copy of CBoot. </description>
</partition>
Note
If modifying the partition sizes, you need to replace the size inside the <size></size> tags of each partition in the XML:
stat -c %s bootloader/efi.img
stat -c %s bootloader/recovery_partition.img
stat -c %s bootloader/state_partition.img
stat -c %s bootloader/oem.img
stat -c %s bootloader/persistent.img
If you want to further automate this you can do from the Linux_for_Tegra directory:
wget 'https://kairos.io/examples/board-configs/flash_l4t_t234_nvme_kairos.tmpl' -O flash_l4t_t234_nvme_kairos.tmpl
cp flash_l4t_t234_nvme_kairos.tmpl tools/kernel_flash/flash_l4t_t234_nvme_kairos.xml
EFI_IMG_SIZE=`stat -c %s bootloader/efi.img`
RECOVERY_IMG_SIZE=`stat -c %s bootloader/recovery_partition.img`
STATE_IMG_SIZE=`stat -c %s bootloader/state_partition.img`
OEM_IMG_SIZE=`stat -c %s bootloader/oem.img`
PERSISTENT_IMG_SIZE=`stat -c %s bootloader/persistent.img`
sed -e "s/@ESP_SIZE@/${EFI_IMG_SIZE}/" \
-e "s/@RECOVERY_SIZE@/${RECOVERY_IMG_SIZE}/" \
-e "s/@STATE_SIZE@/${STATE_IMG_SIZE}/" \
-e "s/@OEM_SIZE@/${OEM_IMG_SIZE}/" \
-e "s/@PERSISTENT_SIZE@/${PERSISTENT_IMG_SIZE}/" \
tools/kernel_flash/flash_l4t_t234_nvme_kairos.xml
Flash
To flash the images to the Orin board
- Put the board in recovery mode
- Inside the Linux_for_Tegra directory find the Jetson board configuration (file ending with .conf). If it is not there, consult the board provider. An example for the Orin NX can be found here. Set the filename without the extension as an env variable, for example:
BOARD_CONFIG=brla4n-orin-nx
- Run:
sudo ./tools/l4t_flash_prerequisites.sh # Install missing dependencies and fix file permissions
# This step should be performed only if there is no esp.img.
# If this is done without being in recovery-mode the flash process fails later on with a USB timeout error.
sudo ./flash.sh --no-flash --no-systemimg $BOARD_CONFIG mmcblk0p1
# Flash images using NVIDIA's tools for the NVMe option and the custom partition layout and board configuration
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_t234_nvme_kairos.xml -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" --network usb0 $BOARD_CONFIG internal
Booting
The Orin board now should boot. If you are connected over the serial you can login with: kairos/kairos, similarly if you have plugged it to the network you should be able to SSH in as well.
Notes
USB Timeout error
It is possible that during flashing on certain kernel versions to see an error message:
[ 0.3623 ] tegrarcm_v2 --new_session --chip 0x23 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[ 0.3630 ] BR_CID: 0x80012344705DD25D1C00000019028240
[ 0.3932 ] Sending bct_br
[ 0.4409 ] ERROR: might be timeout in USB write.
[ 5.5325 ]
See also the relevant Nvidia discussions in the forum:
- https://forums.developer.nvidia.com/t/usb-timeout-when-flashing-agx-orin/235600
- https://forums.developer.nvidia.com/t/cannot-flash-jetson-os-image-to-jetson-agx-orin-devkit-via-sdk-manager/219489/2
The solution here is trying with a different kernel version, as suggested in the Nvidia threads.
Default configuration
To customize the default cloud config of the board, generate the images mounting the cloud config you want in the images in /defaults.yaml:
IMAGE=quay.io/kairos/ubuntu:22.04-core-arm64-nvidia-jetson-orin-nx-master
CLOUD_CONFIG=/cloud/config.yaml
docker run -v $CLOUD_CONFIG:/defaults.yaml --privileged \
-e container_image=$IMAGE \
-e STATE_SIZE="25500" \
-e RECOVERY_SIZE="21000" \
-e DEFAULT_ACTIVE_SIZE="7000" \
-v $PWD/bootloader:/bootloader --entrypoint /prepare_nvidia_orin_images.sh -ti --rm quay.io/kairos/auroraboot:v0.13.0
Debugging
Use the micro USB as debug serial port with minicom to debug any booting issues.
sudo minicom -D /dev/ttyACM0 -8 -b 115200
Flashing port
In order to flash the Nvidia AGX Orin you will need to use the USB Type-C ports. The Micro USB port is reserved only for debugging over the serial console.