Single Image Boot

This document describes what is a Single Image Boot and the updates done to support it.

Introduction

The main objective of this new boot and build flow is to create a single boot image which is able to boot in a set of different boards. The boards may differ in areas such as the boot memory device (SD card, NAND, QSPI). This image then includes all the components needed to boot from u-boot SPL to Linux determining automatically what components need to be bring up for this. To accomplish this objective, 2 fundamental features are needed:

• Identify the board in which the u-boot software is currently running over. This is needed to determine what configuration is needed for this board to bring up any device that is specific for this board.

• Ability to determine the configuration needed for a specific board. For this, u-boot supports a feature called FIT image (Flattened Image Tree). A FIT image is created from a .its (Image Tree Source) file which has a similar structure than a regular device tree (the FIT image also could be generated from a .dtsi file). The .its file describes how the FIT image (.itb) is conformed defining the software pieces that will be part of the image. Unlike a regular device tree, the .its file also defines something called Configurations. Each configuration defines the set of software items that will be used to launch a software item in a subsequent boot stage (for example u-boot SPL launches u-boot or u-boot launches Linux). The following figure shows an example of a .its file that defines 2 Linux kernels, 2 devices trees and 2 configurations. The configuration 1 indicates that the kernel 1 will be launched by u-boot along with the device tree 1, while the configuration 2 indicates that the kernel 2 will be launch along the device tree 2.

In our specific case, there is a configuration specific for the type of board in which the software is running. In this way several device trees are included in the FIT image, and by the time u-boot or Linux is launched, the software could determine which of the device trees would be the appropriate for each board.

A FIT image could be created for u-boot and for Linux. In the case of u-boot FIT image, u-boot SPL is the one that analyzes the configurations to determine what items should be loaded. In our flow the items to load are u-boot raw image, ATF (Arm Trusted Firmware) and u-boot device tree.

In the case of a Linux FIT image, the u-boot is the one that analyzes the configurations to determine which are the items to load. In our flow, the items to load correspond to the Linux image, the device tree and the FPGA configuration.

The load of the FPGA Configuration makes this new boot flow compatible and specific to the HPS Boot First mode flow being the u-boot the one that performs the FPGA configuration. The FPGA configuration image could also be specific to a board, and this is the reason after including the FPGA configuration the Linux FIT image.

The following sections provides some implementations details that were required to support the Single Image Boot. The information shown here applies exclusively for the Stratix10 and Agilex devices.

u-boot Single Image Boot Support

There were several changes needed to support the Single Image Boot. The following section describes the most important changes that allow to understand with more detail how the new build and execution flows work

Board ID Implementation

As indicated earlier, having a single boot image implies to determine in run time what software components are needed to initialize based on the board in which the software is running. The identification of the board is based on the implementation of a register in the GHRD (in the FPGA fabric) that contains the board_ID for this board (this ID is unique for each type of board). Starting from GHRD 21.4, the board_ID can be defined manually from .qsf file using the STRATIX_JTAG_USER_CODE parameter during the hardware build.

u-boot SPL and u-boot obtains the board_ID using socfpga_get_board_id() function in ./arch/arm/mach-socfpga/board.c. The request is done through the HPS-SDM mailbox. The board ID must be a value between 0 and 255.

u-boot SPL uses the board_ID to determine the configuration that identifies the boot images of the SW components that will be used when launching u-boot (ATF, u-boot and FDT). This is done at spl_load_simple_fit() function – in common/spl/spl_fit.c under the call of spl_simple_fit_parse().

u-boot obtains the board_ID and sets this in the ${board_id} environment variable. This is done in arch_misc_init() function at arch/arm/mach-socfpga/misc_soc64.c. The u-boot uses this variable to launch Linux using the command:

bootm ${loadaddr}#board-${board_id};

Notes:
* Board 0 is expected to be used in Simics simulation and for this it's not expected to have FGPA fabric configuration.

Combined U-Boot DefConfig

The defconfig files are used to determine if certain feature or device needs to be compiled and included to the u-boot image (u-boot-nodtb.bin). Previously, there were several defconfig that could be used depending on what it was needed to include. The different versions that were supported included:

• Image to boot from SD card with ATF.
• Image to boot from SD card without ATF.
• Image to boot from NAND with ATF.
• Image to boot from NAND without ATF.
• Image to boot from QSPI with ATF.
• Image to boot from QSPI without ATF.

As the objective was to create a single image that includes all the components needed, all these files needed to be unified in one single defconfig file.

The defconfig file updates considers the following:

• Changes only applies to SOCFPGA 64 family.
• The default defconfig now supports all types of boot devices (SD card, QSPI and NAND).
  Note: Combined defconfig doesn’t apply to N5X as it doesn’t have a QSPI or NAND defconfig.
• The ATF defconfig is the default defconfig.
• The non-ATF defconfig is renamed as legacy_defconfig.
• No change to VAB defconfigs.

The following table shows how the defconfig files were updated:

Next is shown few notes related to some important parameters:

u-boot device tree

The hierarchy of the interesting part of the u-boot device tree is the following. Here is shown how these files are related (one .dtsi file includes others). At the end, this could be seen as one single linear device tree. This is something important to understand hence some of the changes implies to expand or remove some nodes that are defined in a different .dtsi file.

socfpga_soc64_fit-u-boot.dtsi

• This file has the format of a .its file that defines the structure of the u-boot FIT image named u-boot.itb. This is integrated by the following items:

  - Raw u-boot without device tree (“uboot”): u-boot-nodtb.bin
  - ATF(“atf”): bl31.bin
  - u-boot device tree used to boot from eMMC (“fdt-0”): u-boot.dtb

  &binman {
  	u-boot {
  		filename = "u-boot.itb";
  		fit {
  			:
  
  			images {
  				uboot {
  					description = "U-Boot SoC64";
  					:
  					uboot_blob: blob-ext {
  						filename = "u-boot-nodtb.bin";
  					};
  					:
  				};
  
  				atf {
  					description = "ARM Trusted Firmware";
  					:
  					atf_blob: blob-ext {
  						filename = "bl31.bin";
  					};
  					:
  				};
  
  				fdt-0 {
  					description = "U-Boot SoC64 flat device-tree";
  					:
  					uboot_fdt_blob: blob-ext {
  						filename = "u-boot.dtb";
  					};
  					:
  				};
  			};
  
  			:
  		};
  	};
  };
  

• It defines the configuration for the board_0, which is the default one indicating that the software components needed to boot u-boot in this configuration are: { “atf”, “u-boot”, ”fdt-0” }. This configuration is also part of the u-boot.itb FIT image.

  &binman {
  	u-boot {
  		filename = "u-boot.itb";
  		fit {
  			:
  
  			images {
  				uboot {
  					:
  				};
  
  				atf {
  					:
  				};
  
  				fdt-0 {
  					:
  				};
  			};
  
  			board_config: configurations {
  				default = "board-0;
  				board_0 {
  					description = "Intel SoC64 FPGA";
  					firmware = "atf";
  					loadables = "uboot";
  					fdt = "fdt-0";
  					signature {
  						:
  						sign-images = "atf", "fdt-0", "uboot";
  					};
  				};
  			};
  		};
  	};
  };
     

• It defines the structure of the Linux FIT image (kernel.itb), but as we will see later, this node is discarded later, so this file is no longer created during the u-boot build stage.

  &binman {
  	u-boot {
  		filename = "u-boot.itb";
  		fit {
  			:
  		};
  	};
  
  	kernel {
  		filename = "kernel.itb";
  		fit {
  			:
  		};
  	};
  };
     

socfpga_<device>_socdk-u-boot.dtsi

• Enabled the blobs for all the boot devices: nand, mmc and qspi. As the objective is to have one single image, all the possible devices need to be initialized. The mmc controller is used to access SD Card so this node must be enabled (the enabling is done at socfpga_agilex_socdk.dts file).

  &nand {
  	status = "okay";
  	nand-bus-width = <16>;
  	u-boot,dm-pre-reloc;
  };
  
  &mmc {
  	drvsel = <3>;
  	smplsel = <0>;
  	u-boot,dm-pre-reloc;
  };
  
  &qspi {
  	status = "okay";
  };
         

• Select as possible boot device the mmc (SD card), qspi and nand. The u-boot SPL will try to boot from each boot device and will stop upon success.

  /{
  	chosen {
  		stdout-path = "serial0:115200n8";
  		u-boot,spl-boot-order = &mmc,&flash0,&nand;
  	};
  
  	:
  };
          

• It extends the content of the u-boot image (“u-boot.itb”) adding a new device tree used to boot from nand (“fdt-1”).

  &images {
  	fdt-1 {
  		description = "socfpga_socdk_nand";
  		:
  		fdt_1_blob: blob-ext {
  			filename = "arch/arm/dts/socfpga_agilex_socdk_nand.dtb";
  		};
  		:
  	};
  };
  

• It extends the board_config blob to create a new configuration, "board_1", indicating that the software components needed to boot u-boot in this configuration are: { "atf", "uboot", "fdt-1" }. This configuration is used to boot from a nand device.
  Note: For u-boot, only configuration for "board_0" and "board_1" is provided, being "board_0" the default one. When the board_id detected is '1' the "board_1" configuration is used, in any other case "board_0" configuration will be used.

  &board_config {
  	board-1 {
  		description = "board_1";
  		firmware = "atf";
  		loadables = "uboot";
  		fdt = "fdt-1";
  		signature {
  			algo = "crc32";
  			key-name-hint = "dev";
  			sign-images = "atf", "uboot", "fdt-1";
  		};
  	};
  	:
  };
          

• Finally, in this file is deleted the kernel node that was initially attempted to create the kernel.itb in the socfpga_soc64_fit-u-boot.dtsi file. The effect of this is that the kernel.itb FIT image is no longer created during the u-boot build stage.

  &binman {
  	/delete-node/ kernel;
  };
        

u-boot Boot Flow

The following figure shows the u-boot SPL flow related to the Single Boot Image during the process in which this launches u-boot. The steps are described as follow:

S1. U-boot SPL reads the board_ID. Here the value read is appended to the “board_” string.
S2. U-boot SPL tries to find u-boot.itb in the boot flash devices (in order SDcard/eMMC, qspi and nand). U-boot SPL compares the appended board ID string with the description name in each one of the FIT configurations available in the FIT file (u-boot.itb), so it can determine which configuration is the appropriate for this board.
S3. In case there is a match, the u-boot SPL identifies the software items needed to launch u-boot from the matching configuration and obtain the address of them in the FIT file (u-boot.itb). In case there isn’t any match, u-boot SPL does the same, but using the default configuration instead.
S4. The u-boot SPL launches u-boot with the software components identified. In this case the software components are the raw u-boot, the ATF and the appropiate device tree. Notice that in this case the only variation is the device tree.

Linux Kernel FIT Generation

As mentioned previously, before GSRD 21.3, the kernel FIT image (kernel.itb) was built during the u-boot build stage. Now, the Linux FIT image is decoupled from the u-boot build and is being built during the Yocto image creation instead.

Another important change related to the Linux FIT image is that the FPGA configuration is handled by u-boot during the process of booting Linux (following the HPS Boot First flow). This is done based on the following:

• The Linux kernel FIT image includes the FPGA configuration image with a .rbf format (besides the Linux kernel and the device tree).

• The Linux Kernel FIT image may include more than one FPGA configuration files allowing to select the specific one for a board using the board configuration entry in the .ITS file. In this way is possible to specify which set of images applies for certain board (linux kernel, device tree and FPGA configuration).

The fit_kernel_<device>.its is used to indicate Yocto mkimage how to pack the kernel.itb file.

The path of the ITS files is the following. This applies for Agilex and Stratix10 devices:

./gsrd_socfpga/meta-intel-fpga-refdes/recipes-kernel/linux/linux-socfpga-lts/fit_kernel_<device>.its

From the .its file it can be seen that the kernel.itb FIT image is integrated by the following components:

• Kernel Image (“kernel”): Image.Izma

  / {
  	description = "FIT image with kernel, DTB and FPGA core binary";
  	#address-cells = <1>;
  
  	images {
  		kernel {
  			description = "Linux Kernel";
  			data = /incbin/("./Image.lzma");
  			type = "kernel";
  			arch = "arm64";
  			os = "linux";
  			compression = "lzma";
  			load = <0x6000000>;
  			entry = <0x6000000>;
  			hash {
  				algo = "crc32";
  			};
  		};
  
  		:
  	};
  
  	:
  };
        

• Device tree for Simics board(“fdt-0”): socfpga_<device>_vanilla.dtb

  / {
      :
  	images {
  		:
  		fdt-0 {
  			description = "socfpga_socdk_vanilla";
  			data = /incbin/("./socfpga_agilex_vanilla.dtb");
  			type = "flat_dt";
  			arch = "arm64";
  			compression = "none";
  			hash {
  				algo = "crc32";
  			};
  		};
  		:
  	};
  	:
  };
        

• Device tree for NAND (“fdt-1”): socfpga_<device>_socdk_nand.dtb

  / {
  	:
  	images {
  		:
  		fdt-1 {
  			description = "socfpga_socdk_nand";
  			data = /incbin/("./socfpga_agilex_socdk_nand.dtb");
  			type = "flat_dt";
  			arch = "arm64";
  			compression = "none";
  			hash {
  				algo = "crc32";
  			};
  		};
  		:
  	};
  	:	
  };
        

• Device tree for Partial Reconfiguration(“fdt-3”): socfpga_<device>_socdk_pr.dtb

  / {
  	description = "FIT image with kernel, DTB and FPGA core binary";
  	#address-cells = <1>;
  
  	images {
  		:
  		fdt-3 {
  			description = "socfpga_socdk_pr";
  			data = /incbin/("./socfpga_agilex_socdk_pr.dtb");
  			type = "flat_dt";
  			arch = "arm64";
  			compression = "none";
  			hash {
  				algo = "crc32";
  			};
  		};
  
  		:		
  	};
  	:
  };
        

• Device tree for SDcard(“fdt-4”): socfpga_<device>_socdk.dtb

  / {
      :
  	images {
  		:
  		fdt-4 {
  			description = "socfpga_socdk_combined";
  			data = /incbin/("./socfpga_agilex_socdk.dtb");
  			type = "flat_dt";
  			arch = "arm64";
  			compression = "none";
  			hash {
  				algo = "crc32";
  			};
  		};
  		:
  	};
  	:
  };
        

• FPGA fabric configuration for NAND boot (“fpga-1”): nand.core.rbf

  / {
  	:
  	images {
  		:		
  		fpga-1 {
  			description = "FPGA bitstream for NAND";
  			data = /incbin/("./nand.core.rbf");
  			type = "fpga";
  			:
  		};
  		:
      }
  	:
  };
        

• FPGA fabric configuration for partial reconfiguration(“fpga-3”): ghrd_pr.core.rbf

  / {
  	:
  	images {
  		:	
  		fpga-3 {
  			description = "FPGA bitstream for PR";
  			data = /incbin/("./ghrd_pr.core.rbf");
  			type = "fpga";
  			:
  		};
  	};
  	:
  };
        

• FPGA fabric configuration for SD card boot (“fpga-4”): ghrd.core.rbf

  / {
  	:
  	images {
  		:
  		fpga-4 {
  			description = "FPGA bitstream for GHRD";
  			data = /incbin/("./ghrd.core.rbf");
  			type = "fpga";
  			:
  		};
  		:
  	};
  
  	:
  };
        

• Configurations for each one of the boards supported, indicating the SW items that will be loaded. Also defines the board_0 as the default configuration. As an example, the configuration for board-4 indicates that the kernel, fdt-4 and fpga-4 images will be loaded. Notice that the Linux kernel is common for each one the configurations and the fpga configuration and the device tree are board specific.

  / {
  	:
  	images {
  		kernel {
  			:
  		};
  
  		fdt-0 {
  			:
  		};
  
  		fdt-1 {
  			:
  		};
  
  		fdt-3 {
  			:
  		};
  
                  fdt-4 {
  			:
  		};
  
  		fpga-0 {
  			:
  
  		fpga-1 {
  			:
  		};
  
  		fpga-3 {
  			:
  		};
  
  		fpga-4 {
  			:
  		};
  	};
  
  	configurations {
  		default = "board-0";
  
  		board-0 {
  			description = "board_0";
  			kernel = "kernel";
  			fdt = "fdt-0";
  			signature {
  				:
  				sign-images = "fdt-0", "kernel";
  			};
  		};
  
  		board-1 {
  			description = "board_1";
  			kernel = "kernel";
  			fdt = "fdt-1";
  			fpga = "fpga-1";
  			signature {
  				:
  				sign-images = "fdt-1", "kernel", "fpga-1";
  			};
  		};
  
  		board-3 {
  			description = "board_3";
  			kernel = "kernel";
  			fdt = "fdt-3";
  			fpga = "fpga-3";
  			signature {
  				:
  				sign-images = "fdt-3", "kernel", "fpga-3";
  			};
  		};
  
  		board-4 {
  			description = "board_4";
  			kernel = "kernel";
  			fdt = "fdt-0";
  			fpga = "fpga-4";
  			signature {
  				:
  				sign-images = "fdt-4", "kernel", "fpga-4";
  			};
  		};
  
  	};
  };
        

Linux Boot Flow

As mentioned earlier, the process to boot Linux was modified starting from GSRD 21.3 to support the Single Image Boot flow which relies on u-boot getting the appropriate boot images from the Linux FIT image (kernel.itb). The process to achieve this is similar to the flow followed by u-boot SPL, which requires to identify the board_ID for the platform in which this is running and then identify the configuration in the FIT image that matches with this ID and then load the images indicated by this configuration. The images that are loaded as part of the Linux Boot flow are: FPGA configuration, Linux kernel and Linux device tree.

Reading Board ID

u-boot reads the board ID in during the board_init_r stage in the arch_misc_init() function in arch/arm/mach-socfpga/misc_soc64.c file. Here the ID is read using the socfpga_get_board_id() function and then saving this in the ${board_id} environment variable, so this could be available over the rest of the boot flow.

Distro Boot

u-boot 2021.7 uses Distro Boot for booting up Linux from flash devices. In this flow, Distro bootcmd looks for a boot script file named boot.scr.uimg in the flash devices defined by the ${boot_targets} environment variable. Once that the script is found, u-boot proceeds to execute it. From this point, the content of the script controls the rest of the boot flow.
The environment variable is initialized from the BOOT_TARGET_DEVICES key which is defined at include/configs/socfpga_soc64_common.h file:

#define BOOT_TARGET_DEVICES(func) \
	BOOT_TARGET_DEVICES_MMC(func) \
	BOOT_TARGET_DEVICES_QSPI(func) \
	BOOT_TARGET_DEVICES_NAND(func)
      

The order can be manually updated whether from the .h file or directly from the u-boot console. As example use the following command to select only mmc0: setenv boot_targets mmc0.

Boot script

For Agilex and S10, the u-boot looks for the boot.scr.uimg file in the devices specified by the ${boot_targets} environment variable. This script is used to launch Linux and also to perform the FPGA configuration. Depending on the boot device this should be located as follow:

• SD Card: FAT partition with the file name boot.scr.uimg
• NAND Flash: UBI volume named: script
• QSPI flash: Raw offset at 0x211_0000 (defined as $qspiscriptaddr)

The boot.scr.uimg is actually a FIT image and this is generated from a file named u-boot.txt and also a ITS file named u-boot_script.its. The uboot.txt script provides the instructions that the u-boot needs to execute to continue with the Linux boot process. In other hand the u-boot_script.its file defines the structure of the FIT image. These 2 files are provided at:

https://releases.rocketboards.org/2021.11/uboot-script/.

The command used to generate the FIT image is this:

mkimage -f uboot_script.its boot.scr.uimg

The script receives as input the device the target device from which it will try to boot (mmc, qspi or nand). Depending on the target, it will try to find in this device the Linux kernel in this way:

• SD Card: FAT partition with the file name kernel.itb
• NAND Flash: UBI volume named: kernel
• QSPI flash: Raw offset at 0x0212_0000 (defined as $qspibootimageaddr)

In case that the boot.scr.uimg file is found, u-boot proceeds to execute the commands defined in the script which finishes with the call of the bootm command. This command is normally used to launch the Linux image which has been loaded into RAM at the address given as parameter. In order to support the Single Image Boot mode, the bootm command was customized in such a way that it can receive as parameter the board_ID stored in the ${borad_id} environment variable. Having the board_ID, the bootm command is now capable to find the software components that are appropriate for this specific board (normally kernel + device tree) that are included in the FIT image (kernel.itb). Also, the bootm flow was also adapted to perform the FPGA configuration that applies for this board (using the board_id parameter). The format of the bootm command is like this:

bootm ${loadaddr}#board-${board_id}

FPGA Configuration

Previously, the FPGA configuration was performed using a script that u-boot used to call during its initialization. Starting from GSRD 21.3, the FPGA configuration can be performed by u-boot during the Linux boot up process calling the bootm command as part of the u-boot execution of the boot.scr.uimg as indicated in the previous section. The bootm command receives as input parameter the board_id specific for the board in which the software is running over. Using this board_id and the configurations described in the kernel.itb FIT image, the software is able to determine which FPGA configuration image corresponds to the current board and proceeds to perform the FPGA configuration.

Linux DTB patch with location of SPT0

In the RSU flow, Linux requires knowing the location of the SPT0 so it can handle RSU operations, but to minimize the risk of corruption, the Linux MTD doesn’t expose the location of DCMF, DCIO and the factory image. The location of the SPT0 can’t be defined statically in the device tree because this is not fixed as depends on the length of the factory image.
In order to address this problem, it’s necessary that u-boot patches the Linux device tree to assign the offset of SPT0 to the rsu-handle partition (alias of qspi_boot) in rsu_dtb() function. As part of the updates done to support booting from the Single Boot image, the patching of the Linux device tree is done in the flow of the board_prep_linux() function at the arch/arm/mach-socfpga/board.c file. In this function, the linux_qspi_enable and rsu_status commands are called. As part of the rsu_status command, the rsu_dtb command is also called, being this the one that finally patches the Linux device tree.

The following image describes the u-boot flow related to the Single Boot Image during the process in which this launches Linux. A brief description of the steps in the figure are also provided: