RocketBoards
Log inRegister
You are here: Documentation » Building Bootloader for Agilex 7

Building Bootloader for Agilex 7
Building latest bootloaders for Agilex 7 SoC Devices

17 April 2024 - 02:10 | Release 24.1 | | Agilex7, atf, boot, u-boot
For Cyclone V and Arria 10 devices, please refer to Building Bootloader for Cyclone V and Arria 10.
For Stratix 10 devices, please refer to Building Bootloader for Stratix10

Introduction

This page contains instructions on how to build U-Boot in the following configurations:

Device Quartus Version ATF Branch U-Boot Branch Linux Branch
Agilex 24.1 Pro 2.10.0 2023.10 6.1.5685-lts

Starting with SoC EDS Pro version 19.3, the following changes were made:
  • The bootloader source code was removed from SoC EDS. Instead, the user needs to clone the git trees from https://github.com/altera-opensource/u-boot-socfpga.
  • The same U-Boot branch is used for all SoC FPGA devices: Cyclone V SoC, Arria V SoC, Arria 10 SoC, Stratix 10 SoC, Agilex.

Starting with Quartus Pro 20.3, the SoC EDS was discontinued, and the functionality of the tools which were previously part of SoC EDS are provided separately.

U-Boot Build Flow

For Stratix 10 SoC and Agilex, all the handoff information created by the Quartus compilation is part of the configuration bitstream. The bsp-editor is not used, and the bootloader build flow does not depend on the Quartus outputs.

s10-flow.png

Single Boot Image

Starting with U-Boot 2021.07, the following changes were made to enable a single set of binaries to be used with multiple boards and hardware projects:
  • The Quartus hardware project defines a JTAG User Code which is used by the rest of the system as a board_id to indentify the hardware
  • U-Boot has a single defconfig enabling all possible HPS hardware, and depending on the timeouts to determine which hardware is not actually available
  • U-Boot has a single device tree FIT file enabling all possible HPS hardware, but with different configurations inside, selected according to the board_id
  • Linux FIT file also has a different configuration for each board_id. Each configuration includes the kernel, the specific device file, and an optional core.rbf file. If the core.rbf file is specified, the fabric is configured with that file

Refer to Single Image Boot for more details about this feature.

The Agilex GSRDs are also updated to use this feature. See the GSRD documentation for details:

U-Boot Branches

The official Intel SOCFPGA U-Boot repository is located at https://github.com/altera-opensource/u-boot-socfpga.

Notes:
  • A "RC" labeled branch is for internal active development use and customer early access without official customer support.
  • Latest stable branch (no RC labeled) is strongly recommended for development and production use outside of Intel.
  • See doc/README.socfpga for Quartus and Device support.

Boot from SD Card Example

This example shows a simple example booting Agilex from SD card.

Note the following:
  • Hardware design was customized as follows:
    • Disable SGMII and PR to reduce boot time
  • U-Boot was customized as follows:
    • Disable NAND in the unified defconfig file, as we do not need it
    • Boot only from SD card, as opposed to trying SD, QSPI and NAND
    • Use Dwarf4 for debug information, to be compatible with current Arm DS debugger
    • Configure FPGA fabric from boot command using fpga load command explicitly (instead of using the bootm command to do it)
    • Use booti command to boot Linux, with separate files for kernel and device tree

The above customizations may be useful for debugging purposes for example.

The following build instructions produce a QSPI(.jic) and an SDCard Image (.img) which includes the components indicated in the following figure:

bootLoaderImages.jpg

This example uses building U-Boot manually. See GSRD for Agilex 7 F-Series Transceiver-SoC DevKit (P-Tile and E-Tile) for the full fledged booting from SD card example, where U-Boot is built through Yocto recipes.

This example, and the current GSRD release target the production version of the Intel Agilex 7 F-Series Transceiver-SoC Development Kit. You can confirm you have a production version of the board by checking that it's serial number, on a sticker on the back of the board, is greater than AGF61SI0000576. Refer to https://www.intel.com/content/www/us/en/products/details/fpga/development-kits/agilex/f-series-transceiver.html for more details about the board.

Prerequisites

The following are required:
  • Host machine running Linux. Ubuntu 22.04 was used, but other versions may work too.
  • Internet connection to download the tools and clone the U-Boot git tree from github. If you are behind a firewall you will need your system administrator to enable you to get to the git trees.
  • Intel SoC FPGA Development Kit with your desired device: Agilex 7
  • Quartus Prime Pro version 24.1

Note that the examples presented on this page boot to Linux and they require Linux kernel, device tree and rootfilesystem to boot. However, you can omit the Linux binaries and just boot to U-Boot prompt if you want to.

See https://www.rocketboards.org/foswiki/Documentation/AgilexSoCGSRD#Booting_From_NAND_for_the_DK_45SI_45AGF014EA_Version

Setting Up Environment

Create top folder: 
sudo rm -rf agilex_example.sdmmc
mkdir agilex_example.sdmmc
cd agilex_example.sdmmc
export TOP_FOLDER=`pwd`

Download and setup the the toolchain as follows: 

cd $TOP_FOLDER
wget https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel\
/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz
tar xf gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz
rm -f gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz
export PATH=`pwd`/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/bin:$PATH
export ARCH=arm64
export CROSS_COMPILE=aarch64-none-linux-gnu-

Enable Quartus tools to be called from command line: 

export QUARTUS_ROOTDIR=~/intelFPGA_pro/24.1/quartus/
export PATH=$QUARTUS_ROOTDIR/bin:$QUARTUS_ROOTDIR/linux64:$QUARTUS_ROOTDIR/../qsys/bin:$PATH

Build Hardware Design 

cd $TOP_FOLDER
rm -rf ghrd-socfpga agilex_soc_devkit_ghrd
git clone -b QPDS24.1_REL_GSRD_PR  https://github.com/altera-opensource/ghrd-socfpga
mv ghrd-socfpga/agilex_soc_devkit_ghrd .
rm -rf ghrd-socfpga
cd agilex_soc_devkit_ghrd
# disable sgmii and partial reconfiguration - to decrease build time
export HPS_ENABLE_SGMII=0
export ENABLE_PARTIAL_RECONFIGURATION=0
make scrub_clean_all
make generate_from_tcl
make all
cd ..

Build Arm Trusted Firmware 

cd $TOP_FOLDER
rm -rf arm-trusted-firmware
git clone https://github.com/altera-opensource/arm-trusted-firmware
cd arm-trusted-firmware
# comment out next line to use the latest ATF branch
git checkout -b test -t origin/socfpga_v2.10.0
make bl31 PLAT=agilex DEPRECATED=1
cd ..

Build U-Boot 

cd $TOP_FOLDER
rm -rf u-boot-socfpga
git clone https://github.com/altera-opensource/u-boot-socfpga
cd u-boot-socfpga
# comment out next line to use the latest default branch
git checkout -b test -t origin/socfpga_v2023.10
# enable dwarf4 debug info, for compatibility with arm ds
sed -i 's/PLATFORM_CPPFLAGS += -D__ARM__/PLATFORM_CPPFLAGS += -D__ARM__ -gdwarf-4/g' arch/arm/config.mk
# use 'Image' for kernel image instead of 'kernel.itb'
sed -i 's/kernel\.itb/Image/g' include/configs/socfpga_soc64_common.h
# only boot from SD, do not try QSPI and NAND
sed -i 's/u-boot,spl-boot-order.*/u-boot\,spl-boot-order = \&mmc;/g' arch/arm/dts/socfpga_agilex_socdk-u-boot.dtsi
# disable NAND in the device tree
sed -i '/&nand {/!b;n;c\\tstatus = "disabled";' arch/arm/dts/socfpga_agilex_socdk-u-boot.dtsi
# remove the NAND configuration from device tree
sed -i '/images/,/binman/{/binman/!d}' arch/arm/dts/socfpga_agilex_socdk-u-boot.dtsi

# link to atf
ln -s $TOP_FOLDER/arm-trusted-firmware/build/agilex/release/bl31.bin .

# Create configuration custom file. 
cat << EOF > config-fragment-agilex
# - Disable NAND/UBI related settings from defconfig. 
CONFIG_NAND_BOOT=n
CONFIG_SPL_NAND_SUPPORT=n
CONFIG_CMD_NAND_TRIMFFS=n
CONFIG_CMD_NAND_LOCK_UNLOCK=n
CONFIG_NAND_DENALI_DT=n
CONFIG_SYS_NAND_U_BOOT_LOCATIONS=n
CONFIG_SPL_NAND_FRAMEWORK=n
CONFIG_CMD_NAND=n
CONFIG_MTD_RAW_NAND=n
CONFIG_CMD_UBI=n
CONFIG_CMD_UBIFS=n
CONFIG_MTD_UBI=n
CONFIG_ENV_IS_IN_UBI=n
CONFIG_UBI_SILENCE_MSG=n
CONFIG_UBIFS_SILENCE_MSG=n
# - Disable distroboot and use specific boot command. 
CONFIG_DISTRO_DEFAULTS=n
CONFIG_HUSH_PARSER=y
CONFIG_SYS_PROMPT_HUSH_PS2="> "
CONFIG_USE_BOOTCOMMAND=y
CONFIG_BOOTCOMMAND="load mmc 0:1 \${loadaddr} ghrd.core.rbf; bridge disable;fpga load 0 \${loadaddr} \${filesize};bridge enable;run mmcload;run linux_qspi_enable;run rsu_status;run mmcboot"
CONFIG_CMD_FAT=y
CONFIG_CMD_FS_GENERIC=y
CONFIG_DOS_PARTITION=y
CONFIG_SPL_DOS_PARTITION=y
CONFIG_CMD_PART=y
CONFIG_SPL_CRC32=y
CONFIG_LZO=y
CONFIG_CMD_DHCP=y
# Enable more QSPI flash manufacturers
CONFIG_SPI_FLASH_MACRONIX=y
CONFIG_SPI_FLASH_GIGADEVICE=y
CONFIG_SPI_FLASH_WINBOND=y
CONFIG_SPI_FLASH_ISSI=y
EOF

# build U-Boot
make clean && make mrproper
make socfpga_agilex_defconfig
# Use created custom configuration file to merge with the default configuration obtained in .config file. 
./scripts/kconfig/merge_config.sh -O ./ ./.config ./config-fragment-agilex
make -j 64
cd ..

Note: The following commands are ran before starting Linux:

Important: If the fabric is already configured, before running the 'fpga load' command, you must first run the 'bridge disable' command as shown in the above example. If the fabric is not already configured, the command will fail without any adverse effects.

Prepare QSPI Image 

cd $TOP_FOLDER
quartus_pfg -c \
  agilex_soc_devkit_ghrd/output_files/ghrd_agfb014r24b2e2v.sof \
  ghrd.jic \
  -o device=MT25QU128 \
  -o flash_loader=AGFB014R24B2E2V \
  -o hps_path=u-boot-socfpga/spl/u-boot-spl-dtb.hex \
  -o mode=ASX4 \
  -o hps=1

Building Linux Kernel

Download and compile Linux: 
cd $TOP_FOLDER
rm -rf linux-socfpga
git clone https://github.com/altera-opensource/linux-socfpga linux-socfpga
cd linux-socfpga
# comment out next line to use the latest Linux kernel branch
git checkout -b test-kernel -t origin/socfpga-6.1.68-lts
make clean && make mrproper
make defconfig
# enable kernel debugging with RiscFree
./scripts/config --set-val CONFIG_DEBUG_INFO  y
./scripts/config --set-val CONFIG_GDB_SCRIPTS y
make oldconfig
make -j 64 Image dtbs

The following items are built in $TOP_FOLDER:
  • linux-socfpga/arch/arm64/boot/dts/intel/socfpga_agilex_socdk.dtb
  • linux-socfpga/arch/arm64/boot/Image

Building Yocto Rootfs

This section presents how to build the Linux rootfs using Yocto recipes. Note that the yocto recipes actually build everything, but are only interested in the rootfs.

First, make sure you have Yocto system requirements met: https://docs.yoctoproject.org/3.4.1/ref-manual/system-requirements.html#supported-linux-distributions.

The command to install the required packages on Ubuntu is: 
sudo apt install gawk wget git diffstat unzip texinfo gcc build-essential chrpath \
socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \
python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm python3-subunit \
mesa-common-dev zstd liblz4-tool

Note: You can also use a Docker container to build the Yocto recipes, refer to https://rocketboards.org/foswiki/Documentation/DockerYoctoBuild for details. When using a Docker container, it does not matter what Linux distribution or packages you have installed on your host, as all dependencies are provided by the Docker container. 

cd $TOP_FOLDER
rm -rf yocto && mkdir yocto && cd yocto
git clone -b nanbield https://git.yoctoproject.org/poky
git clone -b nanbield https://git.yoctoproject.org/meta-intel-fpga
git clone -b nanbield https://github.com/openembedded/meta-openembedded
source poky/oe-init-build-env ./build
echo 'MACHINE = "agilex7_dk_si_agf014ea"' >> conf/local.conf
echo 'BBLAYERS += " ${TOPDIR}/../meta-intel-fpga "' >> conf/bblayers.conf
echo 'BBLAYERS += " ${TOPDIR}/../meta-openembedded/meta-oe "' >> conf/bblayers.conf
# enable ssh and gdb access
echo 'CORE_IMAGE_EXTRA_INSTALL += "openssh gdbserver"' >> conf/local.conf
bitbake core-image-minimal

The following file is created:
  • $TOP_FOLDER/yocto/build/tmp/deploy/images/agilex7_dk_si_agf014ea/core-image-minimal-agilex7_dk_si_agf014ea.rootfs.tar.gz

Build SD Card Image 

cd $TOP_FOLDER
sudo rm -rf sd_card && mkdir sd_card && cd sd_card
wget https://releases.rocketboards.org/release/2020.11/gsrd/tools/make_sdimage_p3.py
# remove mkfs.fat parameter which has some issues on Ubuntu 22.04
sed -i 's/\"\-F 32\",//g' make_sdimage_p3.py
chmod +x make_sdimage_p3.py
mkdir sdfs &&  cd sdfs
cp $TOP_FOLDER/u-boot-socfpga/u-boot.itb .
cp $TOP_FOLDER/linux-socfpga/arch/arm64/boot/Image .
cp $TOP_FOLDER/linux-socfpga/arch/arm64/boot/dts/intel/socfpga_agilex_socdk.dtb .
cp $TOP_FOLDER/ghrd.core.rbf .
cd ..
mkdir rootfs && cd rootfs
sudo tar xf $TOP_FOLDER/yocto/build/tmp/deploy/images/agilex7_dk_si_agf014ea/core-image-minimal-agilex7_dk_si_agf014ea.rootfs.tar.gz
sudo rm -rf lib/modules/*
cd ..
sudo python3 make_sdimage_p3.py -f \
-P sdfs/*,num=1,format=fat32,size=56M \
-P rootfs/*,num=2,format=ext3,size=56M \
-s 128M \
-n sdcard.img
cd ..

Boot Linux

Write SD card image to SD card and insert it in the slot.

Set MSEL to JTAG.

Write the QSPI flash image: 
cd $TOP_FOLDER/flash_image/
quartus_pgm -m jtag -o "pvi;./ghrd.hps.jic"

Set MSEL to QSPI and power cycle the board. Linux will boot, log in with 'root' as username: 
U-Boot SPL 2023.10-31805-g69d5338be0-dirty (Apr 04 2024 - 21:25:25 -0500)
Reset state: Cold
MPU          1200000 kHz
L4 Main	      400000 kHz
L4 sys free   100000 kHz
L4 MP         200000 kHz
L4 SP         100000 kHz
SDMMC          50000 kHz
DDR: 8192 MiB
SDRAM-ECC: Initialized success with 1721 ms
QSPI: Reference clock at 400000 kHz
WDT:   Started watchdog@ffd00200 with servicing every 1000ms (10s timeout)
Trying to boot from MMC1
## Checking hash(es) for config board-0 … OK
## Checking hash(es) for Image atf … crc32+ OK
## Checking hash(es) for Image uboot … crc32+ OK
## Checking hash(es) for Image fdt-0 … crc32+ OK
NOTICE:  BL31: v2.10.0	(release):QPDS24.1_REL_GSRD_PR
NOTICE:  BL31: Built : 21:24:57, Apr  4 2024

U-Boot 2023.10-31805-g69d5338be0-dirty (Apr 04 2024 - 21:25:25 -0500)socfpga_agilex

CPU:   Intel FPGA SoCFPGA Platform (ARMv8 64bit Cortex-A53)
Model: SoCFPGA Agilex SoCDK
DRAM:  2 GiB (effective 8 GiB)
Core:  28 devices, 23 uclasses, devicetree: separate
WDT:   Started watchdog@ffd00200 with servicing every 1000ms (10s timeout)
MMC:   dwmmc0@ff808000: 0
Loading Environment from FAT... Unable to read "uboot.env" from mmc0:1...
In:    serial0@ffc02000
Out:   serial0@ffc02000
Err:   serial0@ffc02000
Net:   
Warning: ethernet@ff800000 (eth0) using random MAC address - 8e:fa:ef:f4:c3:3c
eth0: ethernet@ff800000
Hit any key to stop autoboot:  5  4  3  2  1  0 
2174976 bytes read in 101 ms (20.5 MiB/s)
FPGA not ready. Bridge reset aborted!
…FPGA reconfiguration OK!
38623744 bytes read in 1765 ms (20.9 MiB/s)
18128 bytes read in 3 ms (5.8 MiB/s)
SF: Detected mt25qu02g with page size 256 Bytes, erase size 64 KiB, total 256 MiB
Enabling QSPI at Linux DTB...
Working FDT set to 8000000
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
libfdt fdt_path_offset() returned FDT_ERR_NOTFOUND
QSPI clock frequency updated
RSU: Firmware or flash content not supporting RSU
RSU: Firmware or flash content not supporting RSU
RSU: Firmware or flash content not supporting RSU
RSU: Firmware or flash content not supporting RSU
## Flattened Device Tree blob at 08000000
   Booting using the fdt blob at 0x8000000
Working FDT set to 8000000
   Loading Device Tree to 000000007eafe000, end 000000007eb05fff … OK
Working FDT set to 7eafe000

Starting kernel …

Deasserting all peripheral resets
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 6.1.68-248670-ge9428cbd9991 (radu@build-machine) (aarch64-none-linux-gnu-gcc (GNU Toolchain for the Arm Architecture 11.2-2022.02 (arm-11.14)) 11.2.1 20220111, GNU ld (GNU Toolchain for the Arm Architecture 11.2-2022.02 (arm-11.14)) 2.37.20220122) #1 SMP PREEMPT Thu Apr  4 21:30:32 CDT 2024
[    0.000000] Machine model: SoCFPGA Agilex SoCDK

…

[    4.256648] socfpga-dwmac ff800000.ethernet eth0: PHY [stmmac-0:04] driver [Micrel KSZ9031 Gigabit PHY] (irq=POLL)
[    4.267016] socfpga-dwmac ff800000.ethernet eth0: No Safety Features support found
[    4.274588] socfpga-dwmac ff800000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[    4.283406] socfpga-dwmac ff800000.ethernet eth0: registered PTP clock
[    4.290430] socfpga-dwmac ff800000.ethernet eth0: configuring for phy/rgmii link mode
[    8.394171] socfpga-dwmac ff800000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

Poky (Yocto Project Reference Distro) 4.3.4 agilex7_dk_si_agf014ea /dev/ttyS0

agilex7_dk_si_agf014ea login: root
root@agilex7_dk_si_agf014ea:~#

Other Examples

Boot from QSPI

See https://rocketboards.org/foswiki/Documentation/AgilexSoCGSRD#Booting_from_QSPI.

Boot with eMMC Storage on HPS

See https://rocketboards.org/foswiki/Projects/AgilexSoCWithEMMCStorageOnHPS.

Boot with NAND Storage on HPS

See https://rocketboards.org/foswiki/Documentation/AgilexSoCGSRD#Booting_From_NAND.

Running U-Boot with the Debugger from Command Line

This section presents examples of how to run U-Boot with the Arm Development Studio from command line. This offers a simple and convenient way to run U-Boot and use it for example to program onboard flash.

1. Use the binaries built for the Boot from SD Card example: 
cd agilex_example.sdmmc

2. Run jtagconfig to determine if the HPS is currently in the JTAG scanchain: 
jtagconfig
1) Agilex SI/SoC Dev Kit [3-4.3.4]
  6BA00477   S10HPS/AGILEX_HPS
  0341A0DD   AGFB014(F25A|R24AR0)
If HPS is not present, the line shown in red above will not appear.

3. Configure the FPGA with the debug SOF: 
quartus_pgm -c 1 -m jtag -o "p;agilex_soc_devkit_ghrd/output_files/ghrd_agfb014r24a3e3vr0_hps_debug.sof@2"
If the HPS is not present in the jtagconfig output above, please remove the "@2" from the command line above.

6. Create debugger script: 
cat <<EOT > run-u-boot.ds
interrupt
restore "u-boot-socfpga/spl/u-boot-spl-dtb.bin" binary 0xFFE00000
loadfile "u-boot-socfpga/spl/u-boot-spl"
core 1
set \$PC = \$ENTRYPOINT
core 2
set \$PC = \$ENTRYPOINT
core 3
set \$PC = \$ENTRYPOINT
core 0
set \$PC = \$ENTRYPOINT
thbreak board_boot_order
continue
wait
set spl_boot_list[0]=0
set \$PC=\$LR
restore "u-boot-socfpga/u-boot.itb" binary 0x2000000
continue
EOT

7. Run the debugger from command line, using the connection parameters reported above by jtagconfig, shown in red below: 
/opt/arm/developmentstudio-2022.2/bin/armdbg  \
--cdb-entry="Intel SoC FPGA::Agilex SoC::Bare Metal Debug::Bare Metal Debug::Cortex-A53x4 SMP::Intel FPGA Download Cable"  \
--cdb-entry-param="rvi_address=Agilex SI/SoC Dev Kit on localhost [3-4.3.4]:Agilex SI/SoC Dev Kit 3-4.3.4"  \
--continue_on_error=true \
--stop_on_connect=false \
-s run-u-boot.ds

8. The serial console will show SPL then U-Boot being run: 
U-Boot SPL 2023.07-rc6-30089-gdab86d36f1-dirty (Dec 14 2023 - 13:09:29 -0600)
Reset state: Cold
MPU          1200000 kHz
L4 Main       400000 kHz
L4 sys free   100000 kHz
L4 MP         200000 kHz
L4 SP         100000 kHz
SDMMC          50000 kHz
DDR: 8192 MiB
SDRAM-ECC: Initialized success with 1719 ms
QSPI: Reference clock at 400000 kHz
WDT:   Started watchdog@ffd00200 with servicing every 1000ms (10s timeout)
Trying to boot from MMC1
## Checking hash(es) for config board-0 … OK
## Checking hash(es) for Image atf … crc32+ OK
## Checking hash(es) for Image uboot … crc32+ OK
## Checking hash(es) for Image fdt-0 … crc32+ OK
NOTICE:  BL31: v2.9.1(release):QPDS24.1_REL_GSRD_PR
NOTICE:  BL31: Built : 13:08:35, Dec 14 2023

U-Boot 2023.07-rc6-30089-gdab86d36f1-dirty (Dec 14 2023 - 13:09:29 -0600)socfpga_agilex

CPU:   Intel FPGA SoCFPGA Platform (ARMv8 64bit Cortex-A53)
Model: SoCFPGA Agilex SoCDK
DRAM:  2 GiB (effective 8 GiB)
Core:  28 devices, 23 uclasses, devicetree: separate
Warning: Device tree includes old 'u-boot,dm-' tags: please fix by 2023.07!
WDT:   Started watchdog@ffd00200 with servicing every 1000ms (10s timeout)
MMC:   dwmmc0@ff808000: 0
Loading Environment from FAT... Unable to read "uboot.env" from mmc0:1...
In:    serial0@ffc02000
Out:   serial0@ffc02000
Err:   serial0@ffc02000
Net:   
Warning: ethernet@ff800000 (eth0) using random MAC address - fe:52:50:7d:c7:a7
eth0: ethernet@ff800000
Hit any key to stop autoboot:  0  
SOCFPGA_AGILEX #

Debugging U-Boot with Arm DS Eclipse

This section presents examples of how to debug U-Boot with from the Arm Development Studio Eclipse-based GUI.

Prerequisites

1. Use the binaries built for the Boot from SD Card example: 
cd agilex_example.sdmmc

2. Run jtagconfig to determine if the HPS is currently in the JTAG scanchain: 
jtagconfig
1) Agilex SI/SoC Dev Kit [3-4.3.4]
  6BA00477   S10HPS/AGILEX_HPS
  0341A0DD   AGFB014(F25A|R24AR0)
If HPS is not present, the line shown in red above will not appear.

3. Configure the FPGA with the debug SOF: 
quartus_pgm -c 1 -m jtag -o "p;agilex_soc_devkit_ghrd/output_files/ghrd_agfb014r24b2e2v_hps_debug.sof@2"
If the HPS is not present in the jtagconfig output above, please remove the "@2" from the command line above.

Debugging SPL

1. Run the steps from Prerequisites section

2. Start the Arm Development Studio Eclipse-based GUI: 
/opt/arm/developmentstudio-2022.2/bin/armds_ide -data workspace &
Note the above creates a new workspace in the agilex_example.sdmmc folder.

3. In Eclipse, Go to Run > Debug Configurations to open the Debug Configurations window.

4. In the Debug Configurations window
  • Select the Generic Arm C/C++ Application on the left panel and right-click it. From the menu that appears, select New Configuration.
  • Edit the Name field from "New_configuration" to something more descriptive, such as "Debug Agilex Bootloader"
5. In the Connection tab:
  • Go to Select target section and select Intel SoC FPGA > Agilex > Bare Metal Debug > Debug Cortex-A53_0
  • Select the Target Connection to be Intel FPGA Download Cable
  • Click the Bare Metal Debug > Connection Browse button and select your cable.
The Debug Configurations window should now look like this: agilex-ds-boot-config.png

6. Go to the Debugger tab, and do the following
  • Select Connect Only
  • Check Execute debugger commands and enter the following commands: 
interrupt
restore "u-boot-socfpga/spl/u-boot-spl-dtb.bin" binary 0xFFE00000
loadfile "u-boot-socfpga/spl/u-boot-spl"
set $PC = $ENTRYPOINT
  • Uncheck Host working directory > Use default and edit the value to add "/../" so that it looks in the parent folder of the workspace
The Debug Configurations window should now look like this: agilex-ds-boot-spl-debugger.png

7. Click the Debug button. Arm Development Studio will run the commands, therefore downloading the SPL to board and starting it. The Eclipse window should now look like this: agilex-ds-boot-spl-started.png

8. At this point you can use standard debug techniques to debug U-Boot SPL: viewing registers, variables, putting breakpoints, running step-by-step etc.

Debugging U-Boot

1. Run the steps from Prerequisites section

2. Create the "Debug Agilex Bootloader" debug configuration as described in Debugging SPL section

3. Change the Execute debugger commands box to contain the following commands: 
interrupt
restore "u-boot-socfpga/spl/u-boot-spl-dtb.bin" binary 0xFFE00000
loadfile "u-boot-socfpga/spl/u-boot-spl"
thb board_boot_order
set $PC = $ENTRYPOINT
continue
wait 60s
set spl_boot_list[0]=0
set $PC=$LR
restore "u-boot-socfpga/u-boot.itb" binary 0x2000000
thb el3:0x1000
continue
wait 60s
symbol-file "u-boot-socfpga/u-boot" 
thb el2:relocate_code
continue
wait 60s
symbol-file "u-boot-socfpga/u-boot" ((gd_t*)$x18)->reloc_off
thb board_init_r
continue
wait 60s

What the above does is:
  • Load and run SPL
  • Make SPL report boot from RAM was selected
  • Load ATF+U-Boot image and run it up until U-Boot starts running
  • Load U-Boot symbols
  • Run U-Boot until the relocation routine is called
  • Relocate the U-Boot symbols
  • Run U-Boot until the board_init_r function

4. Click on the Debug button. All the above will be executed, and Eclipse will show the code stopped at board_init_r function. The Eclipse window should look like this: agilex-ds-boot-uboot-started.png

5. At this point you can use standard debug techniques to debug U-Boot SPL: viewing registers, variables, putting breakpoints, running step-by-step etc.

Managing Secure L3 Registers on Stratix 10, Agilex and N5X

On Stratix 10, Agilex and N5X HPS there are specific peripherals which are critical for system operation which can only be accessed from software running at EL3.

The following HPS software components run at EL3 on these devices and can access Secure L3 registers:
  • U-Boot SPL: initial values for the secure L3 registers are set here through the device tree 'secreg' entries. The user can customize them as needed by editing the device tree.
  • Arm Trusted Firmware (ATF): Both U-Boot and Linux call the ATF SMC (Secure Monitor Call) handler to access a restricted subset of secure L3 registers needed for routine system operation.

This section presents the following:
  • How to use the 'secreg' device tree entries to customize initial secure L3 registers values set by U-Boot SPL
  • How to access registers from the restricted subset from U-Boot, for debug purposes.
  • How to access other secure EL3 register from U-Boot, by by changing the ATF source code to add add them to the restricted subset.

Setting Initial Values of Secure L3 Registers

The initial values for the Secure L3 registes are set from U-Boot SPL. The register values are specified in secreg entries in the U-Boot device tree file.

Refer to u-boot-socfpga/blob/HEAD/doc/device-tree-bindings/misc/socfpga_secreg.txt for documentation the "secreg". Current socfpga_v2021.04 version shows the following: 

* Firewall and privilege register settings in device tree

Required properties:
--------------------
- compatible: should contain "intel,socfpga-secreg"
- intel,offset-settings: 32-bit offset address of block register, and then
						 followed by 32-bit value settings.
Example:
--------
		socfpga_secreg: socfpga-secreg {
			compatible = "intel,socfpga-secreg";
			#address-cells = <1>;
			#size-cells = <1>;
			u-boot,dm-pre-reloc;

			i_sys_mgr@ffd12000 {
				reg = <0xffd12000 0x00000228>;
				intel,offset-settings =
					<0x00000020 0xff010000>,
					<0x00000024 0xffffffff>;
				u-boot,dm-pre-reloc;
			};
		};

Notes about the example:
  • The u-boot,dm-pre-reloc; statement in the example informs U-Boot the driver will be loaded in SPL
  • The i_sys_mgr@ffd12000 statement in the example is informative only to enable readers to quickly see what IP is being set up, it is not actually used by the code
  • The reg =<0xffd12000 0x00000228> entry specifies the IP module base address 0xffd12000 and span of 0x00000228 bytes
  • The <0x00000020 0xff010000>, specifies that the register at offset 0x00000020 from the IP module base address will be set to value 0xff010000

These are the files which currently define the initial value of the Secure L3 registers:

You can edit the above files accordingly to change the default values, or set the initial value of more registers.

Accessing Secure L3 Registers from U-Boot Command Line

A small subset of critical EL3 restricted access registers are made visible through the ATF SMC handler. The current list of registers is defined in arm-trusted-firmware/blob/HEAD/plat/intel/soc/common/socfpga_sip_svc.c.

The secure L3 registers accessible through the ATF SMC handler can also optionally be accessed from U-Boot command line for debug purposes. The feature can be enabled by setting "CONFIG_CMD_SMC=y" in the U-Boot configuration file.

Once the feature is enabled, the following command will be avaible from U-Boot command line interface: 

SOCFPGA # smc
smc - Issue a Secure Monitor Call

Usage:
smc  [arg1 … arg6] [id]
  - fid Function ID
  - arg SMC arguments, passed to X1-X6 (default to zero)
  - id  Secure OS ID / Session ID, passed to W7 (defaults to zero)

The U-Boot environment already includes predefined ids to facilitate the usage of the command: 

smc_fid_rd=0xC2000007
smc_fid_upd=0xC2000009
smc_fid_wr=0xC2000008

The command can be used as follows: 

smc ${smc_fid_rd} <address>
smc ${smc_fid_wr} <address> <value>
smc ${smc_fid_upd} <address> <mask> <value>

See below using the new command to access the BOOT_SCRATCH_COLD0 register (note there is no need to access that register, this is just an example):

1. Read the register: 
SOCFPGA_STRATIX10 # smc ${smc_fid_rd} 0xffd12200
Res:  0 400000 4291895808 0
Note:
  • First value from Res is the return code, 0 means operation succesfull
  • Second value represents the read register value in decimal 400000=0x00061a80
  • Third value is the address in decimal 4291895808=0xffd12200

2. Write the register with a new value: 
SOCFPGA_STRATIX10 # smc ${smc_fid_wr} 0xffd12200 0x00061a81
Res:  0 400001 4291895808 0
Note:
  • First value from Res is the return code, 0 means operation succesfull
  • Second value represents the value written in decimal 400001=0x00061a81
  • Third value is the address in decimal 4291895808=0xffd12200

3. Read back the register to confirm it has been updated: 
SOCFPGA_STRATIX10 # smc ${smc_fid_rd} 0xffd12200
Res:  0 400001 4291895808 0

Enabling Access to more Secure L3 Registers for Debug Purposes

By default, only a small subset of critical EL3 restricted access registers are made visible through the ATF SMC handler. The current list of registers is defined in arm-trusted-firmware/blob/HEAD/plat/intel/soc/common/socfpga_sip_svc.c. For debug purposes, you can add more registers to the restricted register list that can be accessed through the ATF SMC handler.

Warning: Changing the list of EL3 restricted access registers in ATF is risky, and must be done only for debug purposes only! Do not forget to remove the code once debugging has completed!

When trying to access a register which is not made visible by the ATF SMC handler, an error will be reported. See below example trying to read the noc_fw_soc2fpga_soc2fpga_scr register: 
SOCFPGA_STRATIX10 # smc ${smc_fid_rd} 0xffd21200
Res:  4 0 4291957248 0
Note:
  • The non-zero (4) return code means the operation was not succesfull.

After editing the file arm-trusted-firmware/blob/HEAD/plat/intel/soc/common/socfpga_sip_svc.c to add this register to the list, and recompiling ATF, the operation is succesfull: 
SOCFPGA_STRATIX10 # smc ${smc_fid_rd} 0xffd21200
Res:  0 268304641 4291957248
Note:
  • Return code is zero, operation was succesfull
  • Read value is decimal 268304641=0xFFE0101

Boot Scratch Register Usage

On Stratix 10 SoC, Agilex 7 and N5X devices, the boot scratch registers are part of the System Manager and are used to pass values between various software components. The table below summarizes the usage:

Note:
  • If no device is provided it means that it applies for all
  • If a cell is not specified for a device, then this could be used as a scratch memory

Address Name Usage SDM U-Boot ATF Linux
0xFFD1 2200 boot_scratch_cold0 Bits[31:31] N5X. DDR retention Sets this bit Read
(is_ddr_retention_enabled) 		   
Bits[30:28] N5X, Agilex7M. DDR reset type Set this field Read
(get_reset_type) 		   
Bits[27:0] SOC 64-bit storing qspi ref clock(kHz) Sets value cm_set_qspi_controller_clk_hz
Reads value cm_get_qspi_controller_clk_hz
0xFFD1 2204 boot_scratch_cold1 osc1 clock freq   Sets and read(cm_get_osc_clk_hz)    
0xFFD1 2208 boot_scratch_cold2 fpga clock freq   Sets and read
(cm_get_fpga_clk_hz) 		   
0xFFD1 220C boot_scratch_cold3 reserved for customer use        
0xFFD1 2210
0xFFD1 2214 		boot_scratch_cold4
boot_scratch_cold5 		Secondary CPU RELEASE ADDRESS   Main core clears it
(lowlevel_init) 		Main CPU Write
(bl31_platform_setup) 		 
0xFFD1 2218
0xFFD1 221C 		boot_scratch_cold6
boot_scratch_cold7 		64-bit signature with L2 reset indication done. Writes signature (__l2_reset_cpu)
Reads the register (lowlevel_init) 		Writes register (socfpga_system_reset2)
Reads register (plat_get_my_entrypoint) 		 
0xFFD1 2220 boot_scratch_cold8 Bit[31:31] N5X, Agilex 7M. DBE status Set by SDM Check if bit is set
(is_ddr_dbe_triggered) 		   
Bit[30:30] N5X, Agilex 7M. DDR Init Progress   Set and clear bit(ddr_init_inprogress)
Read status (is_ddr_init_hang) 		   
Bit[29:29] Agilex 7M. OCRAM_DBE Error status   Set and clear bit (ddr_init_inprogress)
Read status (hps_ocram_dbe_status) 		   
Bits[28:27] Agilex 7M. Number of IO96B instances   Sets this field
(update_io96b_assigned_to_hps) 		   
Bit[19:19] Agilex 7, Agilex 7M, S10. CPU power domain
is about to be turned on. 		    Handled call under event
(socfpga_pwr_domain_on) 		 
Bit[18:18] Agilex 7, S10. ACF DDR Data rate Set this bit Read this value (sdram_mmr_init_full)    
Bit[17,16,1] ECC_DDR1 Error Flag
ECC_DDR0 Error Flag
ECC_OCRAM Error Flag 		    Checks if any of the flags are set
(socfpga_system_reset2) 		Set via ATF SMC
0xFFD1 2224 boot_scratch_cold9         Write (via ATF SMC)

Links:

QSPI Reference Clock

The QSPI peripheral clocks are provided by the SDM, based on the SDM input clocks and configuration clock settings defined in the Quartus project. However, the HPS needs to know the QSPI reference clock, so that it can properly set the dividers in the QSPI controller to create the desired external QSPI clock frequency.

The HPS obtains the QSPI controller reference clock frequency when it obtains exclusive access to the QSPI from the SDM. The frequency reported by the SDM is stored in the U-Boot environment variable called ${qspi_clock}.

Before booting Linux, U-Boot loads the Linux device tree in memory, then runs the command linux_qspi_enable which sets the QSPI controller reference clock appropriately using the value from the ${qspi_clock} environment variable.

Reconfiguring Core Fabric from U-Boot

The GSRD configures the FPGA core fabric only once, from U-Boot, by using the bootm command. The example in this page configures the fabric only once, from U-Boot, using fpga load command.

Important : If the FPGA fabric is already configured and bridges are enabled, you must call the bridge disable command from U-Boot before issuing the bootm or fppga load commands to reconfigure the fabric. Only do this if you are using an arm-trusted-firmware version more recent than the following:

Enabling Bridges from U-Boot

U-Boot offers the bridge command for controlling the bridges.

The bridge command can be used with either 'enable' or 'disable' parameter, followed by an optional 'mask' parameter indicating which of the bridges needs to be enabled or disabled. When the 'mask' parameter is omitted, all bridges are either enabled or disabled.

See bellow the help message for the command: 

# bridge
bridge - SoCFPGA HPS FPGA bridge control

Usage:
bridge enable [mask] - Enable HPS-to-FPGA (Bit 0), LWHPS-to-FPGA (Bit 1), FPGA-to-HPS (Bit 2), F2SDRAM0 (Bit 3), F2SDRAM1 (Bit 4), F2SDRAM2 (Bit 5) bridges 
bridge disable [mask] - Disable HPS-to-FPGA (Bit 0), LWHPS-to-FPGA (Bit 1), FPGA-to-HPS (Bit 2), F2SDRAM0 (Bit 3), F2SDRAM1 (Bit 4), F2SDRAM2 (Bit 5) bridges
Bit 3, Bit 4 and Bit 5 bridges only available in Stratix 10

The 'mask' is a hexadecimal number, with 3 bits available for Agilex 7, and 6 bits for Stratix 10, as indicated above.

The following table shows examples of enabling and disabling various bridges:

Command Description 
bridge enable
 Enable all bridges 
bridge disable
 Disable all bridges 
bridge enable 1
 Enable HPS-to-FPGA bridge 
bridge enable 2
 Enable LWHPS-to-FPGA bridge 
bridge enable 4
 Enable FPGA-to-HPS bridge 
bridge enable 7
 Enable HPS-to-FPGA, LWHPS-to-FPGA, FPGA-to-HPS bridges 
bridge enable 35
 Enable HPS-to-FPGA, FPGA-to-HPS, F2SDRAM1, F2SDRAM2 bridges
(Stratix10 only) 
bridge disable 30
 Disable F2SDRAM1, F2SDRAM2 bridges
(Stratix10 only)

© 1999-2024 RocketBoards.org by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Privacy Policy - Terms Of Use

This website is using cookies. More info. That's Fine