Projects

Recent Changes

Intel® Agilex® 7 I-Series Transceiver-SoC Development Kit (4x F-Tile) System Example Design With IEEE1588 PTP (25GE/10GE 2-Port)
Multiprocessor Nios V/m system: creating your first GeMRTOS application
Quick Start Guide on how to create a multiprocessor system with Nios V processor and GeMRTOS support
Setting up and Using Bridges on Agilex 7 SoC
Agilex 7 SoC With eMMC Storage On HPS
Stratix 10 SoC with eMMC Storage on HPS

Required Components
Helpful Reference Documentation
Release Contents
Hardware Description
Hardware Setup
Boot the System
Rebuild Source
Similar Projects

You are here: Projects » Reference Design » Arria 10 PCIe Root Port with MSI

Arria 10 PCIe Root Port with MSI
Reference Design on Arria 10 Development Kit demonstrating a Gen2x4 Capable PCIe Root Port.

28 September 2016 - 07:23 | Version 75 | Michael Daum | Reference Design

Board:	Arria10SoCDevelopmentKit
Tools Version:	15.1.1
State:	running
Members:	DavidKoltak

Summary

This reference design demonstrates a PCIe root port running on an Arria 10 SoC Development Kit connected to either a Cyclone V GT FPGA Development Kit PCIe end point or a generally available Intel PCIe Ethernet adapter card end point. A Gen2 x4 PCIe link is shown.

The root port reference design is based on the Arria 10 Golden System Reference Design, which has been modified to add the PCIe root port, benchmarking logic, testcase software, and necessary Linux software infrastructure.

Required Components

Root Port Host Board
- Arria10 SoC DevKit (Rev B.1)
Example End Points
- Cyclone V GT FPGA DevKit
- Intel i350 Ethernet x4 PCIe Card
Pre-compiled Software/Firmware
- SD Card Image
- Cyclone V GT FPGA End Point SOF
Tools and Software
- Linux Development Computer (Ubuntu, CentOS, or similar) with an SD Card reader
- Quartus FPGA Programmer
- A serial terminal application, such as Putty or Minicom

Helpful Reference Documentation

Release Contents

The PCIe RP reference design sources and prebuilt binaries can be downloaded from here.

Folder	File	Description
bin	linux-socfpga-pcierd-15.1.1-a10-bin.tar.gz	Arria 10 binaries archive (including SD Card Image)
hw	cv_ep_ram_gt_design.tar.gz	Cyclone V DMA End Point Design (including SOF file)
^	a10_pcie_soc_devkit.tar.gz	Arria 10 PCIe Root Port Design
src	ghrd_10as066n2.dts	Device tree source

U-boot, Linux kernel and Yocto source packages are also provided through the git trees at https://github.com/altera-opensource, as shown in the table below.

Component	Git Address	Branch	Tag
Linux	linux-socfpga.git	socfpga-4.1	ACDS15.1.1_REL_PCIE_PR
Angstrom	angstrom-socfpga.git	angstrom-v2014.12-socfpga_gsrd15.1	ACDS15.1.1_REL_PCIE_PR
Ref Design	linux-refdesigns.git	socfpga-15.1	ACDS15.1.1_REL_PCIE_PR

Hardware Description

Root Port Design

The root port reference design hardware is shown in the diagram below. This design is based on the Arria 10 SoC Development Kit Golden Hardware Reference Design (GHRD), which is part of the Golden System Reference Design (GSRD). A PCIe Root Port Subsystem and 256kB on chip RAM have been added. Please refer to the QSYS files included with the Root Port Hardware Design (listed in the Build Arria 10 PCIe Root Port section below) for further detail. A few more components, such as latency counters and JTAG Avalon Masters, have been added to the design for debug and benchmarking but are not shown here.

	The Hard Processor System (HPS) lightweight H2F AXI bridge is connected to the Modular Scatter-Gather DMA (mSGDMA), MSI-to-GIC module, and PCIe Hard IP (HIP) CRA slave ports. The HPS H2F AXI bridge is connected to the 256kB on chip RAM and to the PCIe HIP TXS slave port. These two buses provide access to the 256kB RAM, mSGDMA, MSI-to-GIC, and PCIe HIP from the HPS. The mSGDMA can move data between the PCIe HIP TXS slave port, 256kB RAM, and system 1GB SDRAM. The mSGDMA accesses the system 1GB SDRAM through an Altera Address Expander module. Interrupts from the mSGDMA, PCIe HIP, and an MSI-to-GIC module have been routed to the F2H IRQ input port on the HPS. The MSI-to-GIC module translates Message Signal Interrupts from the PCIe HIP module BAR0 master port, which are PCIe transactions to specific addresses with specific data, into HPS Global Interrupt Controller (GIC) compatible interrupt signals.
The PCIe HIP module in the Arria 10 SoC is configured to operate as a root port. It provides Avalon slave bus interfaces to initiate configuration space (CRA) and memory space (TXS) accesses. It also provides Avalon master bus interfaces (BARx) to allow PCIe bus master devices to access SoC FPGA resources. Please refer to the PCIe HIP User's Guide for more information. The PCIe root port has been configured for Generation 2 speed and x4 lanes (Gen2 x4). One Base Address Register (BAR0) is enabled to allow PCIe bus master devices to access the system 1GB SDRAM and the 256kB on chip RAM, as well as route MSI messages to the HPS GIC.
.

HPS (H2F and LWH2F) Memory Map

Address Offset	Size (Bytes)	Peripheral	Remarks
0xC0000000	256k	On Chip Memory	Block memory implemented in the FPGA fabric
0xD0000000	128M	PCIe TXS	Avalon MM Slave of PCIe HIP TXS port
0xFF200000	8	System ID	Hardware configuration system ID
0xFF200010	16	LED PIO
0xFF200020	16	Button PIO	Push Button
0xFF200030	16	DIPSW PIO	DIP Switch
0xFF200100	256	ILC	Interrupt Latency Counter
0xFF210000	16k	PCIe CRA	Avalon MM Slave of PCIe HIP CRA port
0xFF214000	128	MSI-to-GIC Vector
0xFF214080	16	MSI-to-GIC CSR	Avalon MM Slave of MSI-to-GIC CSR port
0xFF2140A0	32	Performance Counter	Hardware timer for benchmarking purposes
0xFF2140C0	32	mSGDMA CSR	Modular Scatter-Gather DMA CSR
0xFF2140E0	32	mSGDMA DESC	Modular Scatter-Gather DMA Descriptors

PCIe HIP BAR0 Memory Map

Address Offset	Size (Bytes)	Peripheral	Remarks
0x00000000	1G	HPS F2SDRAM
0x40000000	256k	On Chip Memory	Block memory implemented in the FPGA fabric
0xFF214000	128	MSI-to-GIC	Served as system-specified address for MSI function during device configuration. This address must be aligned with HPS view of system-specified address for MSI

mSGDMA Memory Map

Address Offset	Size (Bytes)	Peripheral	Remarks
0x00000000	1G	HPS F2SDRAM
0x40000000	256k	On Chip Memory	Block memory implemented in the FPGA fabric
0xD0000000	128M	PCIe TXS	Avalon MM Slave of PCIe HIP TXS port

Cyclone V GT FPGA End Point Example

The example FPGA based end point provided with this reference design is targeted for a Cyclone V GT FPGA Development Kit (5CGTFD9). The design implements a Gen2 x4 PCIe end point using the Altera PCIe HIP module. A Modular Scatter-Gather DMA (mSGDMA) and a 256kB on chip RAM are also implemented to help demonstrate throughput in both the upstream and downstream data transfer directions. A few more components, such as JTAG Avalon Masters, have also been implemented in the design for debug and benchmarking but are not shown here. Please refer to the QSYS files included with the End Point Hardware Design (listed in the Build Cyclone V GT PCIe End Point section below) for further detail.

.

Intel PCIe Ethernet Adapter End Point

The Intel Ethernet Server Adapter I350 End point is a quad-port Gigabit Ethernet server adapters with PCIe 2.0 x4 lanes. More information about Intel Ethernet I350 adapter can be found here.

Hardware Setup

Setup the Arria 10 SoC Development Kit to the default settings as described in Configuring Board of Booting Linux Using Prebuilt SD Card Image in Arria 10 GSRD User Manual. Attach a MicroSD card adapter to J23 on the Arria 10 SoC Development Kit to eventually accept the boot card that will be created in the following steps. Connect J10 ("UART1") of the Arria 10 SoC Development Kit to a host computer running a serial terminal application using a MiniUSB cable.

The Arria 10 Root Port Reference Design can be run with either the Cyclone V FPGA end point example or with an Intel PCIe Ethernet PCIe end point.

Setup Cyclone V GT PCIe End Point

If using the Cyclone V GT FPGA Example End Point, refer to Figure 4-1 of Cyclone V GT FPGA Development Kit User's Guide to set up the FPGA board to the default jumper settings before inserting it into the PCIe slot (J57) of the Arria 10 SoC Development Kit. Using a MiniUSB cable, connect J5 of the Cyclone V GT FPGA Development Kit to a computer with Quartus Programmer installed and with the Cyclone V FPGA End Point SOF, as listed in the Pre-Compiled Software/Firmware section above, downloaded and available.

Setup Intel PCIe Ethernet Adapter End Point

If using an Intel Ethernet PCIe end point, simply insert the card into the PCIe slot (J57) of the Arria 10 SoC Development Kit.

Create SD Card for Arria 10 SoC Development Kit

This section describes how to create an SD Card that contains the default pre-built version of this reference design. Please refer to the Rebuild Source section below and the Arria 10 GSRD User's Guide for further instructions on how to completely rebuild all of the individual components contained in the SD Card image, including the bootloader and root file system, from scratch.

On a Linux based development computer...

1.) Download the SD Card Image as listed in the Pre-Compiled Software/Firmware section above
2.) Use tar xvfz sdimage.tar.gz to extract the SD Card binary image (sdimage.img)
3.) Insert the SD Card to be updated into an SD Card reader attached to the computer
4.) Use dd if=sdimage.img of=/dev/{sdcard} to copy the SD Card binary image to the SD Card device

Boot the System

With the Arria 10 Soc Development Kit powered down, insert the SD Card created above in Create SD Card for Arria 10 SoC Development Kit into the SD Card slot. Open a serial port terminal application on the host PC connected via MiniUSB cable to the Arria 10 Development Kit (115200,8,1,none). Turn on the Arria 10 SoC Development Kit and you should see the UBoot bootloader start...

U-Boot 2014.10 (Dec 22 2015 - 02:04:02)

CPU   : Altera SOCFPGA Arria 10 Platform
BOARD : Altera SOCFPGA Arria 10 Dev Kit
DRAM:  WARNING: Caches not enabled
SOCFPGA DWMMC: 0
FPGA: writing ghrd_10as066n2.rbf
FPGA: Success.emif_reset interrupt acknowledged
DDRCAL: Success
INFO  : Skip relocation as SDRAM is non secure memory
Reserving 2048 Bytes for IRQ stack at: ffe2db10
DRAM  : 1 GiB
WARNING: Caches not enabled
MMC:   *** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Model: SOCFPGA Arria10 Dev Kit
Net:   dwmac.ff800000
Error: dwmac.ff800000 address not set.

Hit any key to stop autoboot:  0
** Unable to read file u-boot.scr **
Optional boot script not found. Continuing to boot normally
4020240 bytes read in 139 ms (27.6 MiB/s)
25277 bytes read in 4 ms (6 MiB/s)
FPGA BRIDGES: enable
Kernel image @ 0x008000 [ 0x000000 - 0x3d5810 ]
## Flattened Device Tree blob at 00000100
   Booting using the fdt blob at 0x000100
   Loading Device Tree to 01ff6000, end 01fff2bc ... OK

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpuset

           (...............)

         Starting Permit User Sessions...
[  OK  ] Started Permit User Sessions.
[  OK  ] Started Getty on tty1.
[  OK  ] Started Serial Getty on ttyS0.
[  OK  ] Reached target Login Prompts.
[  OK  ] Reached target Multi-User System.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.
[   31.237492] eth0: device MAC address 52:49:6a:cf:ab:26

.---O---.
|       |                  .-.           o o
|   |   |-----.-----.-----.| |   .----..-----.-----.
|       |     | __  |  ---'| '--.|  .-'|     |     |
|   |   |  |  |     |---  ||  --'|  |  |  '  | | | |
'---'---'--'--'--.  |-----''----''--'  '-----'-'-'-'
                -'  |
                '---'

The Angstrom Distribution arria10 ttyS0

Angstrom v2014.12 - Kernel 4.1.0-00203-g6de99ee-dirty

arria10 login:

If using the Cyclone V GT FPGA End Point, it still needs to have the end point example design file loaded. The image can be loaded while the system is running. Program the Cyclone V GT FPGA End Point SOF listed in the Required Components section above with the Quartus Programmer from the host PC connected via MiniUSB cable to J5 of the Cyclone V GT FPGA Development Kit. Once the end point design is loaded, perform a WARM reset on the Arria 10 SoC Development Kit by pressing S1 ("warm pb"). UBoot will restart and Linux will be reloaded.

If using an Intel PCIe Ethernet end point, just let UBoot load Linux as usual.

Once the system has booted, log in by entering "root" as the username...

.---O---.
|       |                  .-.           o o
|   |   |-----.-----.-----.| |   .----..-----.-----.
|       |     | __  |  ---'| '--.|  .-'|     |     |
|   |   |  |  |     |---  ||  --'|  |  |  '  | | | |
'---'---'--'--'--.  |-----''----''--'  '-----'-'-'-'
                -'  |
                '---'

The Angstrom Distribution arria10 ttyS0

Angstrom v2014.12 - Kernel 4.1.0-00203-g6de99ee-dirty

arria10 login: root
Last login: Tue Dec 29 04:48:28 UTC 2015 on ttyS0
root@arria10:~#

Cyclone V GT End Point Benchmarking

The default SD Card image contains software to perform benchmark tests using the Cyclone V GT FPGA PCIe End Point design. The benchmarking software will use the root port and end point mSGDMA modules to move data to and from the system SDRAM and the on-chip RAM and report the results in a table.

First, check that the FPGA PCIe end point has been discovered by the PCIe root port and that the proper Linux kernel modules have been loaded. Use the following command and ensure that the Altera Corporation Device e001 and altera_epdma entries appear as shown below:

root@arria10:~# /usr/sbin/lspci
00:00.0 PCI bridge: Altera Corporation Device e000 (rev 01)
01:00.0 Unassigned class [ff00]: Altera Corporation Device e001 (rev 0a)
root@arria10:~# lsmod
Module                  Size  Used by
altera_epdma            4972  0
gpio_altera             4277  4
altera_sysid            1875  0
altera_rpdma            5581  0

Next, run the benchmarking software with the following commands …

root@arria10:~# cd altera
root@arria10:~/altera# ls
dmaxfer
root@arria10:~/altera# ./dmaxfer

==================================================
PCIe throughput test
  RP-OCM = Rootport On-Chip RAM
  EP-OCM = Endpoint On-Chip RAM
  RP-SYS = Rootport System Memory
==================================================

                Source          Destination     Results (MB/s)
-----------------------------------------------------------
  RP-DMA TX     RP-OCM          EP-OCM          ####
  RP-DMA RX     EP-OCM          RP-OCM          ####
  RP-DMA TX     RP-SYS          EP-OCM          ####
  RP-DMA RX     EP-OCM          RP-SYS          ####
-----------------------------------------------------------
  EP-DMA TX     EP-OCM          RP-OCM          ####
  EP-DMA RX     RP-OCM          EP-OCM          ####
  EP-DMA TX     EP-OCM          RP-SYS          ####
  EP-DMA RX     RP-SYS          EP-OCM          ####
root@arria10:~/altera#

The "####" columns above will show the benchmarking results.

The FPGA end point design is used to perform Gen2 x4 performance benchmark testing. The mSGDMA in the root point (RP-DMA) is used to push and pull data from the FPGA end point, and the mSGDMA in the FPGA end point (EP-DMA) is used to push and pull data from the root port. Source and destination for data transfers can be root port external system memory (RP-SYS), root port on-chip memory (RP-OCM), and end point on-chip memory (EP-OCM).

Intel i350 Ethernet x4 PCIe Card Benchmarking

First, check that the Intel i350 Ethernet x4 PCIe Card has been discovered by the PCIe root port and that the proper Linux kernel modules have been loaded. Use the following command and ensure that the Intel Corporation … entries appear as shown below …

root@arria10:~# /usr/sbin/lspci
00:00.0 PCI bridge: Altera Corporation Device e000 (rev 01)
01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

Here is an example of the output that ifconfig should provide (NOTE: it should show 4 interfaces enp1s0f0 - enp1s0f3) …

root@arria10# ifconfig

enp1s0f0  Link encap:Ethernet  HWaddr 00:1b:21:09:14:00  
          inet addr:xxx.xxx.xxx.xxx  Bcast:xxx.xxx.xxx.xxx  Mask:255.255.255.0
          inet6 addr: fe80::21b:21ff:fe09:1400/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST DYNAMIC  MTU:1500  Metric:1
          RX packets:18 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2631 (2.5 KiB)  TX bytes:7516 (7.3 KiB)
          Memory:d0000000-d00fffff 

enp1s0f1  Link encap:Ethernet  HWaddr 00:1b:21:09:14:01  
          UP BROADCAST MULTICAST DYNAMIC  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:d0100000-d01fffff 

enp1s0f2  Link encap:Ethernet  HWaddr 00:1b:21:09:14:02  
          UP BROADCAST MULTICAST DYNAMIC  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:d0200000-d02fffff 

enp1s0f3  Link encap:Ethernet  HWaddr 00:1b:21:09:14:03  
          UP BROADCAST MULTICAST DYNAMIC  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:d0300000-d03fffff 

eth0      Link encap:Ethernet  HWaddr 56:2d:a9:54:f9:8f  
          inet6 addr: fe80::542d:a9ff:fe54:f98f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST DYNAMIC  MTU:1500  Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2025 (1.9 KiB)  TX bytes:2965 (2.8 KiB)
          Interrupt:34 Base address:0x8000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:172 errors:0 dropped:0 overruns:0 frame:0
          TX packets:172 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:13244 (12.9 KiB)  TX bytes:13244 (12.9 KiB)

The default SD Card image contains the iperf test software. This test software typically requires a host machine to run an iperf server with which the iperf test system can communicate. The Ethernet Design Example Demo Testing page provides an example of how to perform this testing. Please refer to iPerf 2 user documentation for more information about using it to test the networking performance.

root@arria10:~# iperf --version
iperf version 2.0.5 (08 Jul 2010) pthreads
root@arria10:~#

Here is some example iperf output …

Upstream result (i350 as client):

Run "iperf -s" on service side before run command below.

root@arria10# iperf -c <server-ip>
------------------------------------------------------------
Client connecting to 137.57.162.43, TCP port 5001
TCP window size: 43.8 KByte (default)
------------------------------------------------------------
[  3] local xxx.xxx.xxx.xxx port yyyyy connected with xxx.xxx.xxx.xxx port yyyyy
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  #.## GBytes   #### Mbits/sec

Downstream result (i350 as server):

Run "iperf -s" on Arria 10 console as below and run "iperf -c <server-ip>" on the client side.

root@arria10# iperf -s 
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local xxx.xxx.xxx.xxx port yyyyy connected with xxx.xxx.xxx.xxx port yyyyyy
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  #.## GBytes   #### Mbits/sec

Rebuild Source

The instructions below are optional. They show how to rebuild the software and/or hardware if you want to experiment with the reference design. This reference design is a modified version of the Golden System Reference Design, so please refer to A10 GSRD v15.1.1 - User Manual for more details regarding the build flow. It is highly recommended that you perform all of the relevant compilation steps listed in in the user's manual before continuing. The following steps are only meant to be a minimal list.

Build Arria 10 PCIe Root Port

Download the Project:

The project is available here: a10_pcie_soc_devkit.tar.gz

Build the project:

The root port design is based on the Arria 10 GHRD.
Follow the instructions listed at Compiling Hardware Design, but replace the GHRD source code file a10_soc_devkit_ghrd.tar.gz with the file listed above.

Build Cyclone V GT PCIe End Point

Download the Project:

The project is available here: cv_ep_ram_gt_design.tar.gz

Build the project:

Perform a standard Quartus build flow (Generate system using the Qsys tool, then compile to generate the SOF file)

Generating U-Boot and U-Boot Device Tree

Refer to steps in here, but use Arria 10 PCIe Root Port design's hps_isw_handoff, generated in the Build Arria 10 PCIe Root Port section above, instead.

A brief list of steps...

$ ~/altera/15.1/embedded/embedded_command_shell.sh
$ bsp-editor
      (NOTE: Follow steps from GSRD User Manual link above)
$ cd /path/to/a10_pcie_soc_devkit/software/uboot_bsp
$ make

Generating the Linux Device Tree

Device tree generator (sopc2dts) doesn't support PCIe node generation in SoCEDS 15.1.1. However, we still can use sopc2dts to generate device tree for other components in the system and modify the *.dts file to include PCIe node. Please refer to ghrd_10as066n2.dts for reference.

A brief list of steps...

$ ~/altera/15.1.1/embedded/embedded_command_shell.sh
$ cd /path/to/a10_pcie_soc_devkit/

$ sopc2dts --input ghrd_10as066n2.sopcinfo --output ghrd_10as066n2.dts  --board hps_a10_common_board_info.xml --board hps_
a10_devkit_board_info.xml --board ghrd_10as066n2_board_info.xml --bridge-removal all --clocks

Edit 'ghrd_10as066n2.dts' to move pcie_0_pcie_a10_hip_avmm node to under sopc0 node. Change reg-names = "Txs", "Cra".

pcie_0_pcie_a10_hip_avmm: pcie@0x010000000 {
   compatible = "altr,pcie-root-port-15.1", "altr,pcie-root-port-1.0";
   reg = <0xd0000000 0x10000000>,
      <0xff210000 0x00004000>;
   reg-names = "<b>Txs</b>", "<b>Cra</b>";
   interrupt-parent = <&arria10_hps_0_arm_gic_0>;
   interrupts = <0 24 4>;
   interrupt-controller;
   #interrupt-cells = <1>;
   device_type = "pci";
   msi-parent = <&pcie_0_msi_to_gic_gen_0>;
   bus-range = <0x00000000 0x000000ff>;
   #address-cells = <3>;
   #size-cells = <2>;
   ranges = <0x82000000 0x00000000 0x00000000 0xd0000000 0x00000000 0x10000000>;
   interrupt-map-mask = <0 0 0 7>;
   interrupt-map = <0 0 0 1 &pcie_0_pcie_a10_hip_avmm 1>,
         <0 0 0 2 &pcie_0_pcie_a10_hip_avmm 2>,
         <0 0 0 3 &pcie_0_pcie_a10_hip_avmm 3>,
         <0 0 0 4 &pcie_0_pcie_a10_hip_avmm 4>;
};

$ dtc -I dts -O dtb -o ghrd_10as066n2.dtb ghrd_10as066n2.dts

Compile Linux

The following are required:

Host PC running Linux (CentOS 6.6 was used for testing)
Access to Internet (in order to get the toolchain and the Linux kernel git trees)

Getting the toolchain:

$ cd ~
$ wget https://releases.linaro.org/14.04/components/toolchain/binaries/gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux.tar.bz2
$ tar xjf gcc-linaro-arm-linux-gnueabihf-4.8-2013.12_linux.tar.bz2
$ export CROSS_COMPILE=~/gcc-linaro-arm-linux-gnueabihf-4.8-2013.12_linux/bin/arm-linux-gnueabihf-
$ export ARCH=arm

NOTE: Newer releases and 32/64-bit releases of the tool chain can be found here.

Build the Linux Kernel and Modules:

Note that the branch tag has been changed to ACDS15.1.1_REL_PCIE_PR.

$ cd ~
$ git clone https://github.com/altera-opensource/linux-socfpga.git
$ cd linux-socfpga
$ git checkout -b pcie_rebuild ACDS15.1.1_REL_PCIE_PR <- NOTE: Changed from GSRD to PCIe
$ make socfpga_defconfig
$ make zImage
$ cp zImage /path/to/sdcard/vfat/partition/zImage
$ make modules
$ make modules_install $(INSTALL_MOD_PATH)=./libs_rebuild/
$ sudo cp -r ./libs_rebuild/ /path/to/sdcard/ext3/partition/libs/

Build Extra Modules:

$ cd ~
$ git clone https://github.com/altera-opensource/linux-refdesigns.git
$ cd linux-refdesigns
$ git checkout -b pcie_rebuild ACDS15.1.1_REL_PCIE_PR
$ cd pcie/modules
$ export KERNEL_SRC=~/linux-socfpga/
$ make
$ mkdir -p /path/to/sdcard/ext3/partition/libs/modules/{kernel_name}/extra
$ sudo cp *.ko /path/to/sdcard/ext3/partition/libs/modules/{kernel_name}/extra/

</verbatim>

Build Linux Kernel and Rootfs using Yocto

Please follow all of the steps listed in the Compiling Linux Kernel and Root Filesystem section of the A10 GSRD v15.1.1 User Manual before continuing. This will ensure that your development computer is setup correctly with all of the necessary build tools available.

The following steps show the final steps necessary to perform the Angstrom distribution build. Note that the branch tag has been changed to ACDS15.1.1_REL_PCIE_PR and the final line has been modified to use the pcie-console-image recipe.

A brief list of steps...

$ cd ~
$ git clone https://github.com/altera-opensource/angstrom-socfpga.git
$ cd angstrom-socfpga/
$ git checkout -b pcie_rebuild ACDS15.1.1_REL_PCIE_PR <- NOTE: Changed from GSRD to PCIe
$ MACHINE=arria10 ./oebb.sh config arria10
$ source ./environment-angstrom-v2014.12
$ export KERNEL_BRANCH="socfpga-4.1"
$ export KERNEL_PROVIDER="linux-altera"
$ export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE KERNEL_TAG KERNEL_BRANCH KERNEL_PROVIDER"
$ export KERNEL_TAG=refs/tags/ACDS15.1.1_REL_PCIE_PR
$ MACHINE=arria10 bitbake virtual/kernel pcie-console-image <- NOTE: Changed from GSRD to build PCIe drivers and software

NOTE: Refer to Step 6: Build Linux Kernel and Rootfs Using Yocto Recipes for more details and comments on resolving any issues that may occur

Creating and Updating the SD Card

Please refer to these instructionsfor further detail about creating an SD Card image.

A brief list of steps...

make_sdimage.py can get from here.

$ mkdir rootfs
$ cd  rootfs
$ sudo tar xzf ../gsrd-console-image-arria10.tar.xz
$ cd ..
$ sudo make_sdimage.py  \
  -f \
  -P uboot_w_dtb-mkpimage.bin,num=3,format=raw,size=10M,type=A2  \
  -P rootfs/*,num=2,format=ext3,size=1500M  \
  -P zImage,ghrd_10as066n2.rbf,socfpga_arria10_socdk.dtb,num=1,format=vfat,size=500M  \
  -s 2G  \
  -n sd_card_image_a10.img

Similar Projects

PCIe Root Port with MSI (using Cyclone V)
Arria V PCIe Root Port with MSI

This website is using cookies. More info. That's Fine

Projects

Recent Changes

Table of Contents

Arria 10 PCIe Root Port with MSI Reference Design on Arria 10 Development Kit demonstrating a Gen2x4 Capable PCIe Root Port.

Summary

Required Components

Helpful Reference Documentation

Release Contents

Hardware Description

Root Port Design

Cyclone V GT FPGA End Point Example

Intel PCIe Ethernet Adapter End Point

Hardware Setup

Setup Cyclone V GT PCIe End Point

Setup Intel PCIe Ethernet Adapter End Point

Create SD Card for Arria 10 SoC Development Kit

Boot the System

Cyclone V GT End Point Benchmarking

Intel i350 Ethernet x4 PCIe Card Benchmarking

Rebuild Source

Build Arria 10 PCIe Root Port

Build Cyclone V GT PCIe End Point

Generating U-Boot and U-Boot Device Tree

Generating the Linux Device Tree

Compile Linux

Build Linux Kernel and Rootfs using Yocto

Creating and Updating the SD Card

Similar Projects

Arria 10 PCIe Root Port with MSI
Reference Design on Arria 10 Development Kit demonstrating a Gen2x4 Capable PCIe Root Port.