PCIe endpoint setup on Xavier

Hi,

I am new to embeded system. I am trying to enable PCIe endpoint mode on a Xavier and get confused for reflashing L4T.

First, I checked my current L4T version, which is R32.3.1, as shown below:

And I also trid to search the file tegra-pcie-eo-mem.c, but got “No such file or directory” as shown above.

Then, I tried to follow the doc, NVIDIA JETSON AGX XAVIER PCIE ENDPOINT SOFTWARE FOR L4T(released May 1, 2019).

There are two questions here:

  1. On section 1.3, it says " All commands described in this application note must be run as root. Some commands use shell I/O redirection, and will not operate correctly if run using sudo." What does the last sentence mean? All I know is that if I want to run as root, I need to use “sudo”, not sure why using “sudo” can be incorrect if we still want to run as root.
  2. On section 1.4, in step 1, the ODM data in the extracted L4T release directory needs to be changed. My question is where the L4T directory is. As I can check my L4T version, I thought I should already get the L4T directory, but I am not sure what keywords I should use to find it, not sure where the folder exactly locates. Although in section 1.1, it mentions “nvidia/drivers/pci/dwc/pcie-tegra.c” and “kernel-4.9/drivers/pci/dwc/”. But I am still not sure what the absolute path is for L4T directory. And I also ran "sudo find pcie-tegra.c"under my desktop folder, i.e. /home/my_hostname, but got message “No such file or directory”.

Thanks so much for the help!

Hi,

We don’t put the source code on the tegra device. You need to download it from our dlc.

in order to run Xavier AGX C5 controller in EP mode you need to modified the file Linux_for_Tegra/p2972-0000.conf.common, and set the ODMDATA=0x9190000 to ODMDATA=0x9191000.

above change is only required to flash Xavier AGX that you are planing to run in EP mode.

if you have other Xavier that you plan to use as host mode, you should flash with default ODMDATA

1 Like

Hi WaynewWWW,

Thanks for the reply. May I ask which file I should choose to download? I found there is a post that saying downloading “L4T source” and “L4T Jetson driver package”. Not quite sure whether I should download both or “L4T source” is enough. And I am not sure what the relationship between these two. Do I already have “L4T Jetson driver package” as I can check L4T version on my board?

I am also not quite sure how to build the kernel with the downloaded files. I only know that I need to download something(which I am not sure what should be) and unzip it. But after that, will I have Linux_for_Tegra folder directly? Then I know I should modify ODMDATA in it for EP mode. Finally, I should do “sudo ./flash.sh jetson-xavier mmcblk0p1” to refresh. Btw, the flash.sh is in Linux_for_Tegra folder, right?

Thanks!

Hi,

L4T source is the driver code.
L4T Jetson driver package is the Linux_for_Tegra folder that already installed by sdkmanager.

To build kernel, please refer to

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide%2Fkernel_custom.html%23

Hi WayneWWW,

Thanks for the reply. I am a bit confused about driver code and kernel code. Sorry for asking questions again, I am not very familiar with the kernel and driver concepts.

  1. Can I say that L4T is the driver, which I should download its source code, i.e. L4T source, as it is not added initially?

  2. In my case(enable PCIe endpoint mode), I dont need to rebuild the kernel, right? I just need to change something in L4T source, and then reflash driver code, right?

  3. In terms of the folder location, I cannot find Linux_for_Tegra folder in root directory. May I ask where it is? (This is what I feel confusing) And where I should download the source code? In root directory? And what command should I use, i.e.

    sudo apt install < what should be here? >

Thanks!

Hi,

The dlc link here:

https://developer.nvidia.com/embedded/downloads

In my case(enable PCIe endpoint mode), I dont need to rebuild the kernel, right? I just need to change something in L4T source, and then reflash driver code, right?

This description is not fully correct. Why do you think you could download some source code and don’t need to rebuild it to get something update?

For newbie to jetson platform, I think you need to firstly know what is on the host machine and what is one the device.

I cannot find Linux_for_Tegra folder in root directory.

Because Linux_for_Tegra is on the ubuntu host machine, you won’t find it on your jetson device. We use tools under this folder to flash the board.

And where I should download the source code? In root directory? And what command should I use, i.e.

It is a tarball on dlc. You could download it to your host machine.

Actually, I just read the PCIe EP document. You don’t need to rebuild the kernel image. Just need to modify some configuration and re-flash your board.

Please note that there are plenty of files in driver package. The code you are trying to download is just the kernel driver. The document mentions it just for someone who wants take a look for this driver code.

If you don’t need to modify this driver, then you don’t need to download it at all.

Hi WayneWWW,

I got the point. So actually the “L4T source” is the kernel driver, right? And since I am dealing with PCIe endpoint, there is no need to download “L4T source”, right?

May I ask why if I change the configuration, I dont need to rebuild the kernel image? What does re-flash refer to with the modified configuration? i.e. when I have my board connected in recovery mode and do re-flash(sudo ./flash.sh jetson-xavier mmcblk0p1), what should I expect?

Thanks!

I got the point. So actually the “ L4T source ” is the kernel driver, right? And since I am dealing with PCIe endpoint, there is no need to download “L4T source”, right?

L4T source is not only kernel driver but also some other components. But it is correct you don’t need to download and build it.

May I ask why if I change the configuration, I dont need to rebuild the kernel image?

According to the PCIe ep document, the answer is yes.

What does re-flash refer to with the modified configuration? i.e. when I have my board connected in recovery mode and do re-flash( sudo ./flash.sh jetson-xavier mmcblk0p1 ), what should I expect?

Please follow the steps on PCIe ep doc and do the refalsh, it will erase all data on your module and re-install the ubuntu again.

1 Like

Hi WayneWWW,

Thanks for the reply. I have checked the host machine, but found

  1. I dont have Linux_for_Tegra folder

Not sure whether I actually need to download L4T.

So I downloaded L4T source, but in the downloaded L4T, I have following directory:


where I didn’t find file. p2972-0000.conf.common

Not sure which step I did wrong.

  1. I have one folder, called nvidia on host machine, which seems related due to this information

    But in my nvidia folder, I only have sdkm_downloads. Not sure whether this is what should be expected.

Thanks!

Hi,

Have you ever installed your jetson with SDKmanager ever once? I mean to complete the installation of your board so that it can boot into ubuntu OS.

I guess you got misled by the file name. Sorry that I don’t know the public source would also have “Linux_for_Tegra” too.

Below is my path to the file you need.

~/nvidia/nvidia_sdk/JetPack_4.3_Linux_P2888-0060/Linux_for_Tegra$ ls -al p2972-0000.conf.common
-rw-rw-rw- 1 waywang waywang 8268 Dec 10 2019 p2972-0000.conf.common

Please note there is one line just prior to the pcie-tegra.c and pointing out it is in “l4t kernel source”.
So pcie-tegra.c is actually in the first screenshot of your file.

Also I notice you are using mac as host OS… SDKmanager does not support other OS but only ubuntu.

Actually, what you need to do is just flash your board with sdkmanager once. Then, your host will have Linux_for_Tegra and p2972-0000.conf.common. No need to worry about where is tegra-pcie.c at all. You could read the document and the document does not give you any instructions on pcie-tegra.c at all.

This is just the driver code for someone who wants to refer to.

1 Like

Hi WayneWWW,

Thanks so much for the help! I finally found the folder and did the change.

I made a mistake for using a host machine that doesn’t have SDK manager(which I wrongly thought is the one I used to initially setup the board). And it is using ubuntu. I used Mac just to check what is downloaded(as I am not sure whether I should download).

Thanks again for your help!

1 Like