PCIE Driver Load

Hello, this might be more of a general linux kernel device tree question, but thought I would just ask here because it’s on a nano, and I have been getting good support on this forum.

I have my Jetson nano booting on my custom carrier board, and it’s connected over 4xpcie to a Xillinx fpga that I would like to talk to. Ive gotten this woking before on a TX2, communicating with Xillybus so that I don’t have to worry about the details of dma, and xillybus provides a driver to go with their fpga design files. I can see my FPGA when I run lspci, on Slot: 01:00.0, and I have compiled the kernel with the xillybus driver enabled, however it doesn’t seem to be picking them up. I have a sneaking suspicion I have to add it in the device tree (which I don’t really know where to start), but I was under the impression that PCIE devices could be dynamically detected and driver loaded.
lspci output:
Slot: 00:01.0
Class: PCI bridge
Vendor: NVIDIA Corporation
Device: Device 0fae
Rev: a1

Slot: 00:02.0
Class: PCI bridge
Vendor: NVIDIA Corporation
Device: Device 0faf
Rev: a1

Slot: 01:00.0
Class: Unassigned class [ff00]
Vendor: Xilinx Corporation
Device: Device ebeb
SVendor: Xilinx Corporation
SDevice: Device ebeb
Rev: 07

Slot: 02:00.0
Class: Ethernet controller
Vendor: Realtek Semiconductor Co., Ltd.
Device: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
SVendor: Realtek Semiconductor Co., Ltd.
SDevice: RTL8111/8168 PCI Express Gigabit Ethernet controller
Rev: 19

xillybusses recommended device tree instantiation:
Required properties:

  • compatible: Should be “xillybus,xillybus-1.00.a”
  • reg: Address and length of the register set for the device
  • interrupts: Contains one interrupt node, typically consisting of three cells.
  • interrupt-parent: the phandle for the interrupt controller that
    services interrupts for this device.

Optional properties:

  • dma-coherent: Present if DMA operations are coherent

Example:

xillybus@ff200400 {
	compatible = "xillybus,xillybus-1.00.a";
	reg = < 0xff200400 0x00000080 >;
	interrupts = < 0 40 1 >;
	interrupt-parent = <&intc>;
} ;

Hi,

PCIe client driver auto loads if device ID and vendor ID matches with PCI table array in driver.
Use “lspci -n” to get vendor and device ID of Xylinx device and check if this device is supported by xilybusses driver. It will define a structure "static const struct pci_device_id " in xilybusses driver to add device id & vendor id of the supported devices.

If the driver needs any parameters from device tree, then it should be added as child node to pci@1,0 node. For example, ethernet@0,0 is DT node for realtek NIC under pci@2,0 controller.

            pci@2,0 {

#if TEGRA_XUSB_PADCONTROL_VERSION >= DT_VERSION_2
phys = <&{/xusb_padctl@7009f000/pads/pcie/lanes/pcie-0}>;
phy-names = “pcie-0”;
#endif
nvidia,num-lanes = <1>;
nvidia,plat-gpios = <&gpio TEGRA_GPIO(X, 3) GPIO_ACTIVE_HIGH>;
status = “okay”;

                    ethernet@0,0 {
                            reg = <0x000000 0 0 0 0>;
                    };
            };

Thanks,
Manikanta

Hey thanks for the help,

I double checked the device ID and vendor ID in my driver, and I think they match,
static const struct pci_device_id xillyids = {
{PCI_DEVICE(PCI_VENDOR_ID_XILINX, PCI_DEVICE_ID_XILLYBUS)}, (where xilinx =0x10ee and xillybus = 0xebeb)
and “lspci -nv” gives me that the device is 1:00.0 ff00: 10ee:ebeb (rev 07)

I also added
xillybus@0,0 {
compatible = “xillybus,xillybus-1.00.a”;
reg = < 0x13000000 0x00000080 >;
interrupts = <0 41 0x04>;
interrupt-parent = <&intc>;
}
to the subnode pci@1,0 (and reflashed just the tree); and while I can see that /proc/iomem has changed to reflect the reg boundries, it still just isn’t loading the driver.

Hi,

Following config options are required to enable Xilinx drivers. These are not enabled by default in Jetpack, I hope you are enabling them and flash custom kernel.
CONFIG_XILLYBUS
CONFIG_XILLYBUS_PCIE

If the driver is enabled then it should be listed in “/sys/bus/pci/drivers”, you can check this directory for confirmation.

Thanks,
Manikanta

I compiled the kernels with CONFIG_XILLYBUS=y and CONFIG_XILLYBUS_PCIE=y inside tegra_defconfig, and my /sys/bus/pci/drivers directory has a folder xillybus_pcie.

running sudo modprobe xillybus_pcie, runs with no output, so it detects it in the drivers

hmmm, looking into the logs I think it’s some issue with drivers and I suspect the interrupts.

syslog output:
Jan 28 07:58:18 turro kernel: [ 1.036066] xillybus_pcie 0000:01:00.0: enabling device (0000 -> 0002)
Jan 28 07:58:18 turro kernel: [ 1.036333] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.054102] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.054128] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.071865] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.071883] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.089617] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.089635] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.107367] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.107384] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.112780] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.112797] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.112801] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.112817] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.112823] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.112838] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.112843] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.112859] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.112864] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.112879] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.112884] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.112899] xillybus_pcie 0000:01:00.0: Malformed message (skipping): opcode=0, channel=000, dir=0, bufno=000, data=0000000
Jan 28 07:58:18 turro kernel: [ 1.112903] xillybus_pcie 0000:01:00.0: Sending a NACK on counter 0 (instead of b) on entry 0
Jan 28 07:58:18 turro kernel: [ 1.112908] xillybus_pcie 0000:01:00.0: Lost sync with interrupt messages. Stopping.
Jan 28 07:58:18 turro kernel: [ 1.138534] xillybus_pcie 0000:01:00.0: No response from FPGA. Aborting.