I am working with a Xilinx FPGA PCIe core connected to the TX1 4x PCIe. Just trying to read and write data across the PCIe bridge through a Linux PCI driver on the TX1.
The TX1 recognizes the PCIe device and reports the correct properties for memory, BAR size, and so on. But the Xilinx side reports that the root complex never initializes the PCIe device, even though the TX1 PCI driver calls pci_device_enable() successfully.
The result is that the Xilinx must initialize the PCIe device through ARM code before read/writes across the bridge will work. And we have to be careful about the order we boot the devices (TX1 after Xilinx).
The folks here say that a root complex ought to be initializing all devices. Is there a reason why the TX1 root complex wouldn’t do that? Some other function call missing from the PCI driver perhaps?
I have couple of follow-up questions… Could you please help us with answers?
The TX1 recognizes the PCIe device and reports the correct properties for memory, BAR size, and so on. But the Xilinx side reports that the root complex never initializes the PCIe device, even though the TX1 PCI driver calls pci_device_enable() successfully.
What I understood from this is that, TX1 system is able to enumerate the Xilinux FPGA based end point device.
Do you have a PCIe device driver that is expected to work with your device loaded into system already? (given that you already mentioned, BTW, the API is pci_enable_device() looks like you have a driver for the device. Please confirm.
I did not understand what you meant by “Xilinx side reports that the root complex never initializes the PCIe device” part… what is “initialization” here referring to exactly?
Yes… order is important. i.e. Xilinx FPGA has to be initialized before host proceeds with enumeration process. If you are already aware of the time taken by FPGA end point to get initialized completely, please make use of ‘nvidia,boot-detect-delay’ device tree entry of PCIe node, to input time (in milliseconds) to delay host probing process.
Are you saying that, in spite of having pci_set_master() call in your device driver, you don’t see the bit getting set in your FPGA based device’s configuration space?