I have a PCI-E card, which can read and write the PCI-E register after burning the system, but cannot DMA transfer, and error segmentation fault will be reported during DMA test.
Supplement: I use winDriver
I’m not sure if I understood the question correctly.
What does it mean by “I have a PCI-E card, which can read and write the PCI-E register after burning the system”
What exactly is reading and writing the PCIe registers? is it CPU or something else? Where are these registers presents? in the BAR of the PCIe card?
I didn’t understand the DMA part either. Do you mean the PCIe card has gotten a DMA engine inside it whose control registers are exposed to the host system through PCIe BAR and when DMA is programmed to operate (i.e. to do read and writes to the host system’s memory), the system is segfaulting?
Could you please check your driver whether it is adhering to Linux PCIe driver structure?
Yes, I am sure that this driver supports Linux ARM platform. I have also checked the driver in the directory /dev. However, when DMA is transferred, there will be Segmentation faults (core Dumped).
Here is the print information for running the example：
XDMA diagnostic utility.
Application accesses hardware using WinDriver.
Found 1 matching device(s) [Vendor ID 0x10EE, Device ID 0x0 (ALL)]:
- Vendor ID: 0x10EE, Device ID: 0x8024
Location: Bus [0x1], Slot [0x0], Function [0x0]
Memory range [BAR 0]: base 0x40100000, size 0x100000
Memory range [BAR 1]: base 0x40200000, size 0x10000
Interrupt: IRQ 0
Interrupt Options (supported interrupts):
Message-Signaled Interrupt (MSI)
PCI Express Generation: Gen2
XDMA main menu
- Scan PCI bus
- Find and open a XDMA device
- Read/write memory and I/O addresses on the device
- Read/write the device’s configuration space
- Read/write the configuration block registers
- Direct Memory Access (DMA)
- Direct Memory Access (DMA) transaction
- Register/unregister plug-and-play and power management events
- Exit
Enter option: 6
XDMA DMA menu
- Perform DMA transfer
- Measure DMA performance
- Exit
Enter option: 2
DMA performance
- DMA host-to-device performance
- DMA device-to-host performance
- DMA host-to-device and device-to-host performance running simultaneously
- Exit menu
Enter option: 1
Select DMA completion method:
- Interrupts
- Polling
- Cancel
Enter option: 2
Enter single transfer buffer size in KBs (to cancel press ‘x’): 128
Enter test duration in seconds (to cancel press ‘x’): 2
Running DMA host-to-device performance test, wait 2 seconds to finish…
Segmentation fault (core dumped)
Could you please give the complete output of ‘sudo lspci -vvvv’ ?
Also, please share a link to the source code of WinDriver?
00:01.0 PCI bridge: NVIDIA Corporation Device 10e5 (rev a1) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 381
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000f000-00000fff
Memory behind bridge: 50100000-502fffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Subsystem: NVIDIA Corporation Device 0000
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
Mapping Address Base: 00000000fee00000
Capabilities: [80] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0
ExtTag+ RBE+
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <512ns, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Off, PwrInd On, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet+ LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR+, OBFF Not Supported ARIFwd-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Kernel driver in use: pcieport
Kernel modules: windrvr1440
01:00.0 Serial controller: Xilinx Corporation Device 8024 (prog-if 01 [16450])
Subsystem: Xilinx Corporation Device 0007
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Region 0: Memory at 50100000 (32-bit, non-prefetchable) [size=1M]
Region 1: Memory at 50200000 (32-bit, non-prefetchable) [size=64K]
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM not supported, Exit Latency L0s unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
Kernel modules: windrvr1440
What kind of driver is ‘windrvr1440’? I have the following observations
- I don’t see BusMaster not enabled (BusMaster-)
- I don’t see MSI interrupt also not enabled (MSI: Enable-)
If the driver has properly configured the device, then, I wouldn’t be observing these (well, I’m ok with MSI not being enabled given legacy interrupt is enabled). It doesn’t look like “windrvr1440” is properly configuring the device for the operation. Please make sure that this driver is written correctly ( 1. How To Write Linux PCI Drivers — The Linux Kernel documentation would be a good start)
Thank you for your reply！
The WinDriver 14.4.0 device driver development tool supports any device, regardless of its silicon vendor, and enables you to focus on your driver’s added-value functionality, instead of on the operating system internals. WinDriver’s driver development solution covers USB, PCI and PCI Express.
For L4T 28.2.1, the device tree is modified to make WinDriver 1440 successfully used.
reference：
Jetpack 3.3 for TX2(L4T 28.2.1): SMMU disable - #2 by vidyas
Therefore, I want to know how to solve this problem in the later version. Can you give me some solutions or suggestions？
For the same PCI-E driver（windrvr1440）, I found that MSI interrupt work on version L4T 28.2.1.
So，What can I do to make MSI interrupt enabled on TX2i？
I can’t say much at this point as I don’t know what is happening in the device driver. Are you sure the same device driver for FPGA is being used in both 28.2.1 and later version?
As I mentioned earlier, the fact that the Bus Master bit itself is not enabled indicates that something very basic seems to be wrong.
Your driver would have pci_enable_device() call which is responsible for setting the bus master enable bit and there would be APIs to assign MSI interrupts also. You may have to put prints in the driver and find out where exactly is it going wrong.
As far as disabling SMMU is concerned, the procedure should apply to the later versions as well and SMMU enabled/disabled has nothing to do with bus-master bit not being set and MSIs not enabled. These are two different issues.
Hello @1120482032 ! I am a representative of Jungo, and, firstly, would like to apologize for the delay in the response. Please open a support ticket here http://techsup.jungo.com/createaccount.cgi so we could try assisting you further. Thanks!