How to send multi vector msi interrupt packet?

We are trying to send msi multi vector interrupt from FPGA endpoint to Jetson AGX Xavier. When we send the msi packet with data 0, vector 0 interrupt handler invoked. But, when we send msi packet with data 1, vector 0 interrupt handler is only getting invoked. How to send msi interrupt packet to invoke vector 1 interrupt handler?

Not quite understand the use-case. Do you mean you insert a PCIe EP device to main PCIe slot on Xavier developer kit, and it can be detected but cannot send data to Xavier?

MicrosoftTeams-image (2)
Refer the above image. we are trying to send msi packet by setting MSI Message data (Byte 10).

Can I get any update on this?

Are you able to set up the same on Xavier developer kit and reproduce the issue? If you can have a setup to reproduce the issue, please share the steps so that we can give it a try.

Hi @DaneLLL ,
In Xavier Endpoint, we don’t form the TLP packet, we just set one bit for the corresponding vector. It takes care of the TLP packet formation.

But, we are forming TLP packet in FPGA and sending it to the Xavier Board Root complex. We don’t know what exact MSI message data value need to be set in the MSI packet in order to differentiate two different MSI vectors.


Could you share your "sudo lspci -vvv” result and fpga driver here?

The “sudo lspci -vvv” log is given below for your reference,

0005:01:00.0 Signal processing controller: Device 19aa:e004 (rev 04)
Subsystem: Device 19aa:e004
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 820
Region 0: Memory at 1f40000000 (32-bit, non-prefetchable) [size=64K]
Capabilities: [40] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr+ NoSnoop+
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L0s L1, Exit Latency L0s unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
Capabilities: [80] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [90] MSI-X: Enable- Count=8 Masked-
Vector table: BAR=0 offset=00006000
PBA: BAR=0 offset=00007000
Capabilities: [a0] MSI: Enable+ Count=2/8 Maskable+ 64bit+
Address: 00000000fffff000 Data: 0000
Masking: 000000fc Pending: 00000000
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 1
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [150 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: [180 v1] #19
Capabilities: [200 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable-, Smallest Translation Unit: 00
Capabilities: [210 v1] Device Serial Number 00-00-00-00-00-00-00-00
Capabilities: [3d0 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=0us PortTPowerOnTime=0us
L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
T_CommonMode=0us LTR1.2_Threshold=0ns
L1SubCtl2: T_PwrOn=40us
Kernel driver in use: test_audio_video

Refer the below image’s tlp_codec/tx_data for the data value that we are giving in the MSI TLP Packet,


The MSI capability looks fine and there are 2 interrupts enabled.

To trigger 1st interrupt FPGA should send a memory write to address 0xfffff000 with data 0x0 and for 2nd interrupt, should send a memory write to address 0xfffff000 with data 0x1.

Please check this in the driver code and see if cat /proc/interrupts has interrupt increasing.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.