Question 1：When we are developing a PCIe DMA driver on Jesson Orin NX (connecting FPGA devices through PCIe interface, FPGA writes DMA video data to Jetson DMA), after applying a DMA address and sending it to FPGA, FPGA cannot write data to Jetson’s DMA memory, indicating that there is memory protection.（sdk：R36.4.4 ，OS ubuntu22，pcie@141e0000 pciex2,pcie_c7）
Question 2：Another issue is that Jetson’s PCIe will reset twice during startup (the first time should be U-Boot reset, and the second time is kernel reset). After the second reset, it will cause our device to malfunction. How can we turn off one of the resets? And when PCIe detects that there is no device link during startup, it will turn off PCIe’s clock output. How can it maintain clock output even without device link?
our pcie driver code step
//step 1-1 pci_get_device
pci_get_device(VP_PCI_VENDOR_ID_XILINX, PcieDeviceID[i], pDrvData->pciDev[i]);
//step 2-1 pci_resource_start
pDrvData->bar_addr[offset].bar_addr[j] = pci_resource_start(pDrvData->pciDev[offset], j);
//step 2-2 pci_resource_len
pDrvData->bar_size[offset].bar_size[j] = pci_resource_len(pDrvData->pciDev[offset], j);
//step 2-3 request_mem_region
request_mem_region(pDrvData->bar_addr[offset].bar_addr[j], pDrvData->bar_size[offset].bar_size[j], Str)
//step 3-1 pci_set_dma_mask
pci_set_dma_mask(pDrvData->pciDev[offset], DMA_BIT_MASK(32)); //I don’t know if a 64 bit address is must needed
//step 3-2 pci_set_consistent_dma_mask(
pci_set_consistent_dma_mask(pDrvData->pciDev[offset], DMA_BIT_MASK(32)); //I don’t know if a 64 bit address is must needed
//step 4-1 dma_alloc_coherent
/Question 1/
//We send the requested DMA address to FPGA (eg addr: 0x80400000),
//when FPGA writes data to the DMA address, it prompts the following 《DMA WRITE KERNEL ERROR MESSAGE》 ,It seems that FPGA cannot write to this DMA address. This DMA address has VPR protection or some other protection:
pDrvData->dmabuf[offset].DmaBuf[j].dma_va = dma_alloc_coherent(&pDrvData->pciDev[offset]->dev,
VP_DMAMEM_SIZE,
&pDrvData->dmabuf[offset].DmaBuf[j].dma_la, GFP_KERNEL /GFP_ATOMIC/);
//step 5-1 pci_enable_device
pci_enable_device(pDrvData->pciDev[offset])
/**********************************************************************************Question 1 log 《DMA WRITE KERNEL ERROR MESSAGE》 **********************************************************************************/
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.499383] [DMA][0][0][virt_addr:ffff000000400000][physical_addr:80400000][80400000][80400000] //dma_alloc_coherent alloc DMA addr 80400000 :FPGA cannot write date to this address,its 32bit adr
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.499928] [DMA][0][1][ffff000000800000][80800000][80800000][80800000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.500684] [DMA][0][2][ffff000000c00000][80c00000][80c00000][80c00000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.501244] [DMA][0][3][ffff000001000000][81000000][81000000][81000000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.501785] [DMA][0][4][ffff000001400000][81400000][81400000][81400000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.502368] [DMA][0][5][ffff000001800000][81800000][81800000][81800000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.503456] [DMA][0][6][ffff000001c00000][81c00000][81c00000][81c00000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.504010] [DMA][0][7][ffff000002000000][82000000][82000000][82000000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.504526] [DMA][0][8][ffff000002400000][82400000][82400000][82400000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.505028] [DMA][0][9][ffff000002800000][82800000][82800000][82800000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.505544] [DMA][0][10][ffff000002c00000][82c00000][82c00000][82c00000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.506048] [DMA][0][11][ffff000003000000][83000000][83000000][83000000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.507132] [DMA][0][12][ffff000003400000][83400000][83400000][83400000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.509775] [DMA][0][13][ffff000003800000][83800000][83800000][83800000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.510352] [DMA][0][14][ffff000003c00000][83c00000][83c00000][83c00000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.510909] [DMA][0][15][ffff000004000000][84000000][84000000][84000000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.511446] [DMA][0][16][ffff000004400000][84400000][84400000][84400000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.511982] [DMA][0][17][ffff000004800000][84800000][84800000][84800000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.512517] [DMA][0][18][ffff000004c00000][84c00000][84c00000][84c00000]
Feb 20 21:25:06 cdgx-desktop kernel: [ 74.513059] [DMA][0][19][ffff000005000000][85000000][85000000][85000000]
Feb 20 21:25:07 cdgx-desktop kernel: [ 75.518088] probe [1] channels pcie Dev
Feb 20 21:25:44 cdgx-desktop kernel: [ 111.844296] minor[0] open[0]
Feb 20 21:25:44 cdgx-desktop kernel: [ 111.844769] Fc_fasync Init
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.855809] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x80400000, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4 //When the FPGA writes data to this address, it prints this error message
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.856012] tegra-mc 2c00000.memory-controller: pcie7aw: secure write @0x00000003ffffff000x00000003ffffff000x00000003ffffff000x00000003ffffff00: VPR violation ((null)) //When the FPGA writes data to this address, it prints this error message
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.856023] tegra-mc 2c00000.memory-controller: pcie@0x00000003ffffff000x00000003ffffff00aw: secure write @0x00000003ffffff00: Route Sanity error ((null)) //When the FPGA writes data to this address, it prints this error message
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.892606] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x80404b00, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.892758] tegra-mc 2c00000.memor@0x00000003ffffff000x00000003ffffff00-controller: pcie7aw: secure write @0x00000003ffffff00: VPR violation ((null))
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.892766] tegr@0x00000003ffffff000x00000003ffffff00-mc 2c00000.memory-controller: pcie7aw: secure write @0x00000003ffffff00: Route Sanity error ((null))
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.892794] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova@0x00000003ffffff000x804080c0, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4
Feb 20 21:25:46 cdgx@0x00000003ffffff00desktop kernel: [@0x00000003ffffff00 113.892936] tegra-mc 2c00000.memory-controller: pcie7aw: secure write @0x00000003ffffff00: VPR violation ((null))
Fe@0x00000003ffffff00 20 21:25:46 cdgx@0x00000003ffffff00desktop kernel: [ 113.892942] tegra-mc 2c00000.memory-controller: pcie7aw: secure write @0x00000003ffffff00: Route Sanity error ((null))
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.892958] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x80417780, fsynr=0x7a0011, cbfrsyn@0x00000003ffffff00a=0x1008, cb=4
Fe@0x00000003ffffff00 20 21:25:46 cdgx-desktop kernel: [ 113.893099] tegra-mc 2c00000.memory-controller: pcie7aw: secure write @0x0000000@0x00000003ffffff00ffffff00: VPR vio@0x00000003ffffff00ation ((null))
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893105] tegra-mc 2c00000.memory-controller: pcie7aw: secure write @0x00000003ffffff00: Route Sanity error ((null))
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893125] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova@0x00000003ffffff000x804254c0, fsynr@0x00000003ffffff000x7a0011, cbfrsynra=0x1008, cb=4
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893266] tegra-mc 2c00000.memory-controller: pcie7aw: secure write @0x00000003ffffff00: VPR violation ((null))
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893272] tegra-mc 2c00000.memory-controller: pcie7aw: secure write @0x00000003ffffff00: Route Sanity error ((null))
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893285] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x80433700, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4 //When the FPGA writes data to this address, it prints this error message
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893437] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x80440fc0, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893589] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x8044de00, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893742] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x8045ac40, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4
Feb 20 21:25:46 cdgx-desktop kernel: [ 113.893899] arm-smmu 8000000.iommu: Unhandled context fault: fsr=0x402, iova=0x80467c80, fsynr=0x7a0011, cbfrsynra=0x1008, cb=4