System crash

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

hi nv
When I use pcie dma to transfer data, sometimes the system crashes
crash-0328.log (51.2 KB)

3

Are you using devkit or you custom carrier board?
Which JetPack SW?

4

hi nv
Jetpack5.1 and it’s our own carrier board

5

please provide more detail and full log.

6

hi nv
I use kzalloc to alloc memory,then use dma_map_single to map dma address region
and use dma to transfer

7

How about directly share your full code and full log…

Also, is this issue able to reproduce on devkit?

8

hi nv
Only appear on our own carrier board
If i use dma_alloc_coherent, work fine
But how to get physical address？it seems virt_to_phys() doesn’t work as expected

9

hi nv
use dma_alloc_coherent() also report error, then the dma stopped working
please help, thanks
log-0329-1.log (138.4 KB)

10

please share the detail steps so that we can reproduce this on NV dekvit.

11

orin A allocate three 32M sized memory,orin B also too
use dma linked list mode to transfer data
then after sending data multiple times, system report error

12

please provide the sample code and your method to build it.

13

ORIN A alloc memroy, orin b also too
//RING_COUNT =3 ALLOC_SIZE = 0X2000000
for(i = 0; i < RING_COUNT; i++){
ram_map = dma_alloc_coherent(cdev, ALLOC_SIZE,
&src_iova,
GFP_KERNEL);
if (!ram_map) {
dev_err(fdev, “%s alloc ep mem failed\n”, func);
return -ENOMEM;
}
printk(“%s ram_map 0x%p src_iova 0x%llx\n”,func,ram_map,src_iova);
dma_desc[i].sar_low = (src_iova & 0xffffffff);
dma_desc[i].sar_high = ((src_iova >> 32) & 0xffffffff);
memset(ram_map, 0xa+i, ALLOC_SIZE);
}

DMA init ,same as the original vnet code
static void tvnet_ep_setup_dma(struct pci_epf_tvnet *tvnet)
{
…
}

transfer data
static int write_test(struct seq_file *s, void data)
{
…
/ Trigger DMA write from src_iova to dst_iova /
desc_widx = desc_cnt->wr_cnt % RING_COUNT;
printk("%s desc_cnt wr_cnt %d rd_cnt %d\n ",func,desc_cnt->wr_cnt, tvnet->desc_cnt.rd_cnt);
dst_iova = ep2h_empty_msg[desc_widx].u.empty_buffer.pcie_address;
ep_dma_virt[desc_widx].size = len;
//ep_dma_virt[desc_widx].sar_low = lower_32_bits(src_iova);
//ep_dma_virt[desc_widx].sar_high = upper_32_bits(src_iova);
ep_dma_virt[desc_widx].dar_low = lower_32_bits(dst_iova);
ep_dma_virt[desc_widx].dar_high = upper_32_bits(dst_iova);
/ CB bit should be set at the end */
mb();
ctrl_d = DMA_CH_CONTROL1_OFF_WRCH_LIE;
ctrl_d |= DMA_CH_CONTROL1_OFF_WRCH_CB;
ep_dma_virt[desc_widx].ctrl_reg.ctrl_d = ctrl_d;

/* DMA write should not go out of order wrt CB bit set */
mb();
    	timeout = jiffies + msecs_to_jiffies(1000);

// start_time = ktime_get();
dma_common_wr8(tvnet->dma_base, DMA_WR_DATA_CH, DMA_WRITE_DOORBELL_OFF);
desc_cnt->wr_cnt++;

while (true) {
	val = dma_common_rd(tvnet->dma_base, DMA_WRITE_INT_STATUS_OFF);
	if (val == BIT(DMA_WR_DATA_CH)) {
		//end_time = ktime_get();
		//printk("dma write int trigger     ns   %lld\n",(ktime_to_ns(end_time)-ktime_to_ns(start_time)));
		dma_common_wr(tvnet->dma_base, val,
			      DMA_WRITE_INT_CLEAR_OFF);
		break;
	}
	if (time_after(jiffies, timeout)) {
		dev_err(tvnet->fdev, "dma took more time, reset dma engine\n");
		dma_common_wr(tvnet->dma_base,
			      DMA_WRITE_ENGINE_EN_OFF_DISABLE,
			      DMA_WRITE_ENGINE_EN_OFF);
		mdelay(1);
		dma_common_wr(tvnet->dma_base,
			      DMA_WRITE_ENGINE_EN_OFF_ENABLE,
			      DMA_WRITE_ENGINE_EN_OFF);
		desc_cnt->wr_cnt--;

#if !ENABLE_DMA
#if (LINUX_VERSION_CODE > KERNEL_VERSION(4, 15, 0))
pci_epc_unmap_addr(epc, epf->func_no, tvnet->tx_dst_pci_addr);
#else
pci_epc_unmap_addr(epc, tvnet->tx_dst_pci_addr);
#endif
#endif
//dma_unmap_single(cdev, src_iova, len, DMA_TO_DEVICE);
return NETDEV_TX_BUSY;
}
}

   	desc_ridx = tvnet->desc_cnt.rd_cnt % RING_COUNT;
/* Clear DMA cycle bit and increment rd_cnt */
ep_dma_virt[desc_ridx].ctrl_reg.ctrl_e.cb = 0;
mb();

…

}

14

please attach it as a file and share us how to run your code.

15

demo.txt (2.9 KB)

16

Hi,

可以麻煩直接提供完整的檔案或是patch file嗎? 並且附上完整的複製問題步驟

這是標準流程. 謝謝

17

如何给您私发文件？

18

Is this still an issue to support? Any result can be shared? Thanks

19

hi nv
Still need help, kernel reported the following error,dst address incorrect?
[ 90.896894] arm-smmu 12000000.iommu: Unhandled context fault: fsr=0x402, iova=0x00000000, fsynr=0x400011, cbfrsynra=0x1014, cb=6
[ 90.909106] mc-err: unknown mcerr fault, int_status=0x00000000, ch_int_status=0x00000000, hubc_int_status=0x00000000 sbs_int_status=0x00000000, hub_int_status=0x00000000