"virt_to_page()" error on TX1

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX1
1

Hi everyone.

I am testing PEX8311 in TX1 and I have a problem like the following URL.

[url]https://devtalk.nvidia.com/default/topic/1032967/?comment=5255873[/url]

So I patched and testing.

But still have problem.

Please advise.

Thanks!!!

  • Test environment
    JetPack 3.3 — L4T R28.2.1

  • PCIe entrie
    root@tegra-ubuntu:~# ls /sys/kernel/debug/70019000.iommu/masters/
    0000:00:01.0 70006000.serial 700d0000.xudc
    0000:01:00.0 70006040.serial 702ef000.adsp
    0000:02:04.0 70006200.serial adsp_audio
    1003000.pcie-controller 70006300.serial flush_all_threshold_map_pages
    50000000.host1x 7000c000.i2c flush_all_threshold_unmap_pages
    54080000.vi 7000c400.i2c mc
    54100000.tsecb 7000c500.i2c sdhci-tegra.0
    54340000.vic 7000c700.i2c sdhci-tegra.3
    54380000.nvjpg 7000d000.i2c serial8250
    54480000.nvdec 7000d100.i2c smmu_test
    544c0000.nvenc 7000d400.spi snd-soc-dummy
    54500000.tsec 7000da00.spi sound
    54600000.isp 70012000.se tegra-carveouts
    54680000.isp 70016000.bpmp tegradc.1
    546c0000.i2c 70027000.sata
    57000000.gpu 70090000.xusb

  • Driver code
    pMemObject->pKernelVa =
    Plx_dma_alloc_coherent(
    pdx,
    pMemObject->Size,
    &BusAddress,
    GFP_KERNEL | __GFP_NOWARN
    );

    if (pMemObject->pKernelVa == NULL)
    {
    return NULL;
    }

    // Store the bus address
    pMemObject->BusPhysical = (U64)BusAddress;

    // Tag all pages as reserved
    for (virt_addr = (PLX_UINT_PTR)pMemObject->pKernelVa;
    virt_addr < ((PLX_UINT_PTR)pMemObject->pKernelVa + pMemObject->Size);
    virt_addr += PAGE_SIZE)
    {
    SetPageReserved(
    virt_to_page( PLX_INT_TO_PTR(virt_addr) )
    );
    }

  • kernel message

root@tegra-ubuntu:/home/hoonix# insmod Plx8311_dbg.ko
[ 38.895199]
[ 38.896726] Plx8311: <========================================================>
[ 38.904349] Plx8311: PLX 8311 driver v7.25 (64-bit)
[ 38.909625] Plx8311: Supports Linux kernel v4.4.38+
[ 38.914530] Plx8311: Allocated global driver object (ffffffc0db386e00)
[ 38.921062] Plx8311: Registered driver (MajorID = 232)
[ 38.926245] Plx8311: --------------------
[ 38.930514] Plx8311: Probe: 86E1 10B5 [D0 02:04.0]
[ 38.935330] Plx8311: Enabled PCI device
[ 38.939170] Plx8311: Created Device (Plx8311-0)
[ 38.943698] Plx8311: Start: 86E1 10B5 [D0 02:04.0]
[ 38.948484] Plx8311: Resource 00
[ 38.951969] Plx8311: Type : Memory
[ 38.956152] Plx8311: PCI BAR 0: 13040000
[ 38.960502] Plx8311: Phys Addr: 13040000
[ 38.964854] Plx8311: Size : 200h (512B)
[ 38.969465] Plx8311: Property : Non-Prefetchable 32-bit
[ 38.975260] Plx8311: Kernel VA: ffffff800d3ea000
[ 38.980307] Plx8311: Resource 01
[ 38.983791] Plx8311: Type : I/O
[ 38.987713] Plx8311: PCI BAR 1: 00001001
[ 38.992066] Plx8311: Phys Addr: 00001000
[ 38.996418] Plx8311: Size : 100h (256B)
[ 39.001030] Plx8311: Resource 02
[ 39.004514] Plx8311: Type : Memory
[ 39.008695] Plx8311: PCI BAR 2: 13000000
[ 39.013049] Plx8311: Phys Addr: 13000000
[ 39.017401] Plx8311: Size : 20000h (128KB)
[ 39.022271] Plx8311: Property : Non-Prefetchable 32-bit
[ 39.027931] Plx8311: Kernel VA: ffffff800d800000
[ 39.032978] Plx8311: Resource 03
[ 39.036462] Plx8311: Type : Memory
[ 39.040643] Plx8311: PCI BAR 3: 13020000
[ 39.044994] Plx8311: Phys Addr: 13020000
[ 39.049344] Plx8311: Size : 20000h (128KB)
[ 39.054228] Plx8311: Property : Non-Prefetchable 32-bit
[ 39.059884] Plx8311: Kernel VA: ffffff800d840000
[ 39.065330] Plx8311: Device 86E1_10B5 = 8311 rev AA
[ 39.070233] Plx8311: Installed ISR for interrupt
[ 39.074913] Plx8311: Attempt to allocate physical memory (8KB)
[ 39.081171] Unable to handle kernel paging request at virtual address 3ffffbcc234fb00
[ 39.089004] pgd = ffffffc0d7b06000
[ 39.092408] [3ffffbcc234fb00] *pgd=0000000000000000, *pud=0000000000000000
[ 39.099291] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 39.104850] Modules linked in: Plx8311_dbg(O+) xt_addrtype br_netfilter bcmdhd bluedroid_pm
[ 39.113223] CPU: 2 PID: 2427 Comm: insmod Tainted: G O 4.4.38+ #1
[ 39.120428] Hardware name: jetson_tx1 (DT)
[ 39.124513] task: ffffffc05fcdb200 ti: ffffffc0d8058000 task.ti: ffffffc0d8058000
[ 39.131982] PC is at set_bit+0x18/0x30
[ 39.135737] LR is at Plx_dma_buffer_alloc+0xf4/0x224 [Plx8311_dbg]
[ 39.141901] pc : [] lr : [] pstate: 60000145
[ 39.149278] sp : ffffffc0d805bb60
[ 39.152581] x29: ffffffc0d805bb60 x28: ffffffbffcf4f290
[ 39.157892] x27: ffffffbffcf49708 x26: ffffffbffcf4d5eb
[ 39.163200] x25: 0000000000000030 x24: 0000000000000333
[ 39.168508] x23: 0000000000000001 x22: ffffffc0012c0e30
[ 39.173817] x21: 0000004000000000 x20: ffffffc0ed368700
[ 39.179125] x19: 0000000000000000 x18: 00000000ffffff00
[ 39.184432] x17: 0000000000000000 x16: 0000000000000000
[ 39.189741] x15: 0000000000000000 x14: 0000000000000000
[ 39.195050] x13: 0000000000000000 x12: 0000000000000069
[ 39.200360] x11: 00e80001577d178f x10: ffffffc000fa1e18
[ 39.205671] x9 : 0000000000000001 x8 : ffffffc0781f6680
[ 39.210981] x7 : ffffff800d5ed000 x6 : 00000001577d1000
[ 39.216291] x5 : ffffffc0dfb1f398 x4 : 00e80001577d170f
[ 39.221600] x3 : 0000000000000400 x2 : 0000000000000001
[ 39.226910] x1 : 03ffffbcc234fb00 x0 : 0000000000000000
[ 39.232220]
[ 39.233705] Process insmod (pid: 2427, stack limit = 0xffffffc0d8058020)
[ 39.240389] Call trace:
[ 39.242829] [] set_bit+0x18/0x30
[ 39.247622] [] PlxPciPhysicalMemoryAllocate+0xb4/0x1c8 [Plx8311_dbg]
[ 39.255533] [] init_module+0x178/0x290 [Plx8311_dbg]
[ 39.262047] [] do_one_initcall+0x110/0x1c4
[ 39.267693] [] do_init_module+0x64/0x1c4
[ 39.273166] [] load_module+0xc38/0x10d8
[ 39.278550] [] SyS_finit_module+0x98/0xb4
[ 39.284108] [] el0_svc_naked+0x24/0x28
[ 39.289627] —[ end trace 5cda80c6047d0787 ]—
Segmentation fault

2

Can you please pinpoint line in your driver where it is failing?

3

Hi vidyas.

Below is a part of the code.

pMemObject->pKernelVa =
        Plx_dma_alloc_coherent(
            pdx,
            pMemObject->Size,
            &BusAddress,
            GFP_KERNEL | __GFP_NOWARN
            );

    if (pMemObject->pKernelVa == NULL)
    {
        return NULL;
    }

    // Store the bus address
    pMemObject->BusPhysical = (U64)BusAddress;

    // Tag all pages as reserved
    for (virt_addr = (PLX_UINT_PTR)pMemObject->pKernelVa;
         virt_addr < ((PLX_UINT_PTR)pMemObject->pKernelVa + pMemObject->Size);
         virt_addr += PAGE_SIZE)
    {
        SetPageReserved(
            virt_to_page( PLX_INT_TO_PTR(virt_addr) ) <= An error occurs here.!!!
            );
    }

Thank for your interest.!!

4

Is it possible to share your driver offline?
What exactly is ‘pdx’ ? (first argument to Plx_dma_alloc_coherent() API )

5

Hi vidyas

Thank you in advance for your help.

  1. The source is available from the broadcom site below.

https://www.broadcom.com/products/pcie-switches-bridges/pcie-bridges/pex8311#downloads

For compile, See below.

step1. add in Makefiles/Common.def in line 58

ifeq ($(ARCH),arm64)
PLX_CPU_BITS = 64
PLX_CPU_ENDIAN = Little
endif

step2. Compile

#export PLX_SDK_DIR=$PWD
#export ARCH=arm64
#export CROSS_COMPILE=aarch64-unknown-linux-gnu-
#export KDIR=/work/bin/kernel
#export PLATFORM_NAME=Tegra

cd $PLX_SDK_DIR

make

cd Driver

#./builddriver 8311 d

  1. "Plx_dma_alloc_coherent() ?

It is defined as follows. in Include/Plx_sysdep.h

#define Plx_dma_alloc_coherent(pdx, size, dma_handle, flag) \
                dma_alloc_coherent(            \
                    &((pdx)->pPciDevice->dev), \
                    (size),                    \
                    (dma_handle),              \
                    (flag)                     \
                    )
6

Hi everyone.

I just uploaded the module with no errors.

Like other boards, I just changed the DTB file to /boot and tested it.

But this time, I wrote all image include new DTB through apply_binaries.sh and flash.sh.

Anyway It is my mistake.

Thanks to vidyas for his attention.

Best regards.