Hello forum.
I am trying to run a very simple CUDA program on a custom Jetson TK-1 based board.
I am having an error that appears to be relating to GMMU mapping of physical memory pages. Here is a log from when the program runs. I have turned on all debug options for GK20A. I’ve also got a few of my own messages sprinkled in.
So the issue is the bar1 test fails because the mapping through the bar1 memory aperature doesn’t reflect in the CPU. The MC decode errors occur at these points.
The board is running a GPU with power always enabled (for now) and fixed voltages.
I am using physical pages in the GMMU because using the SMMU for translation causes an error in vmalloc. at line 128. I will also explain this:
That appears to be a bug in alloc_gmmu_pages (!CONFIG_GK20A_PHYS_PAGE_TABLES):
pages = dma_alloc_attrs(…)
pages is ultimately passed to vmap() without ever setting the value of each PTE.
pages = (struct page **)handle
*kva = vmap(pages, count, 0, pgprot_dmacoherent(PAGE_KERNEL));
This causes a BUG in vmap because pages is essentially an array of null pointers.
It’s possible that there’s another issue that causes this.
I actually put a fix in for this, by creating an array of struct page * and then walking over the return from dma_alloc_attrs and populating it with each pte, and then called vmap(). This made BUG go away, but I wound up at the same two decode errors shown in the log below.
We’d like to get CUDA working on this board so any help would be appreciated.
[ 11.124646] vi vi.0: vi_probe: ++
[ 11.129236] vi vi.0: initialized
[ 11.132658] vi vi.1: vi_probe: ++
[ 11.138564] vi vi.1: initialized
[ 11.183474] gk20a gk20a_as_dev_open:
[ 11.187144] gk20a gk20a_init_client:
[ 11.190938] gk20a gk20a_init_mm_setup_sw:
[ 11.195036] gk20a gk20a_init_mm_setup_sw: channel vm size: 32768MB
[ 11.201212] gk20a gk20a_init_mm_setup_sw: small page-size (4KB) pte array: 256KB
[ 11.208602] gk20a gk20a_init_mm_setup_sw: big page-size (128KB) pte array: 8KB
[ 11.215816] gk20a gk20a_init_bar1_vm: bar1 vm size = 0x8000000
[ 11.221643] gk20a pde_range_from_vaddr_range: addr_lo=0x0 addr_hi=0x7ffffff pde_ss=27
[ 11.229464] gk20a pde_range_from_vaddr_range: pde_lo=0 pde_hi=0
[ 11.235376] pde lo=0x0 hi=0
[ 11.238170] gk20a gk20a_init_bar1_vm: init space for bar1 va_limit=0x8000000 num_pdes=1
[ 11.246162] gk20a alloc_gmmu_pages:
[ 11.249741] gk20a gk20a_init_bar1_vm: bar 1 pdes.kv = 0xed6a9000, pdes.phys = 0xad6a9000
[ 11.257821] gk20a gk20a_init_bar1_vm: pde pa=0xad6a9000 pde_addr_lo=0xad6a9 pde_addr_hi=0x0
[ 11.266421] gk20a gk20a_init_bar1_vm: bar1 inst block physical phys = 0xae84b000, kv = 0xee84b000
[ 11.275285] gk20a gk20a_init_bar1_vm: bar1 inst block ptr: ae84b000
[ 11.281590] gk20a_allocator (gk20a_bar1) gk20a_allocator_init: gk20a_bar1 : base 1, limit 32767, align 1
[ 11.291085] gk20a_allocator (gk20a_bar1-unused) gk20a_allocator_init: gk20a_bar1-unused : base 195936478, limit 195936478, align 1
[ 11.302806] gk20a gk20a_init_mm_setup_sw: done
[ 11.307245] gk20a gk20a_as_alloc_share:
[ 11.311163] gk20a generate_as_share_id:
[ 11.315081] gk20a gk20a_vm_alloc_share:
[ 11.319003] gk20a pde_range_from_vaddr_range: addr_lo=0x0 addr_hi=0x7ffffffff pde_ss=27
[ 11.326995] gk20a pde_range_from_vaddr_range: pde_lo=0 pde_hi=255
[ 11.333111] gk20a gk20a_vm_alloc_share: init space for va_limit=0x800000000 num_pdes=256
[ 11.341189] gk20a alloc_gmmu_pages:
[ 11.344770] gk20a gk20a_vm_alloc_share: pdes.kv = 0xed6db000, pdes.phys = 0xad6db000
[ 11.352532] gk20a_allocator (gk20a_as_1-4KB) gk20a_allocator_init: gk20a_as_1-4KB : base 32768, limit 4194303, align 1
[ 11.363386] gk20a_allocator (gk20a_as_1-128KB) gk20a_allocator_init: gk20a_as_1-128KB : base 131072, limit 262143, align 1
[ 11.374418] gk20a gk20a_as_dev_release:
[ 11.378333] gk20a gk20a_as_release_share:
[ 11.382424] gk20a gk20a_vm_release_share:
[ 11.386510] gk20a gk20a_vm_remove_support:
[ 11.390688] gk20a free_gmmu_pages:
[ 11.394193] gk20a release_as_share_id:
[ 11.398023] gk20a gk20a_deinit_client:
[ 11.401908] gk20a gk20a_as_dev_open:
[ 11.405567] gk20a gk20a_init_client:
[ 11.409226] gk20a gk20a_init_mm_setup_sw:
[ 11.413318] gk20a gk20a_init_mm_setup_sw: skip init
[ 11.418188] gk20a gk20a_as_alloc_share:
[ 11.422106] gk20a generate_as_share_id:
[ 11.426020] gk20a gk20a_vm_alloc_share:
[ 11.429938] gk20a pde_range_from_vaddr_range: addr_lo=0x0 addr_hi=0x7ffffffff pde_ss=27
[ 11.437925] gk20a pde_range_from_vaddr_range: pde_lo=0 pde_hi=255
[ 11.444030] gk20a gk20a_vm_alloc_share: init space for va_limit=0x800000000 num_pdes=256
[ 11.452104] gk20a alloc_gmmu_pages:
[ 11.455676] gk20a gk20a_vm_alloc_share: pdes.kv = 0xed6db000, pdes.phys = 0xad6db000
[ 11.463425] gk20a_allocator (gk20a_as_2-4KB) gk20a_allocator_init: gk20a_as_2-4KB : base 32768, limit 4194303, align 1
[ 11.474119] gk20a_allocator (gk20a_as_2-128KB) gk20a_allocator_init: gk20a_as_2-128KB : base 131072, limit 262143, align 1
[ 11.487576] gk20a gk20a_pm_finalize_poweron:
[ 11.491940] gk20a gk20a_writel: r=0x644 v=0x1100
[ 11.496637] gk20a gk20a_writel: r=0x144 v=0x1
[ 11.501073] gk20a gk20a_writel: r=0x640 v=0x52001100
[ 11.506113] gk20a gk20a_writel: r=0x140 v=0x1
[ 11.510549] gk20a gk20a_writel: r=0x1140 v=0xe
[ 11.515187] gk20a gk20a_readl: r=0x137250 v=0x8110003c
[ 11.520403] gk20a gk20a_writel: r=0x137250 v=0x81100000
[ 11.525705] gk20a gk20a_disable: pmc disable: 00000020
[ 11.525705]
[ 11.532311] gk20a gk20a_readl: r=0x200 v=0xc0010020
[ 11.537266] gk20a gk20a_writel: r=0x200 v=0xc0010000
[ 11.542328] gk20a gk20a_enable: pmc enable: 00000020
[ 11.542328]
[ 11.548758] gk20a gk20a_readl: r=0x200 v=0xc0010000
[ 11.553712] gk20a gk20a_writel: r=0x200 v=0xc0010020
[ 11.558755] gk20a gk20a_readl: r=0x200 v=0xc0010020
[ 11.563730] gk20a gk20a_writel: r=0x12004c v=0x4
[ 11.568425] gk20a gk20a_writel: r=0x122204 v=0x2
[ 11.573123] gk20a gk20a_readl: r=0x122204 v=0x2
[ 11.577731] gk20a gk20a_readl: r=0x0 v=0xea000a1
[ 11.582426] gk20a gk20a_detect_chip: arch: e0, impl: a, rev: a1
[ 11.582426]
[ 11.589809] gk20a gpu_init_hal: gk20a detected
[ 11.594270] gk20a gk20a_init_clk_support:
[ 11.598358] gk20a gk20a_init_clk_reset_enable_hw:
[ 11.603140] gk20a gk20a_init_clk_setup_sw:
[ 11.607332] gk20a gk20a_init_clk_setup_sw: done
[ 11.611854] gk20a gk20a_init_clk_setup_hw:
[ 11.616033] gk20a gk20a_readl: r=0x137250 v=0x81100000
[ 11.621248] gk20a gk20a_writel: r=0x137250 v=0x81100000
[ 11.626562] gk20a set_pll_freq: curr freq: 504MHz, target freq 504MHz
[ 11.632990] gk20a clk_program_gpc_pll:
[ 11.636831] gk20a gk20a_readl: r=0x137004 v=0x14201
[ 11.641803] gk20a gk20a_readl: r=0x137000 v=0x0
[ 11.646417] gk20a gk20a_readl: r=0x137250 v=0x81100000
[ 11.651633] gk20a gk20a_writel: r=0x137250 v=0x81100200
[ 11.657065] gk20a gk20a_readl: r=0x137100 v=0x0
[ 11.661674] gk20a gk20a_writel: r=0x137100 v=0x0
[ 11.666379] gk20a gk20a_readl: r=0x137000 v=0x0
[ 11.670997] gk20a gk20a_readl: r=0x137000 v=0x0
[ 11.675602] gk20a gk20a_writel: r=0x137000 v=0x0
[ 11.680309] gk20a gk20a_readl: r=0x137000 v=0x0
[ 11.684914] gk20a gk20a_writel: r=0x137004 v=0x15401
[ 11.689968] gk20a gk20a_readl: r=0x137000 v=0x0
[ 11.694572] gk20a gk20a_writel: r=0x137000 v=0x1
[ 11.699281] gk20a gk20a_readl: r=0x137000 v=0x1
[ 11.703900] gk20a gk20a_readl: r=0x137000 v=0x20001
[ 11.708857] gk20a gk20a_readl: r=0x137100 v=0x0
[ 11.713461] gk20a gk20a_writel: r=0x137100 v=0x1
[ 11.718161] gk20a gk20a_readl: r=0x137250 v=0x81100200
[ 11.723375] gk20a gk20a_writel: r=0x137250 v=0x81100000
[ 11.728675] gk20a gk20a_readl: r=0x137004 v=0x15401
[ 11.733632] gk20a set_pll_freq: curr freq: 504MHz, target freq 504MHz
[ 11.740058] gk20a gk20a_writel: r=0x9080 v=0x800186a0
[ 11.745184] gk20a gk20a_init_fifo_reset_enable_hw:
[ 11.750049] gk20a gk20a_disable: pmc disable: 00200100
[ 11.750049]
[ 11.756649] gk20a gk20a_readl: r=0x200 v=0xc0010020
[ 11.761600] gk20a gk20a_writel: r=0x200 v=0xc0010020
[ 11.766657] gk20a gk20a_enable: pmc enable: 00200100
[ 11.766657]
[ 11.773084] gk20a gk20a_readl: r=0x200 v=0xc0010020
[ 11.778036] gk20a gk20a_writel: r=0x200 v=0xc0210120
[ 11.783075] gk20a gk20a_readl: r=0x200 v=0xc0210120
[ 11.788047] gk20a gk20a_writel: r=0x204 v=0x1
[ 11.792479] gk20a gk20a_writel: r=0x2100 v=0xffffffff
[ 11.797604] gk20a gk20a_writel: r=0x2140 v=0x7fffffff
[ 11.802729] gk20a gk20a_writel: r=0x2528 v=0x80000000
[ 11.807856] gk20a gk20a_readl: r=0x4013c v=0xefffffff
[ 11.812981] gk20a gk20a_writel: r=0x4013c v=0xeffffeff
[ 11.818193] gk20a gk20a_writel: r=0x40108 v=0xffffffff
[ 11.823405] gk20a gk20a_writel: r=0x4010c v=0xfffffeff
[ 11.828617] gk20a gk20a_writel: r=0x40148 v=0xffffffff
[ 11.833829] gk20a gk20a_writel: r=0x4014c v=0xfffffeff
[ 11.839041] gk20a gk20a_writel: r=0x2a00 v=0xffffffff
[ 11.844167] gk20a gk20a_readl: r=0x2a04 v=0x800003ff
[ 11.849206] gk20a gk20a_writel: r=0x2a04 v=0xbfffffff
[ 11.854333] gk20a gk20a_readl: r=0x4012c v=0x3ff
[ 11.859024] gk20a gk20a_writel: r=0x4012c v=0xffffffff
[ 11.864237] gk20a gk20a_readl: r=0x2a08 v=0x80007fff
[ 11.869276] gk20a gk20a_writel: r=0x2a08 v=0x7fff
[ 11.874055] gk20a gk20a_writel: r=0x2a0c v=0x800186a0
[ 11.879179] gk20a gk20a_init_fifo_reset_enable_hw: done
[ 11.884391] gk20a gk20a_init_mm_reset_enable_hw:
[ 11.889083] gk20a fb_gk20a_reset: reset gk20a fb
[ 11.893689] gk20a gk20a_disable: pmc disable: 2010000c
[ 11.893689]
[ 11.900291] gk20a gk20a_readl: r=0x200 v=0xc0210120
[ 11.905243] gk20a gk20a_writel: r=0x200 v=0xc0210120
[ 11.910302] gk20a gk20a_enable: pmc enable: 2010000c
[ 11.910302]
[ 11.916728] gk20a gk20a_readl: r=0x200 v=0xc0210120
[ 11.921680] gk20a gk20a_writel: r=0x200 v=0xe031012c
[ 11.926719] gk20a gk20a_readl: r=0x200 v=0xe031012c
[ 11.931691] gk20a gk20a_init_mm_setup_sw:
[ 11.935775] gk20a gk20a_init_mm_setup_sw: skip init
[ 11.940640] gk20a gk20a_init_mm_setup_hw:
[ 11.944726] gk20a gk20a_readl: r=0x100c80 v=0x208001
[ 11.949764] gk20a gk20a_writel: r=0x100c80 v=0x208000
[ 11.954967] gk20a gk20a_init_mm_setup_hw: bar1 inst block ptr: 0x000ae84b
[ 11.961741] gk20a gk20a_writel: r=0x1704 v=0x800ae84b
[ 11.966866] gk20a gk20a_mm_fb_flush:
[ 11.970518] gk20a gk20a_writel: r=0x70000 v=0x1
[ 11.975125] gk20a gk20a_readl: r=0x70000 v=0x2
[ 11.979645] gk20a gk20a_mm_fb_flush: fb_flush 0x2
[ 11.984385] gk20a gk20a_readl: r=0x70000 v=0x0
[ 11.988904] gk20a gk20a_mm_fb_flush:
[ 11.992556] gk20a gk20a_writel: r=0x70000 v=0x1
[ 11.997162] gk20a gk20a_readl: r=0x70000 v=0x2
[ 12.001680] gk20a gk20a_mm_fb_flush: fb_flush 0x2
[ 12.006420] gk20a gk20a_readl: r=0x70000 v=0x0
[ 12.010939] gk20a gk20a_init_mm_setup_hw: done
[ 12.015371] gk20a gk20a_init_pmu_support:
[ 12.019455] gk20a gk20a_init_pmu_reset_enable_hw:
[ 12.024234] gk20a pmu_enable_hw:
[ 12.027540] gk20a gk20a_enable: pmc enable: 00002000
[ 12.027540]
[ 12.033967] gk20a gk20a_readl: r=0x200 v=0xe031012c
[ 12.038919] gk20a gk20a_writel: r=0x200 v=0xe031212c
[ 12.043958] gk20a gk20a_readl: r=0x200 v=0xe031212c
[ 12.048931] gk20a gk20a_readl: r=0x10a10c v=0x1
[ 12.053535] gk20a pmu_enable_hw: done
[ 12.057187] gk20a gk20a_init_fifo_setup_sw:
[ 12.061446] gk20a gk20a_init_fifo_setup_sw: size 512 65536
[ 12.061446]
[ 12.068552] gk20a gk20a_init_fifo_setup_sw: addrs ee860000 ae860000
[ 12.068552]
[ 12.076283] gk20a gk20a_gmmu_map: gmmu map 65536
[ 12.076283]
[ 12.082366] gk20a gk20a_vm_alloc_va: size=0x10000 @ pgsz=4KB
[ 12.088013] gk20a_allocator (gk20a_bar1) gk20a_allocator_block_alloc: [in] addr 0, len 16
[ 12.096174] gk20a_allocator (gk20a_bar1) find_free_area: start search addr : 1
[ 12.103380] gk20a_allocator (gk20a_bar1) find_free_area: free space from 1, len 16
[ 12.110931] gk20a_allocator (gk20a_bar1) find_free_area: next free addr: 17
[ 12.117877] gk20a_allocator (gk20a_bar1) merge_block: curr block 1:17
[ 12.124312] gk20a_allocator (gk20a_bar1) link_block: link new block 1:17 between block -1:-1 and block -1:-1
[ 12.134119] gk20a_allocator (gk20a_bar1) gk20a_allocator_block_alloc: [out] addr 1, len 16
[ 12.142365] gk20a gk20a_vm_alloc_va: start_page_nr 1 num_pages 16
[ 12.142365]
[ 12.149918] gk20a gk20a_vm_alloc_va: gk20a_bar1 found addr: 0x1000
[ 12.156086] gk20a __locked_gmmu_map: map offset 1000
[ 12.156086]
[ 12.162511] gk20a __locked_gmmu_map: size 65536
[ 12.162511]
[ 12.168504] gk20a pde_range_from_vaddr_range: addr_lo=0x1000 addr_hi=0x10fff pde_ss=27
[ 12.176403] gk20a pde_range_from_vaddr_range: pde_lo=0 pde_hi=0
[ 12.182308] gk20a __locked_gmmu_map: pde_lo 0 pde_hi 0
[ 12.182308]
[ 12.188907] gk20a validate_gmmu_page_table_gk20a_locked:
[ 12.194293] gk20a validate_gmmu_page_table_gk20a_locked: alloc 4KB ptes for pde 0
[ 12.201759] gk20a zalloc_gmmu_page_table_gk20a:
[ 12.206364] gk20a alloc_gmmu_pages:
[ 12.209972] gk20a zalloc_gmmu_page_table_gk20a: pte = 0xed4b0440, addr=ad740000, size 6
[ 12.217959] gk20a gk20a_writel: r=0x70004 v=0x1
[ 12.222565] gk20a gk20a_readl: r=0x70004 v=0x2
[ 12.227083] gk20a gk20a_mm_l2_invalidate_locked: l2_system_invalidate 0x2
[ 12.233904] gk20a gk20a_readl: r=0x70004 v=0x0
[ 12.238425] gk20a update_gmmu_pde_locked: pde:0 = 0xad7405,0x00000000
[ 12.238425]
[ 12.246325] gk20a pde_range_from_vaddr_range: addr_lo=0x1000 addr_hi=0x10fff pde_ss=27
[ 12.254305] gk20a pde_range_from_vaddr_range: pde_lo=0 pde_hi=0
[ 12.260212] gk20a update_gmmu_ptes_locked: size_idx=0, pde_lo=0, pde_hi=0
[ 12.266985] gk20a pte_index_from_vaddr: addr=0x1000 pte_i=0x1
[ 12.272717] gk20a pte_index_from_vaddr: addr=0x10fff pte_i=0x10
[ 12.278630] gk20a update_gmmu_ptes_locked: pte_lo=1, pte_hi=16
[ 12.284450] gk20a update_gmmu_ptes_locked: pte_cur=1 addr=0x4,ae860000 kind=0 ctag=0 vol=1 refs=1 [0x00000001,0x04ae8601]
[ 12.295383] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.295383]
[ 12.302851] gk20a update_gmmu_ptes_locked: pte_cur=2 addr=0x4,ae861000 kind=0 ctag=0 vol=1 refs=2 [0x00000001,0x04ae8611]
[ 12.313783] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.313783]
[ 12.321250] gk20a update_gmmu_ptes_locked: pte_cur=3 addr=0x4,ae862000 kind=0 ctag=0 vol=1 refs=3 [0x00000001,0x04ae8621]
[ 12.332183] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.332183]
[ 12.339651] gk20a update_gmmu_ptes_locked: pte_cur=4 addr=0x4,ae863000 kind=0 ctag=0 vol=1 refs=4 [0x00000001,0x04ae8631]
[ 12.350583] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.350583]
[ 12.358051] gk20a update_gmmu_ptes_locked: pte_cur=5 addr=0x4,ae864000 kind=0 ctag=0 vol=1 refs=5 [0x00000001,0x04ae8641]
[ 12.368983] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.368983]
[ 12.376451] gk20a update_gmmu_ptes_locked: pte_cur=6 addr=0x4,ae865000 kind=0 ctag=0 vol=1 refs=6 [0x00000001,0x04ae8651]
[ 12.387383] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.387383]
[ 12.394851] gk20a update_gmmu_ptes_locked: pte_cur=7 addr=0x4,ae866000 kind=0 ctag=0 vol=1 refs=7 [0x00000001,0x04ae8661]
[ 12.405782] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.405782]
[ 12.413252] gk20a update_gmmu_ptes_locked: pte_cur=8 addr=0x4,ae867000 kind=0 ctag=0 vol=1 refs=8 [0x00000001,0x04ae8671]
[ 12.424183] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.424183]
[ 12.431651] gk20a update_gmmu_ptes_locked: pte_cur=9 addr=0x4,ae868000 kind=0 ctag=0 vol=1 refs=9 [0x00000001,0x04ae8681]
[ 12.442583] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.442583]
[ 12.450050] gk20a update_gmmu_ptes_locked: pte_cur=10 addr=0x4,ae869000 kind=0 ctag=0 vol=1 refs=10 [0x00000001,0x04ae8691]
[ 12.461165] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.461165]
[ 12.468634] gk20a update_gmmu_ptes_locked: pte_cur=11 addr=0x4,ae86a000 kind=0 ctag=0 vol=1 refs=11 [0x00000001,0x04ae86a1]
[ 12.479739] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.479739]
[ 12.487207] gk20a update_gmmu_ptes_locked: pte_cur=12 addr=0x4,ae86b000 kind=0 ctag=0 vol=1 refs=12 [0x00000001,0x04ae86b1]
[ 12.498312] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.498312]
[ 12.505780] gk20a update_gmmu_ptes_locked: pte_cur=13 addr=0x4,ae86c000 kind=0 ctag=0 vol=1 refs=13 [0x00000001,0x04ae86c1]
[ 12.516886] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.516886]
[ 12.524354] gk20a update_gmmu_ptes_locked: pte_cur=14 addr=0x4,ae86d000 kind=0 ctag=0 vol=1 refs=14 [0x00000001,0x04ae86d1]
[ 12.535459] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.535459]
[ 12.542925] gk20a update_gmmu_ptes_locked: pte_cur=15 addr=0x4,ae86e000 kind=0 ctag=0 vol=1 refs=15 [0x00000001,0x04ae86e1]
[ 12.554106] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.554106]
[ 12.561574] gk20a update_gmmu_ptes_locked: pte_cur=16 addr=0x4,ae86f000 kind=0 ctag=0 vol=1 refs=16 [0x00000001,0x04ae86f1]
[ 12.572678] gk20a update_gmmu_ptes_locked: pte_kv_cur 0xed740000
[ 12.572678]
[ 12.580152] gk20a update_gmmu_ptes_locked: set tlb dirty
[ 12.585451] gk20a gk20a_mm_tlb_invalidate:
[ 12.589623] gk20a __gk20a_mm_tlb_invalidate:
[ 12.593970] gk20a gk20a_readl: r=0x100c80 v=0x208000
[ 12.599009] gk20a gk20a_writel: r=0x100cb8 v=0xad6a90
[ 12.604135] gk20a gk20a_writel: r=0x100cbc v=0x80000001
[ 12.609435] gk20a gk20a_readl: r=0x100c80 v=0x200000
[ 12.614523] gk20a gk20a_readl: r=0x100c80 v=0x208000
[ 12.619563] gk20a gk20a_init_fifo_setup_sw: userd bar1 va = 0x1000
[ 12.626013] gk20a gk20a_readl: r=0x2390 v=0x1
[ 12.630447] gk20a init_engine_info:
[ 12.634013] gk20a gk20a_readl: r=0x22700 v=0x8006183e
[ 12.639139] gk20a gk20a_readl: r=0x22704 v=0x3
[ 12.643658] gk20a init_engine_info: gr info: runlist_id 0
[ 12.649043] gk20a init_engine_info: gr info: engine_id 0
[ 12.654342] gk20a init_engine_info: gr info: pbdma_map[0]=1
[ 12.659900] gk20a init_runlist:
[ 12.663254] gk20a init_runlist: done
[ 12.666828] gk20a gk20a_init_fifo_setup_sw: done
[ 12.671437] gk20a gk20a_init_fifo_setup_hw:
[ 12.675696] gk20a gk20a_init_fifo_setup_hw: test bar1 @ vaddr 0x1000
[ 12.682038] gk20a gk20a_bar1_readl: b=0x1000 v=0xffffffff
[ 12.682045] mc-err: [mcerr] (ptc) csr_ptcr: EMEM decode error on PDE or PTE entry
[ 12.682052] mc-err: [mcerr] status = 0x20063000; addr = 0x00000ac0
[ 12.682052] mc-err: [mcerr] secure: yes, access-type: read, SMMU fault: nr-nw-s
[ 12.708762] cpu vaddr ee860000 1000
[ 12.712239] ptr fbac0000 1000
[ 12.715199] gk20a gk20a_bar1_readl: b=0x1000 v=0xffffffff
[ 12.715202] mc-err: [mcerr] (gpu) csr_gpusrd: EMEM decode error on PDE or PTE entry
[ 12.715208] mc-err: [mcerr] status = 0x60000058; addr = 0xae860000
[ 12.715208] mc-err: [mcerr] secure: no, access-type: read, SMMU fault: nr-nw-s
[ 12.742009] gk20a gk20a.0: gk20a_init_fifo_setup_hw: 1bar1 broken @ gk20a!
[ 12.748870] gk20a gk20a.0: gk20a_pm_finalize_poweron: failed to init gk20a fifo
[ 12.756219] gk20a gk20a_as_dev_release:
[ 12.760135] gk20a gk20a_as_release_share:
[ 12.764224] gk20a gk20a_vm_release_share:
[ 12.768312] gk20a gk20a_vm_remove_support:
[ 12.772490] gk20a free_gmmu_pages:
[ 12.775988] gk20a release_as_share_id:
[ 12.779815] gk20a gk20a_deinit_client:
edit: replaced log with one without an experiment