[227078.417203] nvgpu: 17000000.gp10b nvgpu_set_error_notifier_locked:137 [ERR] error notifier set to 8 for ch 507 [227078.417209] nvgpu: 17000000.gp10b nvgpu_set_error_notifier_locked:137 [ERR] error notifier set to 8 for ch 506 [227078.417212] nvgpu: 17000000.gp10b nvgpu_set_error_notifier_locked:137 [ERR] error notifier set to 8 for ch 505 [227078.417216] nvgpu: 17000000.gp10b nvgpu_set_error_notifier_locked:137 [ERR] error notifier set to 8 for ch 504 [227078.417220] nvgpu: 17000000.gp10b gk20a_fifo_handle_sched_error:2531 [ERR] fifo sched ctxsw timeout error: engine=0, tsg=4, ms=3100 [227078.417342] ---- mlocks ---- [227078.417356] ---- syncpts ---- [227078.417626] ---- channels ---- [227078.417637] channel 2 - 15820000.se [227078.417639] NvHost basic channel registers: [227078.417641] CMDFIFO_STAT_0: 00002040 [227078.417643] CMDFIFO_RDATA_0: 6805f488 [227078.417647] CMDP_OFFSET_0: 00000000 [227078.417648] CMDP_CLASS_0: 00000000 [227078.417650] CHANNELSTAT_0: 00000000 [227078.417652] The CDMA sync queue is empty. [227078.417670] channel 3 - 15830000.se [227078.417672] NvHost basic channel registers: [227078.417674] CMDFIFO_STAT_0: 00002040 [227078.417676] CMDFIFO_RDATA_0: 67222a2e [227078.417678] CMDP_OFFSET_0: 00000000 [227078.417680] CMDP_CLASS_0: 00000000 [227078.417682] CHANNELSTAT_0: 00000000 [227078.417683] The CDMA sync queue is empty. [227078.417687] channel 4 - 15840000.se [227078.417688] NvHost basic channel registers: [227078.417690] CMDFIFO_STAT_0: 00002040 [227078.417692] CMDFIFO_RDATA_0: 46a88004 [227078.417694] CMDP_OFFSET_0: 00000000 [227078.417696] CMDP_CLASS_0: 00000000 [227078.417698] CHANNELSTAT_0: 00000000 [227078.417699] The CDMA sync queue is empty. [227078.417705] ---- host general irq ---- [227078.417707] sync_intc0mask = 0x00000001 [227078.417709] sync_intmask = 0x50000003 [227078.417710] ---- host syncpt irq mask ---- [227078.417712] ---- host syncpt irq status ---- [227078.417714] syncpt_thresh_cpu0_int_status(0) = 0x00000000 [227078.417716] syncpt_thresh_cpu0_int_status(1) = 0x00000000 [227078.417718] syncpt_thresh_cpu0_int_status(2) = 0x00000000 [227078.417720] syncpt_thresh_cpu0_int_status(3) = 0x00000000 [227078.417722] syncpt_thresh_cpu0_int_status(4) = 0x00000000 [227078.417724] syncpt_thresh_cpu0_int_status(5) = 0x00000000 [227078.417726] syncpt_thresh_cpu0_int_status(6) = 0x00000000 [227078.417728] syncpt_thresh_cpu0_int_status(7) = 0x00000000 [227078.417729] syncpt_thresh_cpu0_int_status(8) = 0x00000000 [227078.417731] syncpt_thresh_cpu0_int_status(9) = 0x00000000 [227078.417734] syncpt_thresh_cpu0_int_status(10) = 0x00000000 [227078.417736] syncpt_thresh_cpu0_int_status(11) = 0x00000000 [227078.417738] syncpt_thresh_cpu0_int_status(12) = 0x00000000 [227078.417739] syncpt_thresh_cpu0_int_status(13) = 0x00000000 [227078.417741] syncpt_thresh_cpu0_int_status(14) = 0x00000000 [227078.417743] syncpt_thresh_cpu0_int_status(15) = 0x00000000 [227078.417745] syncpt_thresh_cpu0_int_status(16) = 0x00000000 [227078.417747] syncpt_thresh_cpu0_int_status(17) = 0x00000000 [227078.417752] gp10b pbdma 0: [227078.417755] id: 6 (tsg), next_id: 6 (tsg) chan status: valid [227078.417765] PBDMA_PUT: 00000001002502a4 PBDMA_GET: 0000000100250028 GP_PUT: 00000001 GP_GET: 00000001 FETCH: 00000001 HEADER: 201102a4 HDR: 200120a9 SHADOW0: 00250000 SHADOW1: 0002a601 [227078.417769] gp10b eng 0: [227078.417771] id: 4 (tsg), next_id: 6 (tsg), ctx status: switch [227078.417773] busy [227078.417776] gp10b eng 1: [227078.417778] id: 5 (tsg), next_id: 5 (tsg), ctx status: valid [227078.417940] 498-gp10b, pid 28503, refs 2, deterministic: [227078.417942] channel status: in use pending busy [227078.417946] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 20400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.417951] 499-gp10b, pid 28503, refs 2, deterministic: [227078.417953] channel status: in use pending busy [227078.417957] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 20400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.417961] 500-gp10b, pid 28503, refs 2, deterministic: [227078.417963] channel status: in use pending busy [227078.417966] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 20400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.417971] 501-gp10b, pid 28503, refs 2, deterministic: [227078.417973] channel status: in use pending busy [227078.417976] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 20400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.417980] 502-gp10b, pid 28503, refs 2, deterministic: [227078.417982] channel status: in use on_pbdma_and_eng busy [227078.417985] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 60400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.417989] 503-gp10b, pid 2502, refs 2, deterministic: [227078.417991] channel status: in use pending_acquire busy [227078.417995] RAMFC : TOP: 0000000000000000 PUT: 000000010065dffc GET: 000000010065dfa0 FETCH: 000002010065e41c HEADER: 20140718 COUNT: 04550002 SYNCPOINT 00000000 00000000 SEMAPHORE 00000001 00047e34 1d2c91e6 00081004 [227078.417999] 504-gp10b, pid 2502, refs 2, deterministic: [227078.418000] channel status: in use idle not busy [227078.418004] RAMFC : TOP: 0000000000000000 PUT: 0000000100550294 GET: 0000000100550294 FETCH: 0000020100550294 HEADER: 60400000 COUNT: 84000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.418007] 505-gp10b, pid 2502, refs 2, deterministic: [227078.418009] channel status: in use idle not busy [227078.418012] RAMFC : TOP: 0000000000000000 PUT: 000000010045063c GET: 000000010045063c FETCH: 000002010045063c HEADER: 60400000 COUNT: 84000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.418016] 506-gp10b, pid 2502, refs 2, deterministic: [227078.418018] channel status: in use on_eng_pending busy [227078.418021] RAMFC : TOP: 0000000000000000 PUT: 0000000100364cac GET: 0000000100364b78 FETCH: 0001ba0100365f80 HEADER: 20110180 COUNT: 04550002 SYNCPOINT 00000000 00000000 SEMAPHORE 00000001 00047e24 1d2c91e2 00080004 [227078.418025] 507-gp10b, pid 2502, refs 2, deterministic: [227078.418027] channel status: in use idle not busy [227078.418030] RAMFC : TOP: 0000000000000000 PUT: 000000010026f2c0 GET: 000000010026f2c0 FETCH: 000002010026f2c0 HEADER: 60400000 COUNT: 84000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000001 0002ffb0 00000005 00001004 [227078.418034] 508-gp10b, pid 2325, refs 2: [227078.418035] channel status: in use idle not busy [227078.418039] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 60400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.418042] 509-gp10b, pid 2325, refs 2: [227078.418044] channel status: in use idle not busy [227078.418047] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 60400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.418050] 510-gp10b, pid 2325, refs 2: [227078.418052] channel status: in use idle not busy [227078.418055] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 60400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.418058] 511-gp10b, pid 2325, refs 2: [227078.418060] channel status: in use idle not busy [227078.418063] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000 HEADER: 60400000 COUNT: 00000000 SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000 [227078.418315] nvgpu: 17000000.gp10b gk20a_fifo_handle_mmu_fault_locked:1721 [ERR] fake mmu fault on engine 0, engine subid 0 (gpc), client 12 (rast), addr 0x4891c845000, type 0 (pde), access_type 0x00000003,inst_ptr 0x101114c000 [227078.418320] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:129 [ERR] gr_fecs_os_r : 0 [227078.418323] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:131 [ERR] gr_fecs_cpuctl_r : 0x40 [227078.418325] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:133 [ERR] gr_fecs_idlestate_r : 0x1 [227078.418328] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:135 [ERR] gr_fecs_mailbox0_r : 0x3ff [227078.418331] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:137 [ERR] gr_fecs_mailbox1_r : 0x0 [227078.418333] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:139 [ERR] gr_fecs_irqstat_r : 0x0 [227078.418335] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:141 [ERR] gr_fecs_irqmode_r : 0x4 [227078.418338] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:143 [ERR] gr_fecs_irqmask_r : 0x8704 [227078.418340] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:145 [ERR] gr_fecs_irqdest_r : 0x0 [227078.418343] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:147 [ERR] gr_fecs_debug1_r : 0x40 [227078.418345] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:149 [ERR] gr_fecs_debuginfo_r : 0x0 [227078.418348] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:151 [ERR] gr_fecs_ctxsw_status_1_r : 0x140 [227078.418350] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(0) : 0x4 [227078.418353] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(1) : 0x0 [227078.418356] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(2) : 0x50009 [227078.418358] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(3) : 0x20 [227078.418361] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(4) : 0x1ffda0 [227078.418363] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(5) : 0x0 [227078.418366] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(6) : 0x0 [227078.418368] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(7) : 0x0 [227078.418371] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(8) : 0x0 [227078.418373] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(9) : 0x0 [227078.418376] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(10) : 0x0 [227078.418378] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(11) : 0x0 [227078.418381] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(12) : 0x0 [227078.418383] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(13) : 0x0 [227078.418385] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(14) : 0x0 [227078.418388] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:155 [ERR] gr_fecs_ctxsw_mailbox_r(15) : 0x0 [227078.418390] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:159 [ERR] gr_fecs_engctl_r : 0x0 [227078.418393] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:161 [ERR] gr_fecs_curctx_r : 0x0 [227078.418395] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:163 [ERR] gr_fecs_nxtctx_r : 0x0 [227078.418398] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:169 [ERR] FECS_FALCON_REG_IMB : 0xbadfbadf [227078.418400] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:175 [ERR] FECS_FALCON_REG_DMB : 0xbadfbadf [227078.418403] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:181 [ERR] FECS_FALCON_REG_CSW : 0xbadfbadf [227078.418406] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:187 [ERR] FECS_FALCON_REG_CTX : 0xbadfbadf [227078.418409] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:193 [ERR] FECS_FALCON_REG_EXCI : 0xbadfbadf [227078.418411] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:200 [ERR] FECS_FALCON_REG_PC : 0xbadfbadf [227078.418414] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:206 [ERR] FECS_FALCON_REG_SP : 0xbadfbadf [227078.418417] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:200 [ERR] FECS_FALCON_REG_PC : 0xbadfbadf [227078.418420] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:206 [ERR] FECS_FALCON_REG_SP : 0xbadfbadf [227078.418422] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:200 [ERR] FECS_FALCON_REG_PC : 0xbadfbadf [227078.418425] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:206 [ERR] FECS_FALCON_REG_SP : 0xbadfbadf [227078.418428] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:200 [ERR] FECS_FALCON_REG_PC : 0xbadfbadf [227078.418431] nvgpu: 17000000.gp10b gk20a_fecs_dump_falcon_stats:206 [ERR] FECS_FALCON_REG_SP : 0xbadfbadf [227078.418433] nvgpu: 17000000.gp10b gk20a_fifo_handle_mmu_fault_locked:1726 [ERR] gr_status_r : 0x1000081 [227078.421715] nvgpu: 17000000.gp10b fifo_error_isr:2605 [ERR] channel reset initiated from fifo_error_isr; intr=0x00000100 [227078.465628] nvgpu: 17000000.gp10b gk20a_fifo_tsg_unbind_channel_verify_status:2200 [ERR] Channel 503 to be removed from TSG 5 has NEXT set! [227078.465647] nvgpu: 17000000.gp10b gk20a_tsg_unbind_channel:164 [ERR] Channel 503 unbind failed, tearing down TSG 5