Nvidia Jetson TX2 freezing completely when configuring flank detection (rising, falling or both) on ...

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

I am trying to configure flank detection on gpio389 and gpio398 in my Nvidia Jetson TX2 by following these steps:

  1. First, go to the gpio folder:
cd /sys/class/gpio
  1. Then export the gpio (for example 398 or 389), which were the ones I wanted to try
echo 389 > export
  1. Then, in the gpio folder, set the gpio as input and the edge as rising (or falling or both)
cd gpio389
echo in > direction
echo rising > edge

during operation I saw that my whole application would eventually stop, in fact, the whole jetson freezed sometimes. By debugging I could tell that this happened when the input reached values of about 1.0-2.0V on any of the gpios 389 or 398.

When I tried to replicate this issue on the gpio298 I could see, that it didn’t happen.

Does someone know if there is a way to stop these intermediate levels of voltage from freezing the whole jetson? By now, I can’t avoid to have sometimes inputs between 1.0V and 2.0V.

Thanks!

2

hello jdgalviss,

couple of questions need your feedback,
thanks

  1. we would like to have more details about your environment setups.
    for example,
    which JetPack release you’re working with.
    what’s additional device you had connected to Jetson platform.

  2. how did you observed “jetson freezed sometimes”,
    could you please also running tegra status at the background to observe the system behavior.

3

Hello Jerry,

thanks a lot for your quick response.

  1. JetPack Version: 3.1
    The device connected is a digital sensor, that usually outputs 0.0V or 3.3V, however it sometimes outputs a voltage between 1.0 and 2.0V which always causes the Jetson to completely freeze (i.e. all programs stop, no mouse or keyboard input and the whole desktop freezes).

  2. Like said, as soon as the signal changes to a value between 1.0V and 2.0V it is no longer possible to give mouse or keyboard input and the whole desktop freezes. This is the result of tegra status:

RAM 1340/7846MB (lfb 1222x4MB) CPU [0%@2035,0%@2035,0%@2035,0%@2035,0%@2035,0%@2035] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.5C VDD_IN 7720/7720 VDD_CPU 2964/2964 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 343/343 VDD_DDR 2082/2082
RAM 1339/7846MB (lfb 1222x4MB) CPU [45%@806,0%@2035,100%@2035,41%@806,36%@806,32%@806] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.7C VDD_IN 7720/7720 VDD_CPU 2888/2926 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 382/362 VDD_DDR 2082/2082
RAM 1340/7846MB (lfb 1222x4MB) CPU [45%@806,0%@2035,100%@2035,38%@806,37%@806,36%@806] BCPU@51C MCPU@51C GPU@49C PLL@51.5C Tboard@46C Tdiode@48.75C PMIC@100C thermal@50.7C VDD_IN 7758/7732 VDD_CPU 2888/2913 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 401/375 VDD_DDR 2082/2082
RAM 1341/7846MB (lfb 1221x4MB) CPU [47%@960,0%@2035,100%@2035,41%@960,31%@960,36%@960] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.75C PMIC@100C thermal@50.5C VDD_IN 7720/7729 VDD_CPU 2964/2926 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 343/367 VDD_DDR 2082/2082
RAM 1341/7846MB (lfb 1221x4MB) CPU [51%@1881,1%@2035,100%@2035,32%@1881,31%@1881,38%@1881] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.7C VDD_IN 7682/7720 VDD_CPU 2964/2933 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 343/362 VDD_DDR 2082/2082
RAM 1341/7846MB (lfb 1221x4MB) CPU [44%@806,0%@2035,100%@2035,31%@806,41%@806,31%@806] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.7C VDD_IN 7682/7713 VDD_CPU 2889/2926 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 343/359 VDD_DDR 2082/2082
RAM 1340/7846MB (lfb 1221x4MB) CPU [42%@1574,0%@2035,100%@2035,34%@1574,42%@1574,38%@960] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.7C VDD_IN 7644/7703 VDD_CPU 2888/2920 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 343/356 VDD_DDR 2082/2082
RAM 1342/7846MB (lfb 1221x4MB) CPU [41%@806,0%@2035,100%@2035,42%@806,35%@806,37%@806] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.5C VDD_IN 7644/7696 VDD_CPU 2888/2916 VDD_GPU 152/152 VDD_SOC 1217/1217 VDD_WIFI 343/355 VDD_DDR 2082/2082

########### Here the output of tegrta status freezes while the signal is between 1.0 and 2.0 V and then returns after the signal goes to 0.0V ##################

RAM 1355/7846MB (lfb 1221x4MB) CPU [95%@1113,8%@2035,100%@2035,40%@1113,85%@1113,35%@1113] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@52C Tboard@46C Tdiode@48.75C PMIC@100C thermal@50.7C VDD_IN 7606/7686 VDD_CPU 2964/2921 VDD_GPU 228/160 VDD_SOC 1217/1217 VDD_WIFI 343/353 VDD_DDR 2025/2075
RAM 1364/7846MB (lfb 1221x4MB) CPU [100%@1420,100%@2035,99%@2035,86%@1420,49%@1420,35%@1420] BCPU@52.5C MCPU@52.5C GPU@49.5C PLL@52.5C Tboard@46C Tdiode@49C PMIC@100C thermal@51.3C VDD_IN 9199/7837 VDD_CPU 4634/3093 VDD_GPU 304/174 VDD_SOC 1216/1216 VDD_WIFI 95/327 VDD_DDR 2005/2068
RAM 1343/7846MB (lfb 1220x4MB) CPU [39%@1881,0%@2035,100%@2035,27%@1881,25%@1881,23%@1881] BCPU@52C MCPU@52C GPU@49.5C PLL@52C Tboard@46C Tdiode@48.75C PMIC@100C thermal@51C VDD_IN 7647/7820 VDD_CPU 2889/3074 VDD_GPU 228/179 VDD_SOC 1217/1216 VDD_WIFI 401/334 VDD_DDR 1986/2061
RAM 1345/7846MB (lfb 1221x4MB) CPU [27%@806,5%@2035,96%@2035,36%@806,30%@806,27%@806] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@49C PMIC@100C thermal@50.7C VDD_IN 7419/7786 VDD_CPU 2813/3052 VDD_GPU 152/177 VDD_SOC 1141/1210 VDD_WIFI 362/336 VDD_DDR 1986/2054
RAM 1345/7846MB (lfb 1220x4MB) CPU [28%@806,100%@2035,0%@2035,26%@806,31%@806,23%@806] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.75C PMIC@100C thermal@50.5C VDD_IN 7457/7761 VDD_CPU 2737/3028 VDD_GPU 152/175 VDD_SOC 1217/1211 VDD_WIFI 343/337 VDD_DDR 2025/2052
RAM 1347/7846MB (lfb 1220x4MB) CPU [31%@2035,100%@2035,1%@2035,30%@2035,36%@2035,30%@2035] BCPU@51.5C MCPU@51.5C GPU@49C PLL@51.5C Tboard@46C Tdiode@48.75C PMIC@100C thermal@50.5C VDD_IN 7644/7753 VDD_CPU 2888/3018 VDD_GPU 152/173 VDD_SOC 1217/1211 VDD_WIFI 343/337 VDD_DDR 2082/2054
RAM 1346/7846MB (lfb 1219x4MB) CPU [37%@1574,100%@2035,0%@2035,35%@1574,30%@1574,34%@1574] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.75C PMIC@100C thermal@50.7C VDD_IN 7720/7750 VDD_CPU 2964/3014 VDD_GPU 152/172 VDD_SOC 1217/1211 VDD_WIFI 343/338 VDD_DDR 2082/2056
RAM 1348/7846MB (lfb 1219x4MB) CPU [41%@806,99%@2035,0%@2035,35%@806,38%@806,32%@806] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.7C VDD_IN 7682/7746 VDD_CPU 2888/3006 VDD_GPU 152/171 VDD_SOC 1217/1212 VDD_WIFI 343/338 VDD_DDR 2082/2058
RAM 1348/7846MB (lfb 1219x4MB) CPU [37%@1267,100%@2035,0%@2035,39%@1267,38%@1267,34%@1267] BCPU@51.5C MCPU@51.5C GPU@49.5C PLL@51.5C Tboard@46C Tdiode@48.5C PMIC@100C thermal@50.7C VDD_IN 7720/7744 VDD_CPU 2888/2999 VDD_GPU 152/169 VDD_SOC 1217/1212 VDD_WIFI 343/338 VDD_DDR 2082/2059

Thanks a lot!

4

hello jdgalviss,

seems like you’re taking GPIO-389 and GPIO-398 as input pin,
could you please also share the pin name for them. thanks

5

Hello Jerry,

That’s right, like I said in the first post, this happens on gpio-389 and 398 (pins 29 and 33 on J21 connector in the DevKit Board).

However it doesn’t happen on gpio 388 or 298. So is this only related to these two gpios (389 and 398)?

Thanks a lot!

6

Hi, what’s the voltage range of the corresponding pins of the device attached to J21? And there is a jumper (J24) to set the input voltage of these two pins to 1.8V or 3.3V, what’s your setting on it?

7

Hi,the voltage range is configured in 3.3V according to the jumper J24. I must also clear out that we have seen this behavior in multiple jetsons, around 10 from the 10 we have tested.

Thanks

8

Can you please use the device to check other pins of J21 which are connected to U7, U22 to see if same as the two ones? And what’s the log when system hang?

9

So, so far I have tried with gpio389, gpio398, gpio388 and gpio298. This issue is clear on the first two (gpio389 and gpio398), however it doesn’t occur with gpio388 and gpio298. I ran dmesg and this is what i got (special attention on line 670: Timeout waiting for hardware interrupt ):

[  547.097814] ---- syncpts ----
[  547.101376] id 4 (disp_d) min 351 max 351 refs 1 (previous client : )
[  547.107976] id 5 (disp_e) min 2 max 2 refs 1 (previous client : )
[  547.114484] id 7 (vblank1) min 32554 max 0 refs 1 (previous client : )
[  547.121338] id 18 (17000000.gp10b_507) min 15930 max 15986 refs 1 (previous client : )
[  547.129979] id 19 (17000000.gp10b_506) min 26 max 26 refs 1 (previous client : )
[  547.138263] id 21 (17000000.gp10b_505) min 1790 max 1796 refs 1 (previous client : 17000000.gp10b_505)

[  547.150483] ---- channels ----
[  547.180106] 
               channel 1 - 15820000.se

[  547.192082] NvHost basic channel registers:
[  547.200275] CMDFIFO_STAT_0:  00002040
[  547.208104] CMDFIFO_RDATA_0: 400d1008
[  547.222910] CMDP_OFFSET_0:   00000000
[  547.226615] CMDP_CLASS_0:    00000000
[  547.238235] CHANNELSTAT_0:   00000000
[  547.241916] The CDMA sync queue is empty.

[  547.260083] 
               channel 2 - 15830000.se

[  547.266558] NvHost basic channel registers:
[  547.281240] CMDFIFO_STAT_0:  00002040
[  547.284939] CMDFIFO_RDATA_0: 04f4202e
[  547.288839] CMDP_OFFSET_0:   00000000
[  547.292552] CMDP_CLASS_0:    00000000
[  547.296241] CHANNELSTAT_0:   00000000
[  547.300351] The CDMA sync queue is empty.

[  547.305922] 
               channel 3 - 15840000.se

[  547.312488] NvHost basic channel registers:
[  547.316704] CMDFIFO_STAT_0:  00002040
[  547.320393] CMDFIFO_RDATA_0: 8823040c
[  547.324088] CMDP_OFFSET_0:   00000000
[  547.327761] CMDP_CLASS_0:    00000000
[  547.331515] CHANNELSTAT_0:   00000000
[  547.335244] The CDMA sync queue is empty.

[  547.340938] 
               ---- host general irq ----

[  547.348333] sync_intc0mask = 0x00000001
[  547.352928] sync_intmask = 0x50000003
[  547.359977] 
               ---- host syncpt irq mask ----

[  547.370022] 
               ---- host syncpt irq status ----

[  547.377283] syncpt_thresh_cpu0_int_status(0) = 0x00040000
[  547.382709] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[  547.389861] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[  547.395311] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[  547.401206] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[  547.406695] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[  547.412785] syncpt_thresh_cpu0_int_status(6) = 0x00000000
[  547.418613] syncpt_thresh_cpu0_int_status(7) = 0x00000000
[  547.426161] syncpt_thresh_cpu0_int_status(8) = 0x00000000
[  547.433176] syncpt_thresh_cpu0_int_status(9) = 0x00000000
[  547.441482] syncpt_thresh_cpu0_int_status(10) = 0x00000000
[  547.446992] syncpt_thresh_cpu0_int_status(11) = 0x00000000
[  547.453048] syncpt_thresh_cpu0_int_status(12) = 0x00000000
[  547.458573] syncpt_thresh_cpu0_int_status(13) = 0x00000000
[  547.464112] syncpt_thresh_cpu0_int_status(14) = 0x00000000
[  547.469659] syncpt_thresh_cpu0_int_status(15) = 0x00000000
[  547.475172] syncpt_thresh_cpu0_int_status(16) = 0x00000000
[  547.481279] syncpt_thresh_cpu0_int_status(17) = 0x00000000
[  547.487118] 17000000.gp10b pbdma 0: 
[  547.493818] id: 2 (tsg), next_id: 2 (tsg) chan status: valid
[  547.503108] PUT: 0000001e03f8eff4 GET: 0000001e03f8cc38 FETCH: 000003fd HEADER: 20000df8

[  547.512757] 17000000.gp10b eng 0: 
[  547.516034] id: 0 (tsg), next_id: 2 (tsg), ctx status: switch 
[  547.521933] busy 

[  547.524057] 17000000.gp10b eng 1: 
[  547.527304] id: 2 (tsg), next_id: 2 (tsg), ctx status: invalid 


[  547.537023] 505-17000000.gp10b, pid 1900, refs: 9: 
[  547.541758] channel status:  in use on_pbdma_and_eng busy
[  547.547906] RAMFC : TOP: 8000001f08008360 PUT: 0000001f08008360 GET: 0000001f08008360 FETCH: 0000001f08008360
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001501 SEMAPHORE 0000001e 035a0070 00000001 00001001

[  547.575599] 506-17000000.gp10b, pid 1167, refs: 2: 
[  547.583188] channel status:  in use idle not busy
[  547.588104] RAMFC : TOP: 8000001f08140138 PUT: 0000001f08140138 GET: 0000001f08140138 FETCH: 0000001f08140138
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001301 SEMAPHORE 00000000 00000000 00000000 00000000

[  547.612128] 507-17000000.gp10b, pid 1167, refs: 96: 
[  547.617550] channel status:  in use on_eng_pending busy
[  547.623292] RAMFC : TOP: 8000001f0803e3c0 PUT: 0000001f0803e3c0 GET: 0000001f0803e3c0 FETCH: 0000001f0803e3c0
               HEADER: 20000074 COUNT: 80018001
               SYNCPOINT 00000000 00001201 SEMAPHORE 0000000d fc001000 23a0e234 01100002

[  547.652124] 508-17000000.gp10b, pid 1173, refs: 2: 
[  547.659767] channel status:  in use idle not busy
[  547.664683] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  547.689137] 509-17000000.gp10b, pid 1173, refs: 2: 
[  547.696517] channel status:  in use idle not busy
[  547.702872] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  547.729308] 510-17000000.gp10b, pid 1173, refs: 2: 
[  547.734071] channel status:  in use idle not busy
[  547.738994] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  547.763794] 511-17000000.gp10b, pid 1173, refs: 2: 
[  547.771297] channel status:  in use idle not busy
[  547.777190] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  547.804788] NV_PGRAPH_STATUS: 0x2000a1
[  547.808589] NV_PGRAPH_STATUS1: 0x0
[  547.811990] NV_PGRAPH_STATUS2: 0x0
[  547.815974] NV_PGRAPH_ENGINE_STATUS: 0x1
[  547.820292] NV_PGRAPH_GRFIFO_STATUS : 0xb0b0001
[  547.828232] NV_PGRAPH_GRFIFO_CONTROL : 0x0
[  547.835097] NV_PGRAPH_PRI_FECS_HOST_INT_STATUS : 0x0
[  547.840315] NV_PGRAPH_EXCEPTION  : 0x0
[  547.844106] NV_PGRAPH_FECS_INTR  : 0x0
[  547.847853] NV_PFIFO_ENGINE_STATUS(GR) : 0x9002f000
[  547.854425] NV_PGRAPH_ACTIVITY0: 0x0
[  547.858263] NV_PGRAPH_ACTIVITY1: 0x200
[  547.862087] NV_PGRAPH_ACTIVITY2: 0x0
[  547.865694] NV_PGRAPH_ACTIVITY4: 0x0
[  547.869287] NV_PGRAPH_PRI_SKED_ACTIVITY: 0x0
[  547.873875] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY0: 0x0
[  547.879922] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY1: 0x0
[  547.886256] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY2: 0x0
[  547.892287] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY3: 0x0
[  547.904220] NV_PGRAPH_PRI_GPC0_TPC0_TPCCS_TPC_ACTIVITY0: 0x0
[  547.910563] NV_PGRAPH_PRI_GPC0_TPC1_TPCCS_TPC_ACTIVITY0: 0x0
[  547.916722] NV_PGRAPH_PRI_GPC0_TPCS_TPCCS_TPC_ACTIVITY0: 0x0
[  547.922616] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY0: 0x0
[  547.927866] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY1: 0x0
[  547.933437] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY2: 0x0
[  547.938724] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY3: 0x0
[  547.944148] NV_PGRAPH_PRI_GPCS_TPC0_TPCCS_TPC_ACTIVITY0: 0x0
[  547.956679] NV_PGRAPH_PRI_GPCS_TPC1_TPCCS_TPC_ACTIVITY0: 0x0
[  547.968665] NV_PGRAPH_PRI_GPCS_TPCS_TPCCS_TPC_ACTIVITY0: 0x0
[  547.974588] NV_PGRAPH_PRI_BE0_BECS_BE_ACTIVITY0: 0x0
[  547.980126] NV_PGRAPH_PRI_BE1_BECS_BE_ACTIVITY0: 0x0
[  547.985419] NV_PGRAPH_PRI_BES_BECS_BE_ACTIVITY0: 0x0
[  547.990437] NV_PGRAPH_PRI_DS_MPIPE_STATUS: 0x0
[  547.995013] NV_PGRAPH_PRI_FE_GO_IDLE_TIMEOUT : 0x800
[  548.011924] NV_PGRAPH_PRI_FE_GO_IDLE_INFO : 0x23000710
[  548.018053] NV_PGRAPH_PRI_GPC0_TPC0_TEX_M_TEX_SUBUNITS_STATUS: 0x0
[  548.024776] NV_PGRAPH_PRI_CWD_FS: 0x201
[  548.040894] NV_PGRAPH_PRI_FE_TPC_FS: 0x3
[  548.044906] NV_PGRAPH_PRI_CWD_GPC_TPC_ID(0): 0x100
[  548.050689] NV_PGRAPH_PRI_CWD_SM_ID(0): 0x100
[  548.055551] NV_PGRAPH_PRI_FECS_CTXSW_STATUS_FE_0: 0x22001
[  548.061284] NV_PGRAPH_PRI_FECS_CTXSW_STATUS_1: 0x140
[  548.067266] NV_PGRAPH_PRI_GPC0_GPCCS_CTXSW_STATUS_GPC_0: 0x0
[  548.075200] NV_PGRAPH_PRI_GPC0_GPCCS_CTXSW_STATUS_1: 0x300
[  548.080864] NV_PGRAPH_PRI_FECS_CTXSW_IDLESTATE : 0xe
[  548.086428] NV_PGRAPH_PRI_GPC0_GPCCS_CTXSW_IDLESTATE : 0xf
[  548.110696] NV_PGRAPH_PRI_FECS_CURRENT_CTX : 0x80260b1e
[  548.115945] NV_PGRAPH_PRI_FECS_NEW_CTX : 0x260b1e
[  548.120729] NV_PGRAPH_PRI_BE0_CROP_STATUS1 : 0x5f00000
[  548.125969] NV_PGRAPH_PRI_BES_CROP_STATUS1 : 0x5f00000
[  548.131142] NV_PGRAPH_PRI_BE0_ZROP_STATUS : 0x0
[  548.135756] NV_PGRAPH_PRI_BE0_ZROP_STATUS2 : 0x0
[  548.140407] NV_PGRAPH_PRI_BES_ZROP_STATUS : 0x0
[  548.145025] NV_PGRAPH_PRI_BES_ZROP_STATUS2 : 0x0
[  548.150233] NV_PGRAPH_PRI_BE0_BECS_BE_EXCEPTION: 0x0
[  548.156144] NV_PGRAPH_PRI_BE0_BECS_BE_EXCEPTION_EN: 0x0
[  548.161724] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_EXCEPTION: 0x0
[  548.171418] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_EXCEPTION_EN: 0x30000
[  548.182868] NV_PGRAPH_PRI_GPC0_TPC0_TPCCS_TPC_EXCEPTION: 0x0
[  548.188545] NV_PGRAPH_PRI_GPC0_TPC0_TPCCS_TPC_EXCEPTION_EN: 0x3
[  548.194509] gk20a 17000000.gp10b: gk20a_set_error_notifier_locked: error notifier set to 8 for ch 505
[  548.204323] gk20a 17000000.gp10b: gk20a_fifo_set_ctx_mmu_error_tsg: TSG 2 generated a mmu fault
[  548.213117] ---- mlocks ----

[  548.217550] ---- syncpts ----
[  548.220556] id 4 (disp_d) min 351 max 351 refs 1 (previous client : )
[  548.227178] id 5 (disp_e) min 2 max 2 refs 1 (previous client : )
[  548.235397] id 7 (vblank1) min 32622 max 0 refs 1 (previous client : )
[  548.241974] id 18 (17000000.gp10b_507) min 15930 max 16018 refs 1 (previous client : )
[  548.249976] id 19 (17000000.gp10b_506) min 26 max 26 refs 1 (previous client : )
[  548.257416] id 21 (17000000.gp10b_505) min 1790 max 1798 refs 1 (previous client : 17000000.gp10b_505)

[  548.268925] ---- channels ----
[  548.271996] 
               channel 1 - 15820000.se

[  548.278576] NvHost basic channel registers:
[  548.282939] CMDFIFO_STAT_0:  00002040
[  548.286637] CMDFIFO_RDATA_0: 400d1008
[  548.290324] CMDP_OFFSET_0:   00000000
[  548.294009] CMDP_CLASS_0:    00000000
[  548.299759] CHANNELSTAT_0:   00000000
[  548.303470] The CDMA sync queue is empty.

[  548.309025] 
               channel 2 - 15830000.se

[  548.315529] NvHost basic channel registers:
[  548.319750] CMDFIFO_STAT_0:  00002040
[  548.323439] CMDFIFO_RDATA_0: 04f4202e
[  548.328427] CMDP_OFFSET_0:   00000000
[  548.332119] CMDP_CLASS_0:    00000000
[  548.335778] CHANNELSTAT_0:   00000000
[  548.339462] The CDMA sync queue is empty.

[  548.345046] 
               channel 3 - 15840000.se

[  548.351675] NvHost basic channel registers:
[  548.355906] CMDFIFO_STAT_0:  00002040
[  548.359603] CMDFIFO_RDATA_0: 8823040c
[  548.365590] CMDP_OFFSET_0:   00000000
[  548.369272] CMDP_CLASS_0:    00000000
[  548.373018] CHANNELSTAT_0:   00000000
[  548.376746] The CDMA sync queue is empty.

[  548.382280] 
               ---- host general irq ----

[  548.389096] sync_intc0mask = 0x00000001
[  548.392955] sync_intmask = 0x50000003
[  548.396630] 
               ---- host syncpt irq mask ----

[  548.403715] 
               ---- host syncpt irq status ----

[  548.410982] syncpt_thresh_cpu0_int_status(0) = 0x00040000
[  548.416446] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[  548.421878] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[  548.427309] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[  548.434852] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[  548.440283] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[  548.445695] syncpt_thresh_cpu0_int_status(6) = 0x00000000
[  548.451112] syncpt_thresh_cpu0_int_status(7) = 0x00000000
[  548.456542] syncpt_thresh_cpu0_int_status(8) = 0x00000000
[  548.461954] syncpt_thresh_cpu0_int_status(9) = 0x00000000
[  548.467367] syncpt_thresh_cpu0_int_status(10) = 0x00000000
[  548.472914] syncpt_thresh_cpu0_int_status(11) = 0x00000000
[  548.478495] syncpt_thresh_cpu0_int_status(12) = 0x00000000
[  548.484094] syncpt_thresh_cpu0_int_status(13) = 0x00000000
[  548.489591] syncpt_thresh_cpu0_int_status(14) = 0x00000000
[  548.495098] syncpt_thresh_cpu0_int_status(15) = 0x00000000
[  548.500609] syncpt_thresh_cpu0_int_status(16) = 0x00000000
[  548.506131] syncpt_thresh_cpu0_int_status(17) = 0x00000000
[  548.511635] 17000000.gp10b pbdma 0: 
[  548.515048] id: 2 (tsg), next_id: 2 (tsg) chan status: valid
[  548.520903] PUT: 0000001e03f8eff4 GET: 0000001e03f8cc38 FETCH: 000003fd HEADER: 20000df8

[  548.530506] 17000000.gp10b eng 0: 
[  548.533757] id: 0 (tsg), next_id: 2 (tsg), ctx status: switch 
[  548.539607] busy 

[  548.541751] 17000000.gp10b eng 1: 
[  548.545002] id: 2 (tsg), next_id: 2 (tsg), ctx status: invalid 


[  548.555310] 505-17000000.gp10b, pid 1900, refs: 11: 
[  548.560132] channel status:  in use on_pbdma_and_eng busy
[  548.565731] RAMFC : TOP: 8000001f08008360 PUT: 0000001f08008360 GET: 0000001f08008360 FETCH: 0000001f08008360
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001501 SEMAPHORE 0000001e 035a0070 00000001 00001001

[  548.589317] 506-17000000.gp10b, pid 1167, refs: 2: 
[  548.594033] channel status:  in use idle not busy
[  548.598939] RAMFC : TOP: 8000001f08140138 PUT: 0000001f08140138 GET: 0000001f08140138 FETCH: 0000001f08140138
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001301 SEMAPHORE 00000000 00000000 00000000 00000000

[  548.622413] 507-17000000.gp10b, pid 1167, refs: 96: 
[  548.627239] channel status:  in use on_eng_pending busy
[  548.632662] RAMFC : TOP: 8000001f0803e3c0 PUT: 0000001f0803e3c0 GET: 0000001f0803e3c0 FETCH: 0000001f0803e3c0
               HEADER: 20000074 COUNT: 80018001
               SYNCPOINT 00000000 00001201 SEMAPHORE 0000000d fc001000 23a0e234 01100002

[  548.656158] 508-17000000.gp10b, pid 1173, refs: 2: 
[  548.660875] channel status:  in use idle not busy
[  548.665767] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  548.689351] 509-17000000.gp10b, pid 1173, refs: 2: 
[  548.694081] channel status:  in use idle not busy
[  548.698984] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  548.722468] 510-17000000.gp10b, pid 1173, refs: 2: 
[  548.727185] channel status:  in use idle not busy
[  548.732103] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  548.755629] 511-17000000.gp10b, pid 1173, refs: 2: 
[  548.760356] channel status:  in use idle not busy
[  548.765256] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  548.789993] gk20a 17000000.gp10b: gk20a_channel_timeout_handler: Job on channel 507 timed out
[  548.798536] ---- mlocks ----

[  548.802971] ---- syncpts ----
[  548.805973] id 4 (disp_d) min 351 max 351 refs 1 (previous client : )
[  548.812449] id 5 (disp_e) min 2 max 2 refs 1 (previous client : )
[  548.818572] id 7 (vblank1) min 32657 max 0 refs 1 (previous client : )
[  548.825140] id 18 (17000000.gp10b_507) min 15930 max 16018 refs 1 (previous client : )
[  548.833124] id 19 (17000000.gp10b_506) min 26 max 26 refs 1 (previous client : )
[  548.840553] id 21 (17000000.gp10b_505) min 1798 max 1798 refs 1 (previous client : 17000000.gp10b_505)

[  548.851949] ---- channels ----
[  548.855497] 
               channel 1 - 15820000.se

[  548.861970] NvHost basic channel registers:
[  548.866165] CMDFIFO_STAT_0:  00002040
[  548.869838] CMDFIFO_RDATA_0: 400d1008
[  548.873510] CMDP_OFFSET_0:   00000000
[  548.877206] CMDP_CLASS_0:    00000000
[  548.880893] CHANNELSTAT_0:   00000000
[  548.884576] The CDMA sync queue is empty.

[  548.890213] 
               channel 2 - 15830000.se

[  548.896702] NvHost basic channel registers:
[  548.900915] CMDFIFO_STAT_0:  00002040
[  548.904599] CMDFIFO_RDATA_0: 04f4202e
[  548.908278] CMDP_OFFSET_0:   00000000
[  548.911935] CMDP_CLASS_0:    00000000
[  548.915610] CHANNELSTAT_0:   00000000
[  548.919281] The CDMA sync queue is empty.

[  548.924884] 
               channel 3 - 15840000.se

[  548.931345] NvHost basic channel registers:
[  548.935544] CMDFIFO_STAT_0:  00002040
[  548.939219] CMDFIFO_RDATA_0: 8823040c
[  548.942919] CMDP_OFFSET_0:   00000000
[  548.946616] CMDP_CLASS_0:    00000000
[  548.950296] CHANNELSTAT_0:   00000000
[  548.953978] The CDMA sync queue is empty.

[  548.959508] 
               ---- host general irq ----

[  548.966237] sync_intc0mask = 0x00000001
[  548.970092] sync_intmask = 0x50000003
[  548.973771] 
               ---- host syncpt irq mask ----

[  548.980839] 
               ---- host syncpt irq status ----

[  548.988087] syncpt_thresh_cpu0_int_status(0) = 0x00240000
[  548.994604] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[  549.000035] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[  549.005452] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[  549.010887] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[  549.016297] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[  549.021713] syncpt_thresh_cpu0_int_status(6) = 0x00000000
[  549.027138] syncpt_thresh_cpu0_int_status(7) = 0x00000000
[  549.032562] syncpt_thresh_cpu0_int_status(8) = 0x00000000
[  549.037979] syncpt_thresh_cpu0_int_status(9) = 0x00000000
[  549.043387] syncpt_thresh_cpu0_int_status(10) = 0x00000000
[  549.048883] syncpt_thresh_cpu0_int_status(11) = 0x00000000
[  549.054399] syncpt_thresh_cpu0_int_status(12) = 0x00000000
[  549.059897] syncpt_thresh_cpu0_int_status(13) = 0x00000000
[  549.065390] syncpt_thresh_cpu0_int_status(14) = 0x00000000
[  549.070885] syncpt_thresh_cpu0_int_status(15) = 0x00000000
[  549.076395] syncpt_thresh_cpu0_int_status(16) = 0x00000000
[  549.081903] syncpt_thresh_cpu0_int_status(17) = 0x00000000
[  549.087416] 17000000.gp10b pbdma 0: 
[  549.090841] id: 2 (tsg), next_id: 2 (tsg) chan status: valid
[  549.096708] PUT: 0000001e03f8eff4 GET: 0000001e03f8cc38 FETCH: 000003fd HEADER: 20000df8

[  549.106312] 17000000.gp10b eng 0: 
[  549.109554] id: 0 (tsg), next_id: 2 (tsg), ctx status: switch 
[  549.115393] busy 

[  549.117534] 17000000.gp10b eng 1: 
[  549.120771] id: 2 (tsg), next_id: 2 (tsg), ctx status: invalid 


[  549.129857] 505-17000000.gp10b, pid 1900, refs: 11: 
[  549.134660] channel status: not in use on_pbdma_and_eng busy
[  549.140508] RAMFC : TOP: 8000001f08008360 PUT: 0000001f08008360 GET: 0000001f08008360 FETCH: 0000001f08008360
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001501 SEMAPHORE 0000001e 035a0070 00000001 00001001

[  549.163999] 506-17000000.gp10b, pid 1167, refs: 2: 
[  549.168716] channel status:  in use idle not busy
[  549.173611] RAMFC : TOP: 8000001f08140138 PUT: 0000001f08140138 GET: 0000001f08140138 FETCH: 0000001f08140138
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001301 SEMAPHORE 00000000 00000000 00000000 00000000

[  549.197093] 507-17000000.gp10b, pid 1167, refs: 97: 
[  549.201898] channel status:  in use on_eng_pending busy
[  549.207314] RAMFC : TOP: 8000001f0803e3c0 PUT: 0000001f0803e3c0 GET: 0000001f0803e3c0 FETCH: 0000001f0803e3c0
               HEADER: 20000074 COUNT: 80018001
               SYNCPOINT 00000000 00001201 SEMAPHORE 0000000d fc001000 23a0e234 01100002

[  549.232034] 508-17000000.gp10b, pid 1173, refs: 2: 
[  549.236780] channel status:  in use idle not busy
[  549.241679] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  549.265176] 509-17000000.gp10b, pid 1173, refs: 2: 
[  549.269886] channel status:  in use idle not busy
[  549.274779] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  549.298256] 510-17000000.gp10b, pid 1173, refs: 2: 
[  549.302984] channel status:  in use idle not busy
[  549.307883] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  549.331364] 511-17000000.gp10b, pid 1173, refs: 2: 
[  549.336078] channel status:  in use idle not busy
[  549.340977] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  549.364898] NV_PGRAPH_STATUS: 0x2000a1
[  549.368679] NV_PGRAPH_STATUS1: 0x0
[  549.372078] NV_PGRAPH_STATUS2: 0x0
[  549.375500] NV_PGRAPH_ENGINE_STATUS: 0x1
[  549.379440] NV_PGRAPH_GRFIFO_STATUS : 0xb0b0001
[  549.383988] NV_PGRAPH_GRFIFO_CONTROL : 0x0
[  549.388097] NV_PGRAPH_PRI_FECS_HOST_INT_STATUS : 0x0
[  549.393075] NV_PGRAPH_EXCEPTION  : 0x0
[  549.396839] NV_PGRAPH_FECS_INTR  : 0x0
[  549.400600] NV_PFIFO_ENGINE_STATUS(GR) : 0x9002f000
[  549.405502] NV_PGRAPH_ACTIVITY0: 0x0
[  549.409095] NV_PGRAPH_ACTIVITY1: 0x200
[  549.412868] NV_PGRAPH_ACTIVITY2: 0x0
[  549.416459] NV_PGRAPH_ACTIVITY4: 0x0
[  549.420032] NV_PGRAPH_PRI_SKED_ACTIVITY: 0x0
[  549.424326] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY0: 0x0
[  549.429573] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY1: 0x0
[  549.434817] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY2: 0x0
[  549.440066] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_ACTIVITY3: 0x0
[  549.445305] NV_PGRAPH_PRI_GPC0_TPC0_TPCCS_TPC_ACTIVITY0: 0x0
[  549.452117] NV_PGRAPH_PRI_GPC0_TPC1_TPCCS_TPC_ACTIVITY0: 0x0
[  549.457797] NV_PGRAPH_PRI_GPC0_TPCS_TPCCS_TPC_ACTIVITY0: 0x0
[  549.463472] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY0: 0x0
[  549.468703] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY1: 0x0
[  549.473941] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY2: 0x0
[  549.479194] NV_PGRAPH_PRI_GPCS_GPCCS_GPC_ACTIVITY3: 0x0
[  549.484440] NV_PGRAPH_PRI_GPCS_TPC0_TPCCS_TPC_ACTIVITY0: 0x0
[  549.490119] NV_PGRAPH_PRI_GPCS_TPC1_TPCCS_TPC_ACTIVITY0: 0x0
[  549.495798] NV_PGRAPH_PRI_GPCS_TPCS_TPCCS_TPC_ACTIVITY0: 0x0
[  549.501475] NV_PGRAPH_PRI_BE0_BECS_BE_ACTIVITY0: 0x0
[  549.506461] NV_PGRAPH_PRI_BE1_BECS_BE_ACTIVITY0: 0x0
[  549.511439] NV_PGRAPH_PRI_BES_BECS_BE_ACTIVITY0: 0x0
[  549.516415] NV_PGRAPH_PRI_DS_MPIPE_STATUS: 0x0
[  549.520870] NV_PGRAPH_PRI_FE_GO_IDLE_TIMEOUT : 0x800
[  549.525848] NV_PGRAPH_PRI_FE_GO_IDLE_INFO : 0x23000710
[  549.531013] NV_PGRAPH_PRI_GPC0_TPC0_TEX_M_TEX_SUBUNITS_STATUS: 0x0
[  549.537207] NV_PGRAPH_PRI_CWD_FS: 0x201
[  549.541052] NV_PGRAPH_PRI_FE_TPC_FS: 0x3
[  549.545003] NV_PGRAPH_PRI_CWD_GPC_TPC_ID(0): 0x100
[  549.549822] NV_PGRAPH_PRI_CWD_SM_ID(0): 0x100
[  549.554200] NV_PGRAPH_PRI_FECS_CTXSW_STATUS_FE_0: 0x22001
[  549.559628] NV_PGRAPH_PRI_FECS_CTXSW_STATUS_1: 0x140
[  549.564619] NV_PGRAPH_PRI_GPC0_GPCCS_CTXSW_STATUS_GPC_0: 0x0
[  549.570297] NV_PGRAPH_PRI_GPC0_GPCCS_CTXSW_STATUS_1: 0x300
[  549.575793] NV_PGRAPH_PRI_FECS_CTXSW_IDLESTATE : 0xe
[  549.580773] NV_PGRAPH_PRI_GPC0_GPCCS_CTXSW_IDLESTATE : 0xf
[  549.586267] NV_PGRAPH_PRI_FECS_CURRENT_CTX : 0x80260b1e
[  549.591499] NV_PGRAPH_PRI_FECS_NEW_CTX : 0x260b1e
[  549.596212] NV_PGRAPH_PRI_BE0_CROP_STATUS1 : 0x5f00000
[  549.601367] NV_PGRAPH_PRI_BES_CROP_STATUS1 : 0x5f00000
[  549.606517] NV_PGRAPH_PRI_BE0_ZROP_STATUS : 0x0
[  549.611075] NV_PGRAPH_PRI_BE0_ZROP_STATUS2 : 0x0
[  549.615708] NV_PGRAPH_PRI_BES_ZROP_STATUS : 0x0
[  549.620253] NV_PGRAPH_PRI_BES_ZROP_STATUS2 : 0x0
[  549.624887] NV_PGRAPH_PRI_BE0_BECS_BE_EXCEPTION: 0x0
[  549.629873] NV_PGRAPH_PRI_BE0_BECS_BE_EXCEPTION_EN: 0x0
[  549.635117] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_EXCEPTION: 0x0
[  549.640355] NV_PGRAPH_PRI_GPC0_GPCCS_GPC_EXCEPTION_EN: 0x30000
[  549.646193] NV_PGRAPH_PRI_GPC0_TPC0_TPCCS_TPC_EXCEPTION: 0x0
[  549.651873] NV_PGRAPH_PRI_GPC0_TPC0_TPCCS_TPC_EXCEPTION_EN: 0x3
[  549.657831] gk20a 17000000.gp10b: gk20a_set_error_notifier_locked: error notifier set to 8 for ch 507
[  549.667118] ---- mlocks ----

[  549.671571] ---- syncpts ----
[  549.674569] id 4 (disp_d) min 351 max 351 refs 1 (previous client : )
[  549.681049] id 5 (disp_e) min 2 max 2 refs 1 (previous client : )
[  549.687171] id 7 (vblank1) min 32709 max 0 refs 1 (previous client : )
[  549.693731] id 18 (17000000.gp10b_507) min 15930 max 16018 refs 1 (previous client : )
[  549.701671] id 19 (17000000.gp10b_506) min 26 max 26 refs 1 (previous client : )
[  549.709086] id 21 (17000000.gp10b_505) min 1798 max 1798 refs 1 (previous client : 17000000.gp10b_505)

[  549.720479] ---- channels ----
[  549.723540] 
               channel 1 - 15820000.se

[  549.730005] NvHost basic channel registers:
[  549.734202] CMDFIFO_STAT_0:  00002040
[  549.737877] CMDFIFO_RDATA_0: 400d1008
[  549.741550] CMDP_OFFSET_0:   00000000
[  549.745237] CMDP_CLASS_0:    00000000
[  549.748925] CHANNELSTAT_0:   00000000
[  549.752640] The CDMA sync queue is empty.

[  549.758168] 
               channel 2 - 15830000.se

[  549.764621] NvHost basic channel registers:
[  549.768828] CMDFIFO_STAT_0:  00002040
[  549.772508] CMDFIFO_RDATA_0: 04f4202e
[  549.776189] CMDP_OFFSET_0:   00000000
[  549.779847] CMDP_CLASS_0:    00000000
[  549.783527] CHANNELSTAT_0:   00000000
[  549.787201] The CDMA sync queue is empty.

[  549.792734] 
               channel 3 - 15840000.se

[  549.799210] NvHost basic channel registers:
[  549.803417] CMDFIFO_STAT_0:  00002040
[  549.807097] CMDFIFO_RDATA_0: 8823040c
[  549.810781] CMDP_OFFSET_0:   00000000
[  549.814473] CMDP_CLASS_0:    00000000
[  549.818151] CHANNELSTAT_0:   00000000
[  549.821834] The CDMA sync queue is empty.

[  549.827363] 
               ---- host general irq ----

[  549.834084] sync_intc0mask = 0x00000001
[  549.837936] sync_intmask = 0x50000003
[  549.841617] 
               ---- host syncpt irq mask ----

[  549.848680] 
               ---- host syncpt irq status ----

[  549.855927] syncpt_thresh_cpu0_int_status(0) = 0x00240000
[  549.861344] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[  549.866757] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[  549.872174] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[  549.877602] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[  549.883027] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[  549.888454] syncpt_thresh_cpu0_int_status(6) = 0x00000000
[  549.893871] syncpt_thresh_cpu0_int_status(7) = 0x00000000
[  549.899294] syncpt_thresh_cpu0_int_status(8) = 0x00000000
[  549.904713] syncpt_thresh_cpu0_int_status(9) = 0x00000000
[  549.910127] syncpt_thresh_cpu0_int_status(10) = 0x00000000
[  549.915624] syncpt_thresh_cpu0_int_status(11) = 0x00000000
[  549.921118] syncpt_thresh_cpu0_int_status(12) = 0x00000000
[  549.926612] syncpt_thresh_cpu0_int_status(13) = 0x00000000
[  549.932115] syncpt_thresh_cpu0_int_status(14) = 0x00000000
[  549.937614] syncpt_thresh_cpu0_int_status(15) = 0x00000000
[  549.943108] syncpt_thresh_cpu0_int_status(16) = 0x00000000
[  549.948614] syncpt_thresh_cpu0_int_status(17) = 0x00000000
[  549.954123] 17000000.gp10b pbdma 0: 
[  549.957548] id: 2 (tsg), next_id: 2 (tsg) chan status: valid
[  549.963406] PUT: 0000001e03f8eff4 GET: 0000001e03f8cc38 FETCH: 000003fd HEADER: 20000df8

[  549.973028] 17000000.gp10b eng 0: 
[  549.976277] id: 0 (tsg), next_id: 2 (tsg), ctx status: switch 
[  549.982123] busy 

[  549.984266] 17000000.gp10b eng 1: 
[  549.987490] id: 2 (tsg), next_id: 2 (tsg), ctx status: invalid 


[  549.996574] 505-17000000.gp10b, pid 1900, refs: 11: 
[  550.001387] channel status: not in use on_pbdma_and_eng busy
[  550.007238] RAMFC : TOP: 8000001f08008360 PUT: 0000001f08008360 GET: 0000001f08008360 FETCH: 0000001f08008360
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001501 SEMAPHORE 0000001e 035a0070 00000001 00001001

[  550.030727] 506-17000000.gp10b, pid 1167, refs: 2: 
[  550.035460] channel status:  in use idle not busy
[  550.040354] RAMFC : TOP: 8000001f08140138 PUT: 0000001f08140138 GET: 0000001f08140138 FETCH: 0000001f08140138
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00001301 SEMAPHORE 00000000 00000000 00000000 00000000

[  550.063832] 507-17000000.gp10b, pid 1167, refs: 97: 
[  550.068629] channel status:  in use on_eng_pending busy
[  550.074044] RAMFC : TOP: 8000001f0803e3c0 PUT: 0000001f0803e3c0 GET: 0000001f0803e3c0 FETCH: 0000001f0803e3c0
               HEADER: 20000074 COUNT: 80018001
               SYNCPOINT 00000000 00001201 SEMAPHORE 0000000d fc001000 23a0e234 01100002

[  550.097545] 508-17000000.gp10b, pid 1173, refs: 2: 
[  550.102275] channel status:  in use idle not busy
[  550.107180] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  550.130656] 509-17000000.gp10b, pid 1173, refs: 2: 
[  550.135428] channel status:  in use idle not busy
[  550.140330] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  550.163829] 510-17000000.gp10b, pid 1173, refs: 2: 
[  550.168554] channel status:  in use idle not busy
[  550.173451] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  550.196949] 511-17000000.gp10b, pid 1173, refs: 2: 
[  550.201669] channel status:  in use idle not busy
[  550.206567] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  550.230432] gk20a 17000000.gp10b: gk20a_fifo_handle_mmu_fault: fake mmu fault on engine 0, engine subid 0 (gpc), client 0 (l1 0), addr 0x00000004:0x90000000, type 1 (pde size), info 0x00000001,inst_ptr 0x1000100000
               
[  550.250955] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_os_r : 0
[  550.258368] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_cpuctl_r : 0x40
[  550.266380] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_idlestate_r : 0x1
[  550.274562] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_mailbox0_r : 0x3f
[  550.282763] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_mailbox1_r : 0x0
[  550.290871] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_irqstat_r : 0x0
[  550.298902] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_irqmode_r : 0x4
[  550.306928] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_irqmask_r : 0x8704
[  550.315205] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_irqdest_r : 0x0
[  550.323228] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_debug1_r : 0x40
[  550.331251] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_debuginfo_r : 0x0
[  550.339445] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(0) : 0x0
[  550.348255] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(1) : 0x1
[  550.357063] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(2) : 0x50009
[  550.366223] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(3) : 0x20
[  550.375107] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(4) : 0x3ffd20
[  550.384335] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(5) : 0x0
[  550.393124] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(6) : 0x0
[  550.401921] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_ctxsw_mailbox_r(7) : 0x0
[  550.410717] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_engctl_r : 0x0
[  550.418663] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_curctx_r : 0x0
[  550.426604] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: gr_fecs_nxtctx_r : 0x0
[  550.434544] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_IMB : 0xbadfbadf
[  550.443347] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_DMB : 0xbadfbadf
[  550.452146] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_CSW : 0xbadfbadf
[  550.460937] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_CTX : 0xbadfbadf
[  550.469734] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_EXCI : 0xbadfbadf
[  550.478623] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_PC : 0xbadfbadf
[  550.487362] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_SP : 0xbadfbadf
[  550.496082] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_PC : 0xbadfbadf
[  550.504809] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_SP : 0xbadfbadf
[  550.513518] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_PC : 0xbadfbadf
[  550.522226] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_SP : 0xbadfbadf
[  550.530950] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_PC : 0xbadfbadf
[  550.539666] gk20a 17000000.gp10b: gk20a_fecs_dump_falcon_stats: FECS_FALCON_REG_SP : 0xbadfbadf
[  550.548394] gk20a 17000000.gp10b: gk20a_fifo_handle_mmu_fault: gr_status_r : 0x2000a1
[  550.562349] gk20a 17000000.gp10b: gk20a_fifo_set_ctx_mmu_error_tsg: TSG 0 generated a mmu fault
[  552.016134] mmc1: Timeout waiting for hardware interrupt.
[  552.021538] sdhci: =========== REGISTER DUMP (mmc1)===========
[  552.027368] sdhci: Sys addr: 0x00000000 | Version:  0x00000404
[  552.033194] sdhci: Blk size: 0x00007040 | Blk cnt:  0x00000000
[  552.039020] sdhci: Argument: 0xa5000040 | Trn mode: 0x00000003
[  552.044845] sdhci: Present:  0x01fb0000 | Host ctl: 0x00000016
[  552.050669] sdhci: Power:    0x00000001 | Blk gap:  0x00000000
[  552.056494] sdhci: Wake-up:  0x00000000 | Clock:    0x00000007
[  552.062318] sdhci: Timeout:  0x0000000e | Int stat: 0x00000003
[  552.068143] sdhci: Int enab: 0x02ff000b | Sig enab: 0x02fc000b
[  552.073968] sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
[  552.079793] sdhci: Caps:     0x3f6cd08c | Caps_1:   0x18006f73
[  552.085619] sdhci: Cmd:      0x0000353a | Max curr: 0x00000000
[  552.091442] sdhci: Host ctl2: 0x0000300b
[  552.095363] sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x00000000fc200010
[  552.101879] sdhci: ===========================================
[  552.108833] dhd_bcmsdh_send_buf: sdio error -1, abort command and terminate frame.
[  552.735836] gk20a 17000000.gp10b: gk20a_fifo_is_valid_engine_id: engine_id is not in active list/invalid -1
[  552.737281] dhdcdc_query_ioctl: dhdcdc_msg failed w/status -5
[  552.737295] CFG80211-ERROR) wl_cfg80211_get_station : NOT assoc, error -5
[  552.741696] xhci-tegra 3530000.xhci: tegra_xhci_mbox_work mailbox command 6
[  552.765182] gk20a 17000000.gp10b: gk20a_fifo_handle_sched_error: fifo sched error : 0x0000000a, failed to find engine

Thanks

10

hello jdgalviss,

looks like some of kernel message has been overwritten, we don’t know what’s causing this syncpt failure.
could you please reproduce this again to gather kernel logs in details.
you may need to use uart console to keep saving these messages, or using below commands to output logs into a file. thanks

sudo cat /proc/kmsg | tee log.txt
11

gpio388 and gpio298 are not routed to level shift U7 or U22, please try other pins routed to level shift.