cpu0 locked out for 3ms every 100ms

Hi. On the TX1, I am finding that every 100ms, cpu core0 will be busy for ~3ms which will also delay Cuda calls from threads running on other cores. I have found this to be the case on Jetpack L4t 3.3. Is there a process running which could be causing this? Are Cuda calls dependent on core 0 servicing them?

I don’t know about CUDA (GPU is integrated with the memory controller, and all cores can access the memory controller), but there is a lot of hardware I/O dependent upon core 0. You might mention what I/O your CUDA is dependent upon.

akmal.ali,

How do you observe this case? Do you have any usecase to reproduce issue?

Further investigation:
On a fresh TX1 flashed using Jetpack L4T 3.3.

Run Jetson clocks script

sudo /home/nvidia/jetson_clocks.sh

Turn cores 1-3 off:

echo 0 > /sys/devices/system/cpu/cpu1/online
echo 0 > /sys/devices/system/cpu/cpu2/online
echo 0 > /sys/devices/system/cpu/cpu3/online

At this point, something causes core 0 to remain busy for 3ms every 100ms. I was able to observe this through continuous toggling of gpio pins.

You can also observe by pinging from another computer (usually ping <1ms). Observing delay > 3ms every 100ms.

# replace  with ip address of TX1
sudo ping -D -i 0.001 IP_ADDRESS | grep -v '=0\.\|=1\.'

Further investigation, shows that the scenario occurs when cores 1-3 are turned off and the cpu frequency is set to 1224000 or higher.
i.e.

echo 0 > /sys/devices/system/cpu/cpu1/online
echo 0 > /sys/devices/system/cpu/cpu2/online
echo 0 > /sys/devices/system/cpu/cpu3/online

# Problem:
echo 1224000 > /sys/devices/system/cpu/cpu3/cpufreq/scaling_max_freq
echo 1224000 > /sys/devices/system/cpu/cpu3/cpufreq/scaling_min_freq

e.g output:

sudo ping -D -i 0.001 192.168.2.22 | grep -v '=0\.\|=1\.'
PING 192.168.2.22 (192.168.2.22) 56(84) bytes of data.
[1546508777.695448] 64 bytes from 192.168.2.22: icmp_seq=92 ttl=63 time=3.66 ms
[1546508777.795500] 64 bytes from 192.168.2.22: icmp_seq=189 ttl=63 time=4.00 ms
[1546508777.895389] 64 bytes from 192.168.2.22: icmp_seq=286 ttl=63 time=3.83 ms
[1546508777.995406] 64 bytes from 192.168.2.22: icmp_seq=383 ttl=63 time=3.97 ms
[1546508778.095435] 64 bytes from 192.168.2.22: icmp_seq=480 ttl=63 time=4.38 ms
[1546508778.195419] 64 bytes from 192.168.2.22: icmp_seq=577 ttl=63 time=3.93 ms
[1546508778.295427] 64 bytes from 192.168.2.22: icmp_seq=674 ttl=63 time=3.95 ms
[1546508778.395408] 64 bytes from 192.168.2.22: icmp_seq=771 ttl=63 time=3.92 ms
[1546508778.495420] 64 bytes from 192.168.2.22: icmp_seq=868 ttl=63 time=3.96 ms
[1546508778.595397] 64 bytes from 192.168.2.22: icmp_seq=965 ttl=63 time=3.92 ms
[1546508778.695398] 64 bytes from 192.168.2.22: icmp_seq=1062 ttl=63 time=3.95 ms
[1546508778.795430] 64 bytes from 192.168.2.22: icmp_seq=1159 ttl=63 time=3.97 ms
[1546508778.899415] 64 bytes from 192.168.2.22: icmp_seq=1260 ttl=63 time=3.91 ms
[1546508779.003408] 64 bytes from 192.168.2.22: icmp_seq=1361 ttl=63 time=3.94 ms
[1546508779.103509] 64 bytes from 192.168.2.22: icmp_seq=1458 ttl=63 time=4.03 ms
[1546508779.203425] 64 bytes from 192.168.2.22: icmp_seq=1555 ttl=63 time=3.85 ms
[1546508779.303417] 64 bytes from 192.168.2.22: icmp_seq=1652 ttl=63 time=3.92 ms
[1546508779.403549] 64 bytes from 192.168.2.22: icmp_seq=1749 ttl=63 time=4.06 ms
[1546508779.503407] 64 bytes from 192.168.2.22: icmp_seq=1846 ttl=63 time=3.80 ms
[1546508779.603420] 64 bytes from 192.168.2.22: icmp_seq=1943 ttl=63 time=3.97 ms
[1546508779.707411] 64 bytes from 192.168.2.22: icmp_seq=2044 ttl=63 time=3.93 ms
[1546508779.811417] 64 bytes from 192.168.2.22: icmp_seq=2145 ttl=63 time=3.96 ms
[1546508779.911386] 64 bytes from 192.168.2.22: icmp_seq=2242 ttl=63 time=3.91 ms

As you know, there are still many other processes running on device, if you leave only single cpu core running, it should have chance to delay each process.

BTW, I am not pretty sure why you use “ping” to observe such delay…

How do you observe this 3ms from ping? I don’t see it…

Ping was just another example to illustrate that every 100ms, core 0 appears to be busy doing something for approximately 3/4 ms.

Toggling GPIOs is a better example, see attached picture from signal analyser.

Main point is that, after running jetson_clocks.sh. Core 0 on the TX1 appears to be blocked for ~3ms every 100ms.

Disabling cores 1,2,3 is just used to illustrate the problem better so that the toggleGpio app runs on core 0 and the gaps can be seen in the signal analyser.

[Question? :How to attach images?] I only get two img tags.

p.s

with regards to ping. Network latency from my host machine to the TX1 is 0.6ms. However every 100ms in this state, the reply is 3ms late which I attribute to the cpu being busy and unable to respond to the ping.

[1546511906.301604] 64 bytes from 192.168.2.22: icmp_seq=1 ttl=63 time=0.641 ms
[1546511906.302537] 64 bytes from 192.168.2.22: icmp_seq=2 ttl=63 time=0.617 ms
[1546511906.307278] 64 bytes from 192.168.2.22: icmp_seq=3 ttl=63 time=4.33 ms
[1546511906.307938] 64 bytes from 192.168.2.22: icmp_seq=4 ttl=63 time=0.618 ms
[1546511906.308967] 64 bytes from 192.168.2.22: icmp_seq=5 ttl=63 time=0.633 ms
[1546511906.309974] 64 bytes from 192.168.2.22: icmp_seq=6 ttl=63 time=0.639 ms
[1546511906.310954] 64 bytes from 192.168.2.22: icmp_seq=7 ttl=63 time=0.633 ms
[1546511906.311928] 64 bytes from 192.168.2.22: icmp_seq=8 ttl=63 time=0.611 ms
[1546511906.312954] 64 bytes from 192.168.2.22: icmp_seq=9 ttl=63 time=0.630 ms
[1546511906.314002] 64 bytes from 192.168.2.22: icmp_seq=10 ttl=63 time=0.669 ms
[1546511906.314949] 64 bytes from 192.168.2.22: icmp_seq=11 ttl=63 time=0.627 ms
[1546511906.315926] 64 bytes from 192.168.2.22: icmp_seq=12 ttl=63 time=0.608 ms
[1546511906.316954] 64 bytes from 192.168.2.22: icmp_seq=13 ttl=63 time=0.629 ms
[1546511906.317976] 64 bytes from 192.168.2.22: icmp_seq=14 ttl=63 time=0.639 ms
[1546511906.318968] 64 bytes from 192.168.2.22: icmp_seq=15 ttl=63 time=0.631 ms
[1546511906.319938] 64 bytes from 192.168.2.22: icmp_seq=16 ttl=63 time=0.612 ms
[1546511906.320943] 64 bytes from 192.168.2.22: icmp_seq=17 ttl=63 time=0.614 ms
[1546511906.321967] 64 bytes from 192.168.2.22: icmp_seq=18 ttl=63 time=0.636 ms
[1546511906.322925] 64 bytes from 192.168.2.22: icmp_seq=19 ttl=63 time=0.599 ms
[1546511906.323900] 64 bytes from 192.168.2.22: icmp_seq=20 ttl=63 time=0.579 ms
[1546511906.324929] 64 bytes from 192.168.2.22: icmp_seq=21 ttl=63 time=0.598 ms
[1546511906.325933] 64 bytes from 192.168.2.22: icmp_seq=22 ttl=63 time=0.602 ms
[1546511906.326920] 64 bytes from 192.168.2.22: icmp_seq=23 ttl=63 time=0.594 ms
[1546511906.327901] 64 bytes from 192.168.2.22: icmp_seq=24 ttl=63 time=0.578 ms
[1546511906.328929] 64 bytes from 192.168.2.22: icmp_seq=25 ttl=63 time=0.600 ms
[1546511906.329952] 64 bytes from 192.168.2.22: icmp_seq=26 ttl=63 time=0.613 ms
[1546511906.330930] 64 bytes from 192.168.2.22: icmp_seq=27 ttl=63 time=0.603 ms
[1546511906.331914] 64 bytes from 192.168.2.22: icmp_seq=28 ttl=63 time=0.591 ms
[1546511906.332944] 64 bytes from 192.168.2.22: icmp_seq=29 ttl=63 time=0.603 ms
[1546511906.333945] 64 bytes from 192.168.2.22: icmp_seq=30 ttl=63 time=0.605 ms
[1546511906.334920] 64 bytes from 192.168.2.22: icmp_seq=31 ttl=63 time=0.592 ms
[1546511906.335907] 64 bytes from 192.168.2.22: icmp_seq=32 ttl=63 time=0.579 ms
[1546511906.336942] 64 bytes from 192.168.2.22: icmp_seq=33 ttl=63 time=0.601 ms
[1546511906.337950] 64 bytes from 192.168.2.22: icmp_seq=34 ttl=63 time=0.607 ms
[1546511906.338923] 64 bytes from 192.168.2.22: icmp_seq=35 ttl=63 time=0.595 ms
[1546511906.339908] 64 bytes from 192.168.2.22: icmp_seq=36 ttl=63 time=0.578 ms
[1546511906.340942] 64 bytes from 192.168.2.22: icmp_seq=37 ttl=63 time=0.600 ms
[1546511906.341950] 64 bytes from 192.168.2.22: icmp_seq=38 ttl=63 time=0.606 ms
[1546511906.342924] 64 bytes from 192.168.2.22: icmp_seq=39 ttl=63 time=0.592 ms
[1546511906.343907] 64 bytes from 192.168.2.22: icmp_seq=40 ttl=63 time=0.574 ms
[1546511906.344912] 64 bytes from 192.168.2.22: icmp_seq=41 ttl=63 time=0.576 ms
[1546511906.345942] 64 bytes from 192.168.2.22: icmp_seq=42 ttl=63 time=0.603 ms
[1546511906.346952] 64 bytes from 192.168.2.22: icmp_seq=43 ttl=63 time=0.622 ms
[1546511906.347912] 64 bytes from 192.168.2.22: icmp_seq=44 ttl=63 time=0.577 ms
[1546511906.348947] 64 bytes from 192.168.2.22: icmp_seq=45 ttl=63 time=0.598 ms
[1546511906.349924] 64 bytes from 192.168.2.22: icmp_seq=46 ttl=63 time=0.590 ms
[1546511906.350912] 64 bytes from 192.168.2.22: icmp_seq=47 ttl=63 time=0.576 ms
[1546511906.351906] 64 bytes from 192.168.2.22: icmp_seq=48 ttl=63 time=0.574 ms
[1546511906.352939] 64 bytes from 192.168.2.22: icmp_seq=49 ttl=63 time=0.596 ms
[1546511906.353923] 64 bytes from 192.168.2.22: icmp_seq=50 ttl=63 time=0.581 ms
[1546511906.354925] 64 bytes from 192.168.2.22: icmp_seq=51 ttl=63 time=0.589 ms
[1546511906.355910] 64 bytes from 192.168.2.22: icmp_seq=52 ttl=63 time=0.574 ms
[1546511906.356945] 64 bytes from 192.168.2.22: icmp_seq=53 ttl=63 time=0.599 ms
[1546511906.357930] 64 bytes from 192.168.2.22: icmp_seq=54 ttl=63 time=0.592 ms
[1546511906.358919] 64 bytes from 192.168.2.22: icmp_seq=55 ttl=63 time=0.583 ms
[1546511906.359903] 64 bytes from 192.168.2.22: icmp_seq=56 ttl=63 time=0.570 ms
[1546511906.360947] 64 bytes from 192.168.2.22: icmp_seq=57 ttl=63 time=0.605 ms
[1546511906.361974] 64 bytes from 192.168.2.22: icmp_seq=58 ttl=63 time=0.626 ms
[1546511906.362935] 64 bytes from 192.168.2.22: icmp_seq=59 ttl=63 time=0.600 ms
[1546511906.363912] 64 bytes from 192.168.2.22: icmp_seq=60 ttl=63 time=0.579 ms
[1546511906.364959] 64 bytes from 192.168.2.22: icmp_seq=61 ttl=63 time=0.606 ms
[1546511906.365965] 64 bytes from 192.168.2.22: icmp_seq=62 ttl=63 time=0.613 ms
[1546511906.366939] 64 bytes from 192.168.2.22: icmp_seq=63 ttl=63 time=0.599 ms
[1546511906.367933] 64 bytes from 192.168.2.22: icmp_seq=64 ttl=63 time=0.595 ms
[1546511906.368956] 64 bytes from 192.168.2.22: icmp_seq=65 ttl=63 time=0.606 ms
[1546511906.369933] 64 bytes from 192.168.2.22: icmp_seq=66 ttl=63 time=0.591 ms
[1546511906.370923] 64 bytes from 192.168.2.22: icmp_seq=67 ttl=63 time=0.586 ms
[1546511906.371919] 64 bytes from 192.168.2.22: icmp_seq=68 ttl=63 time=0.580 ms
[1546511906.372960] 64 bytes from 192.168.2.22: icmp_seq=69 ttl=63 time=0.610 ms
[1546511906.373960] 64 bytes from 192.168.2.22: icmp_seq=70 ttl=63 time=0.610 ms
[1546511906.374936] 64 bytes from 192.168.2.22: icmp_seq=71 ttl=63 time=0.599 ms
[1546511906.375919] 64 bytes from 192.168.2.22: icmp_seq=72 ttl=63 time=0.578 ms
[1546511906.376971] 64 bytes from 192.168.2.22: icmp_seq=73 ttl=63 time=0.618 ms
[1546511906.378024] 64 bytes from 192.168.2.22: icmp_seq=74 ttl=63 time=0.668 ms
[1546511906.378943] 64 bytes from 192.168.2.22: icmp_seq=75 ttl=63 time=0.605 ms
[1546511906.379926] 64 bytes from 192.168.2.22: icmp_seq=76 ttl=63 time=0.582 ms
[1546511906.380974] 64 bytes from 192.168.2.22: icmp_seq=77 ttl=63 time=0.616 ms
[1546511906.381957] 64 bytes from 192.168.2.22: icmp_seq=78 ttl=63 time=0.609 ms
[1546511906.382941] 64 bytes from 192.168.2.22: icmp_seq=79 ttl=63 time=0.600 ms
[1546511906.383927] 64 bytes from 192.168.2.22: icmp_seq=80 ttl=63 time=0.582 ms
[1546511906.384959] 64 bytes from 192.168.2.22: icmp_seq=81 ttl=63 time=0.606 ms
[1546511906.385963] 64 bytes from 192.168.2.22: icmp_seq=82 ttl=63 time=0.609 ms
[1546511906.386958] 64 bytes from 192.168.2.22: icmp_seq=83 ttl=63 time=0.605 ms
[1546511906.387971] 64 bytes from 192.168.2.22: icmp_seq=84 ttl=63 time=0.608 ms
[1546511906.388969] 64 bytes from 192.168.2.22: icmp_seq=85 ttl=63 time=0.607 ms
[1546511906.389974] 64 bytes from 192.168.2.22: icmp_seq=86 ttl=63 time=0.611 ms
[1546511906.390965] 64 bytes from 192.168.2.22: icmp_seq=87 ttl=63 time=0.607 ms
[1546511906.391941] 64 bytes from 192.168.2.22: icmp_seq=88 ttl=63 time=0.582 ms
[1546511906.392963] 64 bytes from 192.168.2.22: icmp_seq=89 ttl=63 time=0.604 ms
[1546511906.393972] 64 bytes from 192.168.2.22: icmp_seq=90 ttl=63 time=0.609 ms
[1546511906.394973] 64 bytes from 192.168.2.22: icmp_seq=91 ttl=63 time=0.623 ms
[1546511906.395996] 64 bytes from 192.168.2.22: icmp_seq=92 ttl=63 time=0.636 ms
[1546511906.397009] 64 bytes from 192.168.2.22: icmp_seq=93 ttl=63 time=0.645 ms
[1546511906.398022] 64 bytes from 192.168.2.22: icmp_seq=94 ttl=63 time=0.658 ms
[1546511906.399006] 64 bytes from 192.168.2.22: icmp_seq=95 ttl=63 time=0.646 ms
[1546511906.399972] 64 bytes from 192.168.2.22: icmp_seq=96 ttl=63 time=0.611 ms
[1546511906.400963] 64 bytes from 192.168.2.22: icmp_seq=97 ttl=63 time=0.603 ms
[1546511906.401975] 64 bytes from 192.168.2.22: icmp_seq=98 ttl=63 time=0.611 ms
[1546511906.402968] 64 bytes from 192.168.2.22: icmp_seq=99 ttl=63 time=0.605 ms
[1546511906.407258] 64 bytes from 192.168.2.22: icmp_seq=100 ttl=63 time=3.89 ms
[1546511906.407902] 64 bytes from 192.168.2.22: icmp_seq=101 ttl=63 time=0.601 ms
[1546511906.408903] 64 bytes from 192.168.2.22: icmp_seq=102 ttl=63 time=0.603 ms
[1546511906.409895] 64 bytes from 192.168.2.22: icmp_seq=103 ttl=63 time=0.594 ms
[1546511906.410894] 64 bytes from 192.168.2.22: icmp_seq=104 ttl=63 time=0.594 ms
[1546511906.411907] 64 bytes from 192.168.2.22: icmp_seq=105 ttl=63 time=0.596 ms
[1546511906.412907] 64 bytes from 192.168.2.22: icmp_seq=106 ttl=63 time=0.602 ms
[1546511906.413861] 64 bytes from 192.168.2.22: icmp_seq=107 ttl=63 time=0.559 ms
[1546511906.414929] 64 bytes from 192.168.2.22: icmp_seq=108 ttl=63 time=0.629 ms
[1546511906.415932] 64 bytes from 192.168.2.22: icmp_seq=109 ttl=63 time=0.636 ms
[1546511906.416940] 64 bytes from 192.168.2.22: icmp_seq=110 ttl=63 time=0.635 ms
[1546511906.417918] 64 bytes from 192.168.2.22: icmp_seq=111 ttl=63 time=0.612 ms
[1546511906.418888] 64 bytes from 192.168.2.22: icmp_seq=112 ttl=63 time=0.590 ms
[1546511906.419874] 64 bytes from 192.168.2.22: icmp_seq=113 ttl=63 time=0.577 ms
[1546511906.420920] 64 bytes from 192.168.2.22: icmp_seq=114 ttl=63 time=0.613 ms
[1546511906.421917] 64 bytes from 192.168.2.22: icmp_seq=115 ttl=63 time=0.609 ms
[1546511906.422898] 64 bytes from 192.168.2.22: icmp_seq=116 ttl=63 time=0.604 ms
[1546511906.423911] 64 bytes from 192.168.2.22: icmp_seq=117 ttl=63 time=0.613 ms
[1546511906.424940] 64 bytes from 192.168.2.22: icmp_seq=118 ttl=63 time=0.636 ms
[1546511906.425970] 64 bytes from 192.168.2.22: icmp_seq=119 ttl=63 time=0.655 ms
[1546511906.426931] 64 bytes from 192.168.2.22: icmp_seq=120 ttl=63 time=0.631 ms
[1546511906.427908] 64 bytes from 192.168.2.22: icmp_seq=121 ttl=63 time=0.608 ms
[1546511906.428920] 64 bytes from 192.168.2.22: icmp_seq=122 ttl=63 time=0.606 ms
[1546511906.429921] 64 bytes from 192.168.2.22: icmp_seq=123 ttl=63 time=0.599 ms
[1546511906.430894] 64 bytes from 192.168.2.22: icmp_seq=124 ttl=63 time=0.585 ms
[1546511906.431881] 64 bytes from 192.168.2.22: icmp_seq=125 ttl=63 time=0.569 ms
[1546511906.432946] 64 bytes from 192.168.2.22: icmp_seq=126 ttl=63 time=0.622 ms
[1546511906.433936] 64 bytes from 192.168.2.22: icmp_seq=127 ttl=63 time=0.613 ms
[1546511906.434900] 64 bytes from 192.168.2.22: icmp_seq=128 ttl=63 time=0.589 ms
[1546511906.435890] 64 bytes from 192.168.2.22: icmp_seq=129 ttl=63 time=0.578 ms
[1546511906.436939] 64 bytes from 192.168.2.22: icmp_seq=130 ttl=63 time=0.612 ms
[1546511906.437937] 64 bytes from 192.168.2.22: icmp_seq=131 ttl=63 time=0.611 ms
[1546511906.438926] 64 bytes from 192.168.2.22: icmp_seq=132 ttl=63 time=0.600 ms
[1546511906.439889] 64 bytes from 192.168.2.22: icmp_seq=133 ttl=63 time=0.576 ms
[1546511906.440916] 64 bytes from 192.168.2.22: icmp_seq=134 ttl=63 time=0.597 ms
[1546511906.441919] 64 bytes from 192.168.2.22: icmp_seq=135 ttl=63 time=0.599 ms
[1546511906.442922] 64 bytes from 192.168.2.22: icmp_seq=136 ttl=63 time=0.602 ms
[1546511906.443901] 64 bytes from 192.168.2.22: icmp_seq=137 ttl=63 time=0.583 ms
[1546511906.444922] 64 bytes from 192.168.2.22: icmp_seq=138 ttl=63 time=0.601 ms
[1546511906.445898] 64 bytes from 192.168.2.22: icmp_seq=139 ttl=63 time=0.582 ms
[1546511906.446913] 64 bytes from 192.168.2.22: icmp_seq=140 ttl=63 time=0.590 ms
[1546511906.447906] 64 bytes from 192.168.2.22: icmp_seq=141 ttl=63 time=0.582 ms
[1546511906.448893] 64 bytes from 192.168.2.22: icmp_seq=142 ttl=63 time=0.582 ms
[1546511906.449922] 64 bytes from 192.168.2.22: icmp_seq=143 ttl=63 time=0.596 ms
[1546511906.450900] 64 bytes from 192.168.2.22: icmp_seq=144 ttl=63 time=0.581 ms
[1546511906.451891] 64 bytes from 192.168.2.22: icmp_seq=145 ttl=63 time=0.579 ms
[1546511906.452887] 64 bytes from 192.168.2.22: icmp_seq=146 ttl=63 time=0.575 ms
[1546511906.453885] 64 bytes from 192.168.2.22: icmp_seq=147 ttl=63 time=0.574 ms
[1546511906.454885] 64 bytes from 192.168.2.22: icmp_seq=148 ttl=63 time=0.575 ms
[1546511906.455889] 64 bytes from 192.168.2.22: icmp_seq=149 ttl=63 time=0.578 ms
[1546511906.456885] 64 bytes from 192.168.2.22: icmp_seq=150 ttl=63 time=0.575 ms
[1546511906.457875] 64 bytes from 192.168.2.22: icmp_seq=151 ttl=63 time=0.566 ms
[1546511906.458875] 64 bytes from 192.168.2.22: icmp_seq=152 ttl=63 time=0.566 ms
[1546511906.459884] 64 bytes from 192.168.2.22: icmp_seq=153 ttl=63 time=0.573 ms
[1546511906.460882] 64 bytes from 192.168.2.22: icmp_seq=154 ttl=63 time=0.572 ms
[1546511906.461875] 64 bytes from 192.168.2.22: icmp_seq=155 ttl=63 time=0.565 ms
[1546511906.462881] 64 bytes from 192.168.2.22: icmp_seq=156 ttl=63 time=0.572 ms
[1546511906.463883] 64 bytes from 192.168.2.22: icmp_seq=157 ttl=63 time=0.572 ms
[1546511906.464882] 64 bytes from 192.168.2.22: icmp_seq=158 ttl=63 time=0.572 ms
[1546511906.465883] 64 bytes from 192.168.2.22: icmp_seq=159 ttl=63 time=0.572 ms
[1546511906.466883] 64 bytes from 192.168.2.22: icmp_seq=160 ttl=63 time=0.572 ms
[1546511906.467886] 64 bytes from 192.168.2.22: icmp_seq=161 ttl=63 time=0.575 ms
[1546511906.468884] 64 bytes from 192.168.2.22: icmp_seq=162 ttl=63 time=0.574 ms
[1546511906.469882] 64 bytes from 192.168.2.22: icmp_seq=163 ttl=63 time=0.572 ms
[1546511906.470883] 64 bytes from 192.168.2.22: icmp_seq=164 ttl=63 time=0.573 ms
[1546511906.471893] 64 bytes from 192.168.2.22: icmp_seq=165 ttl=63 time=0.577 ms
[1546511906.472885] 64 bytes from 192.168.2.22: icmp_seq=166 ttl=63 time=0.573 ms
[1546511906.473883] 64 bytes from 192.168.2.22: icmp_seq=167 ttl=63 time=0.572 ms
[1546511906.474884] 64 bytes from 192.168.2.22: icmp_seq=168 ttl=63 time=0.571 ms
[1546511906.475915] 64 bytes from 192.168.2.22: icmp_seq=169 ttl=63 time=0.581 ms
[1546511906.476930] 64 bytes from 192.168.2.22: icmp_seq=170 ttl=63 time=0.601 ms
[1546511906.477928] 64 bytes from 192.168.2.22: icmp_seq=171 ttl=63 time=0.600 ms
[1546511906.478933] 64 bytes from 192.168.2.22: icmp_seq=172 ttl=63 time=0.600 ms
[1546511906.479938] 64 bytes from 192.168.2.22: icmp_seq=173 ttl=63 time=0.603 ms
[1546511906.480908] 64 bytes from 192.168.2.22: icmp_seq=174 ttl=63 time=0.584 ms
[1546511906.481928] 64 bytes from 192.168.2.22: icmp_seq=175 ttl=63 time=0.597 ms
[1546511906.482936] 64 bytes from 192.168.2.22: icmp_seq=176 ttl=63 time=0.601 ms
[1546511906.483912] 64 bytes from 192.168.2.22: icmp_seq=177 ttl=63 time=0.580 ms
[1546511906.484933] 64 bytes from 192.168.2.22: icmp_seq=178 ttl=63 time=0.599 ms
[1546511906.485942] 64 bytes from 192.168.2.22: icmp_seq=179 ttl=63 time=0.600 ms
[1546511906.486942] 64 bytes from 192.168.2.22: icmp_seq=180 ttl=63 time=0.599 ms
[1546511906.487954] 64 bytes from 192.168.2.22: icmp_seq=181 ttl=63 time=0.618 ms
[1546511906.488948] 64 bytes from 192.168.2.22: icmp_seq=182 ttl=63 time=0.600 ms
[1546511906.489953] 64 bytes from 192.168.2.22: icmp_seq=183 ttl=63 time=0.604 ms
[1546511906.490957] 64 bytes from 192.168.2.22: icmp_seq=184 ttl=63 time=0.607 ms
[1546511906.491975] 64 bytes from 192.168.2.22: icmp_seq=185 ttl=63 time=0.615 ms
[1546511906.492958] 64 bytes from 192.168.2.22: icmp_seq=186 ttl=63 time=0.609 ms
[1546511906.493959] 64 bytes from 192.168.2.22: icmp_seq=187 ttl=63 time=0.609 ms
[1546511906.494955] 64 bytes from 192.168.2.22: icmp_seq=188 ttl=63 time=0.623 ms
[1546511906.495997] 64 bytes from 192.168.2.22: icmp_seq=189 ttl=63 time=0.643 ms
[1546511906.496956] 64 bytes from 192.168.2.22: icmp_seq=190 ttl=63 time=0.614 ms
[1546511906.497959] 64 bytes from 192.168.2.22: icmp_seq=191 ttl=63 time=0.608 ms
[1546511906.498953] 64 bytes from 192.168.2.22: icmp_seq=192 ttl=63 time=0.606 ms
[1546511906.499964] 64 bytes from 192.168.2.22: icmp_seq=193 ttl=63 time=0.612 ms
[1546511906.500962] 64 bytes from 192.168.2.22: icmp_seq=194 ttl=63 time=0.610 ms
[1546511906.501976] 64 bytes from 192.168.2.22: icmp_seq=195 ttl=63 time=0.625 ms
[1546511906.502960] 64 bytes from 192.168.2.22: icmp_seq=196 ttl=63 time=0.609 ms
[1546511906.507284] 64 bytes from 192.168.2.22: icmp_seq=197 ttl=63 time=3.93 ms
[1546511906.507971] 64 bytes from 192.168.2.22: icmp_seq=198 ttl=63 time=0.642 ms
[1546511906.508978] 64 bytes from 192.168.2.22: icmp_seq=199 ttl=63 time=0.646 ms
[1546511906.509973] 64 bytes from 192.168.2.22: icmp_seq=200 ttl=63 time=0.644 ms
[1546511906.510976] 64 bytes from 192.168.2.22: icmp_seq=201 ttl=63 time=0.647 ms
[1546511906.511991] 64 bytes from 192.168.2.22: icmp_seq=202 ttl=63 time=0.649 ms
[1546511906.512959] 64 bytes from 192.168.2.22: icmp_seq=203 ttl=63 time=0.618 ms
[1546511906.513952] 64 bytes from 192.168.2.22: icmp_seq=204 ttl=63 time=0.608 ms
[1546511906.514960] 64 bytes from 192.168.2.22: icmp_seq=205 ttl=63 time=0.613 ms
[1546511906.515966] 64 bytes from 192.168.2.22: icmp_seq=206 ttl=63 time=0.613 ms
[1546511906.516966] 64 bytes from 192.168.2.22: icmp_seq=207 ttl=63 time=0.613 ms
[1546511906.517959] 64 bytes from 192.168.2.22: icmp_seq=208 ttl=63 time=0.609 ms
[1546511906.518946] 64 bytes from 192.168.2.22: icmp_seq=209 ttl=63 time=0.596 ms
[1546511906.519921] 64 bytes from 192.168.2.22: icmp_seq=210 ttl=63 time=0.584 ms
[1546511906.520908] 64 bytes from 192.168.2.22: icmp_seq=211 ttl=63 time=0.579 ms
[1546511906.521922] 64 bytes from 192.168.2.22: icmp_seq=212 ttl=63 time=0.589 ms
[1546511906.522911] 64 bytes from 192.168.2.22: icmp_seq=213 ttl=63 time=0.578 ms
[1546511906.523960] 64 bytes from 192.168.2.22: icmp_seq=214 ttl=63 time=0.608 ms
[1546511906.524964] 64 bytes from 192.168.2.22: icmp_seq=215 ttl=63 time=0.610 ms
[1546511906.525969] 64 bytes from 192.168.2.22: icmp_seq=216 ttl=63 time=0.614 ms
[1546511906.526960] 64 bytes from 192.168.2.22: icmp_seq=217 ttl=63 time=0.611 ms
[1546511906.527968] 64 bytes from 192.168.2.22: icmp_seq=218 ttl=63 time=0.611 ms
[1546511906.528967] 64 bytes from 192.168.2.22: icmp_seq=219 ttl=63 time=0.609 ms
[1546511906.529953] 64 bytes from 192.168.2.22: icmp_seq=220 ttl=63 time=0.604 ms
[1546511906.530970] 64 bytes from 192.168.2.22: icmp_seq=221 ttl=63 time=0.612 ms
[1546511906.531973] 64 bytes from 192.168.2.22: icmp_seq=222 ttl=63 time=0.613 ms
[1546511906.532970] 64 bytes from 192.168.2.22: icmp_seq=223 ttl=63 time=0.609 ms
[1546511906.533954] 64 bytes from 192.168.2.22: icmp_seq=224 ttl=63 time=0.604 ms
[1546511906.534955] 64 bytes from 192.168.2.22: icmp_seq=225 ttl=63 time=0.605 ms
[1546511906.535945] 64 bytes from 192.168.2.22: icmp_seq=226 ttl=63 time=0.589 ms
[1546511906.536958] 64 bytes from 192.168.2.22: icmp_seq=227 ttl=63 time=0.608 ms
[1546511906.538000] 64 bytes from 192.168.2.22: icmp_seq=228 ttl=63 time=0.640 ms
[1546511906.538957] 64 bytes from 192.168.2.22: icmp_seq=229 ttl=63 time=0.606 ms
[1546511906.539959] 64 bytes from 192.168.2.22: icmp_seq=230 ttl=63 time=0.606 ms
[1546511906.540975] 64 bytes from 192.168.2.22: icmp_seq=231 ttl=63 time=0.616 ms
[1546511906.541970] 64 bytes from 192.168.2.22: icmp_seq=232 ttl=63 time=0.612 ms
[1546511906.542973] 64 bytes from 192.168.2.22: icmp_seq=233 ttl=63 time=0.613 ms
[1546511906.543964] 64 bytes from 192.168.2.22: icmp_seq=234 ttl=63 time=0.608 ms
[1546511906.544958] 64 bytes from 192.168.2.22: icmp_seq=235 ttl=63 time=0.605 ms
[1546511906.545958] 64 bytes from 192.168.2.22: icmp_seq=236 ttl=63 time=0.604 ms
[1546511906.546959] 64 bytes from 192.168.2.22: icmp_seq=237 ttl=63 time=0.605 ms
[1546511906.547972] 64 bytes from 192.168.2.22: icmp_seq=238 ttl=63 time=0.609 ms
[1546511906.548970] 64 bytes from 192.168.2.22: icmp_seq=239 ttl=63 time=0.609 ms
[1546511906.549980] 64 bytes from 192.168.2.22: icmp_seq=240 ttl=63 time=0.608 ms
[1546511906.550980] 64 bytes from 192.168.2.22: icmp_seq=241 ttl=63 time=0.589 ms
[1546511906.551998] 64 bytes from 192.168.2.22: icmp_seq=242 ttl=63 time=0.613 ms
[1546511906.552967] 64 bytes from 192.168.2.22: icmp_seq=243 ttl=63 time=0.603 ms
[1546511906.553980] 64 bytes from 192.168.2.22: icmp_seq=244 ttl=63 time=0.607 ms
[1546511906.554983] 64 bytes from 192.168.2.22: icmp_seq=245 ttl=63 time=0.609 ms
[1546511906.555984] 64 bytes from 192.168.2.22: icmp_seq=246 ttl=63 time=0.608 ms
[1546511906.556965] 64 bytes from 192.168.2.22: icmp_seq=247 ttl=63 time=0.599 ms
[1546511906.557963] 64 bytes from 192.168.2.22: icmp_seq=248 ttl=63 time=0.596 ms
[1546511906.558977] 64 bytes from 192.168.2.22: icmp_seq=249 ttl=63 time=0.601 ms
[1546511906.559979] 64 bytes from 192.168.2.22: icmp_seq=250 ttl=63 time=0.601 ms
[1546511906.560997] 64 bytes from 192.168.2.22: icmp_seq=251 ttl=63 time=0.622 ms
[1546511906.561972] 64 bytes from 192.168.2.22: icmp_seq=252 ttl=63 time=0.600 ms
[1546511906.562967] 64 bytes from 192.168.2.22: icmp_seq=253 ttl=63 time=0.598 ms
[1546511906.563969] 64 bytes from 192.168.2.22: icmp_seq=254 ttl=63 time=0.598 ms
[1546511906.564977] 64 bytes from 192.168.2.22: icmp_seq=255 ttl=63 time=0.603 ms
[1546511906.565955] 64 bytes from 192.168.2.22: icmp_seq=256 ttl=63 time=0.580 ms
[1546511906.566989] 64 bytes from 192.168.2.22: icmp_seq=257 ttl=63 time=0.619 ms
[1546511906.567968] 64 bytes from 192.168.2.22: icmp_seq=258 ttl=63 time=0.598 ms
[1546511906.568983] 64 bytes from 192.168.2.22: icmp_seq=259 ttl=63 time=0.604 ms
[1546511906.570005] 64 bytes from 192.168.2.22: icmp_seq=260 ttl=63 time=0.628 ms
[1546511906.570981] 64 bytes from 192.168.2.22: icmp_seq=261 ttl=63 time=0.605 ms
[1546511906.571959] 64 bytes from 192.168.2.22: icmp_seq=262 ttl=63 time=0.592 ms
[1546511906.572961] 64 bytes from 192.168.2.22: icmp_seq=263 ttl=63 time=0.592 ms
[1546511906.573965] 64 bytes from 192.168.2.22: icmp_seq=264 ttl=63 time=0.595 ms
[1546511906.574984] 64 bytes from 192.168.2.22: icmp_seq=265 ttl=63 time=0.603 ms
[1546511906.575982] 64 bytes from 192.168.2.22: icmp_seq=266 ttl=63 time=0.601 ms
[1546511906.576983] 64 bytes from 192.168.2.22: icmp_seq=267 ttl=63 time=0.601 ms
[1546511906.577990] 64 bytes from 192.168.2.22: icmp_seq=268 ttl=63 time=0.609 ms
[1546511906.578991] 64 bytes from 192.168.2.22: icmp_seq=269 ttl=63 time=0.607 ms
[1546511906.579978] 64 bytes from 192.168.2.22: icmp_seq=270 ttl=63 time=0.602 ms
[1546511906.580965] 64 bytes from 192.168.2.22: icmp_seq=271 ttl=63 time=0.590 ms
[1546511906.581957] 64 bytes from 192.168.2.22: icmp_seq=272 ttl=63 time=0.586 ms
[1546511906.582973] 64 bytes from 192.168.2.22: icmp_seq=273 ttl=63 time=0.590 ms
[1546511906.583970] 64 bytes from 192.168.2.22: icmp_seq=274 ttl=63 time=0.591 ms
[1546511906.584974] 64 bytes from 192.168.2.22: icmp_seq=275 ttl=63 time=0.589 ms
[1546511906.585978] 64 bytes from 192.168.2.22: icmp_seq=276 ttl=63 time=0.594 ms
[1546511906.587012] 64 bytes from 192.168.2.22: icmp_seq=277 ttl=63 time=0.630 ms
[1546511906.588031] 64 bytes from 192.168.2.22: icmp_seq=278 ttl=63 time=0.639 ms
[1546511906.589008] 64 bytes from 192.168.2.22: icmp_seq=279 ttl=63 time=0.609 ms
[1546511906.589984] 64 bytes from 192.168.2.22: icmp_seq=280 ttl=63 time=0.597 ms
[1546511906.590969] 64 bytes from 192.168.2.22: icmp_seq=281 ttl=63 time=0.587 ms
[1546511906.591961] 64 bytes from 192.168.2.22: icmp_seq=282 ttl=63 time=0.575 ms
[1546511906.592999] 64 bytes from 192.168.2.22: icmp_seq=283 ttl=63 time=0.599 ms
[1546511906.593989] 64 bytes from 192.168.2.22: icmp_seq=284 ttl=63 time=0.599 ms
[1546511906.594972] 64 bytes from 192.168.2.22: icmp_seq=285 ttl=63 time=0.587 ms
[1546511906.596021] 64 bytes from 192.168.2.22: icmp_seq=286 ttl=63 time=0.628 ms
[1546511906.597007] 64 bytes from 192.168.2.22: icmp_seq=287 ttl=63 time=0.614 ms
[1546511906.597978] 64 bytes from 192.168.2.22: icmp_seq=288 ttl=63 time=0.591 ms
[1546511906.598975] 64 bytes from 192.168.2.22: icmp_seq=289 ttl=63 time=0.584 ms
[1546511906.599958] 64 bytes from 192.168.2.22: icmp_seq=290 ttl=63 time=0.572 ms
[1546511906.600993] 64 bytes from 192.168.2.22: icmp_seq=291 ttl=63 time=0.598 ms
[1546511906.601990] 64 bytes from 192.168.2.22: icmp_seq=292 ttl=63 time=0.600 ms
[1546511906.602997] 64 bytes from 192.168.2.22: icmp_seq=293 ttl=63 time=0.570 ms
[1546511906.607268] 64 bytes from 192.168.2.22: icmp_seq=294 ttl=63 time=3.86 ms
[1546511906.607931] 64 bytes from 192.168.2.22: icmp_seq=295 ttl=63 time=0.618 ms
[1546511906.608969] 64 bytes from 192.168.2.22: icmp_seq=296 ttl=63 time=0.643 ms
[1546511906.609963] 64 bytes from 192.168.2.22: icmp_seq=297 ttl=63 time=0.639 ms
[1546511906.610940] 64 bytes from 192.168.2.22: icmp_seq=298 ttl=63 time=0.628 ms
[1546511906.611922] 64 bytes from 192.168.2.22: icmp_seq=299 ttl=63 time=0.607 ms
[1546511906.612967] 64 bytes from 192.168.2.22: icmp_seq=300 ttl=63 time=0.634 ms
[1546511906.613943] 64 bytes from 192.168.2.22: icmp_seq=301 ttl=63 time=0.610 ms
[1546511906.614918] 64 bytes from 192.168.2.22: icmp_seq=302 ttl=63 time=0.596 ms
[1546511906.615901] 64 bytes from 192.168.2.22: icmp_seq=303 ttl=63 time=0.579 ms
[1546511906.616941] 64 bytes from 192.168.2.22: icmp_seq=304 ttl=63 time=0.606 ms
[1546511906.617926] 64 bytes from 192.168.2.22: icmp_seq=305 ttl=63 time=0.600 ms
[1546511906.618935] 64 bytes from 192.168.2.22: icmp_seq=306 ttl=63 time=0.602 ms
[1546511906.619950] 64 bytes from 192.168.2.22: icmp_seq=307 ttl=63 time=0.610 ms
[1546511906.620948] 64 bytes from 192.168.2.22: icmp_seq=308 ttl=63 time=0.607 ms
[1546511906.621948] 64 bytes from 192.168.2.22: icmp_seq=309 ttl=63 time=0.607 ms
[1546511906.622955] 64 bytes from 192.168.2.22: icmp_seq=310 ttl=63 time=0.609 ms
[1546511906.623958] 64 bytes from 192.168.2.22: icmp_seq=311 ttl=63 time=0.615 ms
[1546511906.624946] 64 bytes from 192.168.2.22: icmp_seq=312 ttl=63 time=0.608 ms
[1546511906.625958] 64 bytes from 192.168.2.22: icmp_seq=313 ttl=63 time=0.619 ms
[1546511906.626931] 64 bytes from 192.168.2.22: icmp_seq=314 ttl=63 time=0.592 ms
[1546511906.627943] 64 bytes from 192.168.2.22: icmp_seq=315 ttl=63 time=0.606 ms
[1546511906.628954] 64 bytes from 192.168.2.22: icmp_seq=316 ttl=63 time=0.610 ms
[1546511906.629947] 64 bytes from 192.168.2.22: icmp_seq=317 ttl=63 time=0.608 ms
[1546511906.630959] 64 bytes from 192.168.2.22: icmp_seq=318 ttl=63 time=0.621 ms
[1546511906.631944] 64 bytes from 192.168.2.22: icmp_seq=319 ttl=63 time=0.609 ms
[1546511906.632939] 64 bytes from 192.168.2.22: icmp_seq=320 ttl=63 time=0.604 ms
[1546511906.633977] 64 bytes from 192.168.2.22: icmp_seq=321 ttl=63 time=0.639 ms
[1546511906.634936] 64 bytes from 192.168.2.22: icmp_seq=322 ttl=63 time=0.603 ms
[1546511906.635948] 64 bytes from 192.168.2.22: icmp_seq=323 ttl=63 time=0.609 ms
[1546511906.636949] 64 bytes from 192.168.2.22: icmp_seq=324 ttl=63 time=0.609 ms
[1546511906.637941] 64 bytes from 192.168.2.22: icmp_seq=325 ttl=63 time=0.606 ms
[1546511906.638938] 64 bytes from 192.168.2.22: icmp_seq=326 ttl=63 time=0.603 ms
[1546511906.639940] 64 bytes from 192.168.2.22: icmp_seq=327 ttl=63 time=0.605 ms
[1546511906.640947] 64 bytes from 192.168.2.22: icmp_seq=328 ttl=63 time=0.607 ms
[1546511906.641940] 64 bytes from 192.168.2.22: icmp_seq=329 ttl=63 time=0.605 ms
[1546511906.642941] 64 bytes from 192.168.2.22: icmp_seq=330 ttl=63 time=0.606 ms
[1546511906.643954] 64 bytes from 192.168.2.22: icmp_seq=331 ttl=63 time=0.606 ms
[1546511906.644943] 64 bytes from 192.168.2.22: icmp_seq=332 ttl=63 time=0.607 ms
[1546511906.645947] 64 bytes from 192.168.2.22: icmp_seq=333 ttl=63 time=0.607 ms
[1546511906.646947] 64 bytes from 192.168.2.22: icmp_seq=334 ttl=63 time=0.607 ms
[1546511906.647951] 64 bytes from 192.168.2.22: icmp_seq=335 ttl=63 time=0.610 ms
[1546511906.648945] 64 bytes from 192.168.2.22: icmp_seq=336 ttl=63 time=0.606 ms
[1546511906.649956] 64 bytes from 192.168.2.22: icmp_seq=337 ttl=63 time=0.617 ms
[1546511906.650944] 64 bytes from 192.168.2.22: icmp_seq=338 ttl=63 time=0.605 ms
[1546511906.651961] 64 bytes from 192.168.2.22: icmp_seq=339 ttl=63 time=0.612 ms
[1546511906.652958] 64 bytes from 192.168.2.22: icmp_seq=340 ttl=63 time=0.609 ms
[1546511906.653974] 64 bytes from 192.168.2.22: icmp_seq=341 ttl=63 time=0.625 ms
[1546511906.654955] 64 bytes from 192.168.2.22: icmp_seq=342 ttl=63 time=0.611 ms
[1546511906.655944] 64 bytes from 192.168.2.22: icmp_seq=343 ttl=63 time=0.604 ms
[1546511906.656945] 64 bytes from 192.168.2.22: icmp_seq=344 ttl=63 time=0.603 ms
[1546511906.657946] 64 bytes from 192.168.2.22: icmp_seq=345 ttl=63 time=0.607 ms
[1546511906.658963] 64 bytes from 192.168.2.22: icmp_seq=346 ttl=63 time=0.613 ms
[1546511906.659953] 64 bytes from 192.168.2.22: icmp_seq=347 ttl=63 time=0.607 ms
[1546511906.660971] 64 bytes from 192.168.2.22: icmp_seq=348 ttl=63 time=0.626 ms
[1546511906.661955] 64 bytes from 192.168.2.22: icmp_seq=349 ttl=63 time=0.609 ms
[1546511906.662947] 64 bytes from 192.168.2.22: icmp_seq=350 ttl=63 time=0.606 ms
[1546511906.663923] 64 bytes from 192.168.2.22: icmp_seq=351 ttl=63 time=0.583 ms
[1546511906.664948] 64 bytes from 192.168.2.22: icmp_seq=352 ttl=63 time=0.605 ms
[1546511906.665986] 64 bytes from 192.168.2.22: icmp_seq=353 ttl=63 time=0.638 ms
[1546511906.666955] 64 bytes from 192.168.2.22: icmp_seq=354 ttl=63 time=0.608 ms
[1546511906.667962] 64 bytes from 192.168.2.22: icmp_seq=355 ttl=63 time=0.617 ms
[1546511906.668964] 64 bytes from 192.168.2.22: icmp_seq=356 ttl=63 time=0.610 ms
[1546511906.669966] 64 bytes from 192.168.2.22: icmp_seq=357 ttl=63 time=0.613 ms
[1546511906.670979] 64 bytes from 192.168.2.22: icmp_seq=358 ttl=63 time=0.623 ms
[1546511906.671969] 64 bytes from 192.168.2.22: icmp_seq=359 ttl=63 time=0.614 ms
[1546511906.672969] 64 bytes from 192.168.2.22: icmp_seq=360 ttl=63 time=0.613 ms
[1546511906.673968] 64 bytes from 192.168.2.22: icmp_seq=361 ttl=63 time=0.612 ms
[1546511906.674970] 64 bytes from 192.168.2.22: icmp_seq=362 ttl=63 time=0.613 ms
[1546511906.675974] 64 bytes from 192.168.2.22: icmp_seq=363 ttl=63 time=0.619 ms
[1546511906.676954] 64 bytes from 192.168.2.22: icmp_seq=364 ttl=63 time=0.607 ms
[1546511906.677967] 64 bytes from 192.168.2.22: icmp_seq=365 ttl=63 time=0.612 ms
[1546511906.678967] 64 bytes from 192.168.2.22: icmp_seq=366 ttl=63 time=0.611 ms
[1546511906.679971] 64 bytes from 192.168.2.22: icmp_seq=367 ttl=63 time=0.613 ms
[1546511906.680971] 64 bytes from 192.168.2.22: icmp_seq=368 ttl=63 time=0.613 ms
[1546511906.681959] 64 bytes from 192.168.2.22: icmp_seq=369 ttl=63 time=0.610 ms
[1546511906.682970] 64 bytes from 192.168.2.22: icmp_seq=370 ttl=63 time=0.611 ms
[1546511906.683972] 64 bytes from 192.168.2.22: icmp_seq=371 ttl=63 time=0.611 ms
[1546511906.684971] 64 bytes from 192.168.2.22: icmp_seq=372 ttl=63 time=0.609 ms
[1546511906.685966] 64 bytes from 192.168.2.22: icmp_seq=373 ttl=63 time=0.608 ms
[1546511906.686959] 64 bytes from 192.168.2.22: icmp_seq=374 ttl=63 time=0.605 ms
[1546511906.687937] 64 bytes from 192.168.2.22: icmp_seq=375 ttl=63 time=0.581 ms
[1546511906.688955] 64 bytes from 192.168.2.22: icmp_seq=376 ttl=63 time=0.599 ms
[1546511906.689980] 64 bytes from 192.168.2.22: icmp_seq=377 ttl=63 time=0.611 ms
[1546511906.690954] 64 bytes from 192.168.2.22: icmp_seq=378 ttl=63 time=0.599 ms
[1546511906.691963] 64 bytes from 192.168.2.22: icmp_seq=379 ttl=63 time=0.605 ms
[1546511906.692956] 64 bytes from 192.168.2.22: icmp_seq=380 ttl=63 time=0.601 ms
[1546511906.693954] 64 bytes from 192.168.2.22: icmp_seq=381 ttl=63 time=0.599 ms
[1546511906.694996] 64 bytes from 192.168.2.22: icmp_seq=382 ttl=63 time=0.637 ms
[1546511906.696001] 64 bytes from 192.168.2.22: icmp_seq=383 ttl=63 time=0.639 ms
[1546511906.696963] 64 bytes from 192.168.2.22: icmp_seq=384 ttl=63 time=0.605 ms
[1546511906.697998] 64 bytes from 192.168.2.22: icmp_seq=385 ttl=63 time=0.635 ms
[1546511906.698969] 64 bytes from 192.168.2.22: icmp_seq=386 ttl=63 time=0.605 ms
[1546511906.699962] 64 bytes from 192.168.2.22: icmp_seq=387 ttl=63 time=0.602 ms
[1546511906.700968] 64 bytes from 192.168.2.22: icmp_seq=388 ttl=63 time=0.605 ms
[1546511906.701955] 64 bytes from 192.168.2.22: icmp_seq=389 ttl=63 time=0.600 ms
[1546511906.702970] 64 bytes from 192.168.2.22: icmp_seq=390 ttl=63 time=0.604 ms
[1546511906.707290] 64 bytes from 192.168.2.22: icmp_seq=391 ttl=63 time=3.91 ms
[1546511906.707982] 64 bytes from 192.168.2.22: icmp_seq=392 ttl=63 time=0.639 ms
[1546511906.709002] 64 bytes from 192.168.2.22: icmp_seq=393 ttl=63 time=0.647 ms
[1546511906.710000] 64 bytes from 192.168.2.22: icmp_seq=394 ttl=63 time=0.647 ms
[1546511906.711001] 64 bytes from 192.168.2.22: icmp_seq=395 ttl=63 time=0.646 ms
[1546511906.712001] 64 bytes from 192.168.2.22: icmp_seq=396 ttl=63 time=0.644 ms
[1546511906.712996] 64 bytes from 192.168.2.22: icmp_seq=397 ttl=63 time=0.643 ms
[1546511906.713971] 64 bytes from 192.168.2.22: icmp_seq=398 ttl=63 time=0.613 ms
[1546511906.714963] 64 bytes from 192.168.2.22: icmp_seq=399 ttl=63 time=0.605 ms
[1546511906.715962] 64 bytes from 192.168.2.22: icmp_seq=400 ttl=63 time=0.606 ms
[1546511906.716966] 64 bytes from 192.168.2.22: icmp_seq=401 ttl=63 time=0.611 ms
[1546511906.717962] 64 bytes from 192.168.2.22: icmp_seq=402 ttl=63 time=0.606 ms
[1546511906.718955] 64 bytes from 192.168.2.22: icmp_seq=403 ttl=63 time=0.603 ms
[1546511906.719967] 64 bytes from 192.168.2.22: icmp_seq=404 ttl=63 time=0.606 ms
[1546511906.720960] 64 bytes from 192.168.2.22: icmp_seq=405 ttl=63 time=0.608 ms
[1546511906.721952] 64 bytes from 192.168.2.22: icmp_seq=406 ttl=63 time=0.600 ms
[1546511906.722963] 64 bytes from 192.168.2.22: icmp_seq=407 ttl=63 time=0.605 ms
[1546511906.723957] 64 bytes from 192.168.2.22: icmp_seq=408 ttl=63 time=0.603 ms
[1546511906.724953] 64 bytes from 192.168.2.22: icmp_seq=409 ttl=63 time=0.600 ms
[1546511906.725965] 64 bytes from 192.168.2.22: icmp_seq=410 ttl=63 time=0.604 ms
[1546511906.726955] 64 bytes from 192.168.2.22: icmp_seq=411 ttl=63 time=0.599 ms
[1546511906.727969] 64 bytes from 192.168.2.22: icmp_seq=412 ttl=63 time=0.606 ms
[1546511906.728972] 64 bytes from 192.168.2.22: icmp_seq=413 ttl=63 time=0.606 ms
[1546511906.729995] 64 bytes from 192.168.2.22: icmp_seq=414 ttl=63 time=0.631 ms
[1546511906.730971] 64 bytes from 192.168.2.22: icmp_seq=415 ttl=63 time=0.607 ms
[1546511906.731968] 64 bytes from 192.168.2.22: icmp_seq=416 ttl=63 time=0.605 ms
[1546511906.732962] 64 bytes from 192.168.2.22: icmp_seq=417 ttl=63 time=0.597 ms
[1546511906.733975] 64 bytes from 192.168.2.22: icmp_seq=418 ttl=63 time=0.602 ms
[1546511906.734977] 64 bytes from 192.168.2.22: icmp_seq=419 ttl=63 time=0.603 ms
[1546511906.735977] 64 bytes from 192.168.2.22: icmp_seq=420 ttl=63 time=0.603 ms
[1546511906.736980] 64 bytes from 192.168.2.22: icmp_seq=421 ttl=63 time=0.606 ms
[1546511906.737978] 64 bytes from 192.168.2.22: icmp_seq=422 ttl=63 time=0.603 ms
[1546511906.738981] 64 bytes from 192.168.2.22: icmp_seq=423 ttl=63 time=0.603 ms
[1546511906.739989] 64 bytes from 192.168.2.22: icmp_seq=424 ttl=63 time=0.604 ms
[1546511906.740988] 64 bytes from 192.168.2.22: icmp_seq=425 ttl=63 time=0.604 ms
[1546511906.742000] 64 bytes from 192.168.2.22: icmp_seq=426 ttl=63 time=0.611 ms
[1546511906.742996] 64 bytes from 192.168.2.22: icmp_seq=427 ttl=63 time=0.611 ms
[1546511906.743996] 64 bytes from 192.168.2.22: icmp_seq=428 ttl=63 time=0.618 ms
[1546511906.744988] 64 bytes from 192.168.2.22: icmp_seq=429 ttl=63 time=0.601 ms
[1546511906.745985] 64 bytes from 192.168.2.22: icmp_seq=430 ttl=63 time=0.600 ms
[1546511906.746987] 64 bytes from 192.168.2.22: icmp_seq=431 ttl=63 time=0.601 ms
[1546511906.747967] 64 bytes from 192.168.2.22: icmp_seq=432 ttl=63 time=0.586 ms
[1546511906.748984] 64 bytes from 192.168.2.22: icmp_seq=433 ttl=63 time=0.603 ms
[1546511906.749991] 64 bytes from 192.168.2.22: icmp_seq=434 ttl=63 time=0.603 ms
[1546511906.750988] 64 bytes from 192.168.2.22: icmp_seq=435 ttl=63 time=0.601 ms
[1546511906.751987] 64 bytes from 192.168.2.22: icmp_seq=436 ttl=63 time=0.605 ms
[1546511906.752985] 64 bytes from 192.168.2.22: icmp_seq=437 ttl=63 time=0.603 ms
[1546511906.753998] 64 bytes from 192.168.2.22: icmp_seq=438 ttl=63 time=0.609 ms
[1546511906.754998] 64 bytes from 192.168.2.22: icmp_seq=439 ttl=63 time=0.609 ms
[1546511906.756001] 64 bytes from 192.168.2.22: icmp_seq=440 ttl=63 time=0.610 ms
[1546511906.756991] 64 bytes from 192.168.2.22: icmp_seq=441 ttl=63 time=0.608 ms
[1546511906.757992] 64 bytes from 192.168.2.22: icmp_seq=442 ttl=63 time=0.608 ms
[1546511906.759005] 64 bytes from 192.168.2.22: icmp_seq=443 ttl=63 time=0.611 ms
[1546511906.759998] 64 bytes from 192.168.2.22: icmp_seq=444 ttl=63 time=0.613 ms
[1546511906.761006] 64 bytes from 192.168.2.22: icmp_seq=445 ttl=63 time=0.610 ms
[1546511906.762018] 64 bytes from 192.168.2.22: icmp_seq=446 ttl=63 time=0.624 ms
[1546511906.763014] 64 bytes from 192.168.2.22: icmp_seq=447 ttl=63 time=0.616 ms
[1546511906.764009] 64 bytes from 192.168.2.22: icmp_seq=448 ttl=63 time=0.619 ms
[1546511906.765016] 64 bytes from 192.168.2.22: icmp_seq=449 ttl=63 time=0.616 ms
[1546511906.766001] 64 bytes from 192.168.2.22: icmp_seq=450 ttl=63 time=0.611 ms
[1546511906.767017] 64 bytes from 192.168.2.22: icmp_seq=451 ttl=63 time=0.617 ms
[1546511906.768002] 64 bytes from 192.168.2.22: icmp_seq=452 ttl=63 time=0.610 ms
[1546511906.769020] 64 bytes from 192.168.2.22: icmp_seq=453 ttl=63 time=0.618 ms
[1546511906.770025] 64 bytes from 192.168.2.22: icmp_seq=454 ttl=63 time=0.622 ms
[1546511906.771023] 64 bytes from 192.168.2.22: icmp_seq=455 ttl=63 time=0.619 ms
[1546511906.772023] 64 bytes from 192.168.2.22: icmp_seq=456 ttl=63 time=0.618 ms
[1546511906.773024] 64 bytes from 192.168.2.22: icmp_seq=457 ttl=63 time=0.621 ms
[1546511906.774022] 64 bytes from 192.168.2.22: icmp_seq=458 ttl=63 time=0.618 ms
[1546511906.775021] 64 bytes from 192.168.2.22: icmp_seq=459 ttl=63 time=0.618 ms
[1546511906.776002] 64 bytes from 192.168.2.22: icmp_seq=460 ttl=63 time=0.609 ms
[1546511906.777002] 64 bytes from 192.168.2.22: icmp_seq=461 ttl=63 time=0.609 ms
[1546511906.778008] 64 bytes from 192.168.2.22: icmp_seq=462 ttl=63 time=0.615 ms
[1546511906.779015] 64 bytes from 192.168.2.22: icmp_seq=463 ttl=63 time=0.614 ms
[1546511906.780025] 64 bytes from 192.168.2.22: icmp_seq=464 ttl=63 time=0.618 ms
[1546511906.781023] 64 bytes from 192.168.2.22: icmp_seq=465 ttl=63 time=0.621 ms
[1546511906.782011] 64 bytes from 192.168.2.22: icmp_seq=466 ttl=63 time=0.615 ms
[1546511906.783010] 64 bytes from 192.168.2.22: icmp_seq=467 ttl=63 time=0.612 ms
[1546511906.784012] 64 bytes from 192.168.2.22: icmp_seq=468 ttl=63 time=0.614 ms
[1546511906.785020] 64 bytes from 192.168.2.22: icmp_seq=469 ttl=63 time=0.616 ms
[1546511906.786018] 64 bytes from 192.168.2.22: icmp_seq=470 ttl=63 time=0.614 ms
[1546511906.787023] 64 bytes from 192.168.2.22: icmp_seq=471 ttl=63 time=0.616 ms
[1546511906.788032] 64 bytes from 192.168.2.22: icmp_seq=472 ttl=63 time=0.620 ms
[1546511906.789017] 64 bytes from 192.168.2.22: icmp_seq=473 ttl=63 time=0.615 ms
[1546511906.790028] 64 bytes from 192.168.2.22: icmp_seq=474 ttl=63 time=0.618 ms
[1546511906.791029] 64 bytes from 192.168.2.22: icmp_seq=475 ttl=63 time=0.620 ms
[1546511906.792028] 64 bytes from 192.168.2.22: icmp_seq=476 ttl=63 time=0.604 ms
[1546511906.793026] 64 bytes from 192.168.2.22: icmp_seq=477 ttl=63 time=0.602 ms
[1546511906.794051] 64 bytes from 192.168.2.22: icmp_seq=478 ttl=63 time=0.625 ms
[1546511906.795065] 64 bytes from 192.168.2.22: icmp_seq=479 ttl=63 time=0.642 ms
[1546511906.796067] 64 bytes from 192.168.2.22: icmp_seq=480 ttl=63 time=0.642 ms
[1546511906.797033] 64 bytes from 192.168.2.22: icmp_seq=481 ttl=63 time=0.610 ms
[1546511906.798037] 64 bytes from 192.168.2.22: icmp_seq=482 ttl=63 time=0.602 ms
[1546511906.799042] 64 bytes from 192.168.2.22: icmp_seq=483 ttl=63 time=0.608 ms
[1546511906.800052] 64 bytes from 192.168.2.22: icmp_seq=484 ttl=63 time=0.617 ms
[1546511906.801048] 64 bytes from 192.168.2.22: icmp_seq=485 ttl=63 time=0.610 ms
[1546511906.802046] 64 bytes from 192.168.2.22: icmp_seq=486 ttl=63 time=0.610 ms
[1546511906.803067] 64 bytes from 192.168.2.22: icmp_seq=487 ttl=63 time=0.631 ms
[1546511906.807269] 64 bytes from 192.168.2.22: icmp_seq=488 ttl=63 time=3.83 ms
[1546511906.807957] 64 bytes from 192.168.2.22: icmp_seq=489 ttl=63 time=0.642 ms
[1546511906.808963] 64 bytes from 192.168.2.22: icmp_seq=490 ttl=63 time=0.646 ms
[1546511906.809962] 64 bytes from 192.168.2.22: icmp_seq=491 ttl=63 time=0.646 ms
[1546511906.810974] 64 bytes from 192.168.2.22: icmp_seq=492 ttl=63 time=0.649 ms
[1546511906.811981] 64 bytes from 192.168.2.22: icmp_seq=493 ttl=63 time=0.658 ms
[1546511906.812973] 64 bytes from 192.168.2.22: icmp_seq=494 ttl=63 time=0.647 ms
[1546511906.813940] 64 bytes from 192.168.2.22: icmp_seq=495 ttl=63 time=0.614 ms
[1546511906.814940] 64 bytes from 192.168.2.22: icmp_seq=496 ttl=63 time=0.612 ms
[1546511906.815943] 64 bytes from 192.168.2.22: icmp_seq=497 ttl=63 time=0.614 ms
[1546511906.816944] 64 bytes from 192.168.2.22: icmp_seq=498 ttl=63 time=0.614 ms
[1546511906.817945] 64 bytes from 192.168.2.22: icmp_seq=499 ttl=63 time=0.612 ms
[1546511906.818945] 64 bytes from 192.168.2.22: icmp_seq=500 ttl=63 time=0.612 ms
[1546511906.819948] 64 bytes from 192.168.2.22: icmp_seq=501 ttl=63 time=0.614 ms
[1546511906.820921] 64 bytes from 192.168.2.22: icmp_seq=502 ttl=63 time=0.602 ms
[1546511906.821914] 64 bytes from 192.168.2.22: icmp_seq=503 ttl=63 time=0.593 ms
[1546511906.822895] 64 bytes from 192.168.2.22: icmp_seq=504 ttl=63 time=0.578 ms
[1546511906.823937] 64 bytes from 192.168.2.22: icmp_seq=505 ttl=63 time=0.603 ms
[1546511906.824953] 64 bytes from 192.168.2.22: icmp_seq=506 ttl=63 time=0.619 ms
[1546511906.825941] 64 bytes from 192.168.2.22: icmp_seq=507 ttl=63 time=0.608 ms
[1546511906.826941] 64 bytes from 192.168.2.22: icmp_seq=508 ttl=63 time=0.607 ms
[1546511906.827943] 64 bytes from 192.168.2.22: icmp_seq=509 ttl=63 time=0.609 ms
[1546511906.828933] 64 bytes from 192.168.2.22: icmp_seq=510 ttl=63 time=0.603 ms
[1546511906.829949] 64 bytes from 192.168.2.22: icmp_seq=511 ttl=63 time=0.610 ms
[1546511906.830937] 64 bytes from 192.168.2.22: icmp_seq=512 ttl=63 time=0.603 ms
[1546511906.831946] 64 bytes from 192.168.2.22: icmp_seq=513 ttl=63 time=0.604 ms
[1546511906.832928] 64 bytes from 192.168.2.22: icmp_seq=514 ttl=63 time=0.596 ms
[1546511906.833907] 64 bytes from 192.168.2.22: icmp_seq=515 ttl=63 time=0.579 ms
[1546511906.834928] 64 bytes from 192.168.2.22: icmp_seq=516 ttl=63 time=0.595 ms
[1546511906.835935] 64 bytes from 192.168.2.22: icmp_seq=517 ttl=63 time=0.600 ms
[1546511906.836940] 64 bytes from 192.168.2.22: icmp_seq=518 ttl=63 time=0.598 ms
[1546511906.837912] 64 bytes from 192.168.2.22: icmp_seq=519 ttl=63 time=0.578 ms
[1546511906.838935] 64 bytes from 192.168.2.22: icmp_seq=520 ttl=63 time=0.598 ms
[1546511906.839948] 64 bytes from 192.168.2.22: icmp_seq=521 ttl=63 time=0.601 ms
[1546511906.840966] 64 bytes from 192.168.2.22: icmp_seq=522 ttl=63 time=0.619 ms
[1546511906.841957] 64 bytes from 192.168.2.22: icmp_seq=523 ttl=63 time=0.612 ms
[1546511906.842939] 64 bytes from 192.168.2.22: icmp_seq=524 ttl=63 time=0.601 ms
[1546511906.843937] 64 bytes from 192.168.2.22: icmp_seq=525 ttl=63 time=0.598 ms
[1546511906.844945] 64 bytes from 192.168.2.22: icmp_seq=526 ttl=63 time=0.600 ms
[1546511906.845946] 64 bytes from 192.168.2.22: icmp_seq=527 ttl=63 time=0.601 ms
[1546511906.846945] 64 bytes from 192.168.2.22: icmp_seq=528 ttl=63 time=0.601 ms
[1546511906.847925] 64 bytes from 192.168.2.22: icmp_seq=529 ttl=63 time=0.583 ms
[1546511906.848956] 64 bytes from 192.168.2.22: icmp_seq=530 ttl=63 time=0.606 ms
[1546511906.849947] 64 bytes from 192.168.2.22: icmp_seq=531 ttl=63 time=0.601 ms
[1546511906.850954] 64 bytes from 192.168.2.22: icmp_seq=532 ttl=63 time=0.604 ms
[1546511906.851938] 64 bytes from 192.168.2.22: icmp_seq=533 ttl=63 time=0.596 ms
[1546511906.852955] 64 bytes from 192.168.2.22: icmp_seq=534 ttl=63 time=0.603 ms
[1546511906.853970] 64 bytes from 192.168.2.22: icmp_seq=535 ttl=63 time=0.619 ms
[1546511906.854963] 64 bytes from 192.168.2.22: icmp_seq=536 ttl=63 time=0.609 ms
[1546511906.855964] 64 bytes from 192.168.2.22: icmp_seq=537 ttl=63 time=0.603 ms
[1546511906.856960] 64 bytes from 192.168.2.22: icmp_seq=538 ttl=63 time=0.604 ms
[1546511906.857975] 64 bytes from 192.168.2.22: icmp_seq=539 ttl=63 time=0.629 ms
[1546511906.858959] 64 bytes from 192.168.2.22: icmp_seq=540 ttl=63 time=0.604 ms
[1546511906.859961] 64 bytes from 192.168.2.22: icmp_seq=541 ttl=63 time=0.604 ms
[1546511906.860951] 64 bytes from 192.168.2.22: icmp_seq=542 ttl=63 time=0.603 ms
[1546511906.861959] 64 bytes from 192.168.2.22: icmp_seq=543 ttl=63 time=0.600 ms
[1546511906.862962] 64 bytes from 192.168.2.22: icmp_seq=544 ttl=63 time=0.604 ms
[1546511906.863964] 64 bytes from 192.168.2.22: icmp_seq=545 ttl=63 time=0.602 ms
[1546511906.864970] 64 bytes from 192.168.2.22: icmp_seq=546 ttl=63 time=0.609 ms
[1546511906.865967] 64 bytes from 192.168.2.22: icmp_seq=547 ttl=63 time=0.605 ms
[1546511906.866964] 64 bytes from 192.168.2.22: icmp_seq=548 ttl=63 time=0.605 ms
[1546511906.867954] 64 bytes from 192.168.2.22: icmp_seq=549 ttl=63 time=0.598 ms
[1546511906.868939] 64 bytes from 192.168.2.22: icmp_seq=550 ttl=63 time=0.580 ms
[1546511906.869953] 64 bytes from 192.168.2.22: icmp_seq=551 ttl=63 time=0.597 ms
[1546511906.870969] 64 bytes from 192.168.2.22: icmp_seq=552 ttl=63 time=0.603 ms
[1546511906.871963] 64 bytes from 192.168.2.22: icmp_seq=553 ttl=63 time=0.597 ms
[1546511906.872962] 64 bytes from 192.168.2.22: icmp_seq=554 ttl=63 time=0.595 ms
[1546511906.873966] 64 bytes from 192.168.2.22: icmp_seq=555 ttl=63 time=0.600 ms
[1546511906.874969] 64 bytes from 192.168.2.22: icmp_seq=556 ttl=63 time=0.600 ms
[1546511906.875960] 64 bytes from 192.168.2.22: icmp_seq=557 ttl=63 time=0.595 ms
[1546511906.876956] 64 bytes from 192.168.2.22: icmp_seq=558 ttl=63 time=0.592 ms
[1546511906.877959] 64 bytes from 192.168.2.22: icmp_seq=559 ttl=63 time=0.594 ms
[1546511906.878941] 64 bytes from 192.168.2.22: icmp_seq=560 ttl=63 time=0.578 ms
[1546511906.879981] 64 bytes from 192.168.2.22: icmp_seq=561 ttl=63 time=0.603 ms
[1546511906.880987] 64 bytes from 192.168.2.22: icmp_seq=562 ttl=63 time=0.608 ms
[1546511906.881979] 64 bytes from 192.168.2.22: icmp_seq=563 ttl=63 time=0.601 ms
[1546511906.882986] 64 bytes from 192.168.2.22: icmp_seq=564 ttl=63 time=0.607 ms
[1546511906.884001] 64 bytes from 192.168.2.22: icmp_seq=565 ttl=63 time=0.609 ms
[1546511906.885002] 64 bytes from 192.168.2.22: icmp_seq=566 ttl=63 time=0.610 ms
[1546511906.886005] 64 bytes from 192.168.2.22: icmp_seq=567 ttl=63 time=0.612 ms
[1546511906.887018] 64 bytes from 192.168.2.22: icmp_seq=568 ttl=63 time=0.622 ms
[1546511906.888027] 64 bytes from 192.168.2.22: icmp_seq=569 ttl=63 time=0.620 ms
[1546511906.888979] 64 bytes from 192.168.2.22: icmp_seq=570 ttl=63 time=0.586 ms
[1546511906.890017] 64 bytes from 192.168.2.22: icmp_seq=571 ttl=63 time=0.621 ms
[1546511906.890991] 64 bytes from 192.168.2.22: icmp_seq=572 ttl=63 time=0.605 ms
[1546511906.892005] 64 bytes from 192.168.2.22: icmp_seq=573 ttl=63 time=0.610 ms
[1546511906.893015] 64 bytes from 192.168.2.22: icmp_seq=574 ttl=63 time=0.621 ms
[1546511906.894017] 64 bytes from 192.168.2.22: icmp_seq=575 ttl=63 time=0.622 ms
[1546511906.895027] 64 bytes from 192.168.2.22: icmp_seq=576 ttl=63 time=0.641 ms
[1546511906.896052] 64 bytes from 192.168.2.22: icmp_seq=577 ttl=63 time=0.655 ms
[1546511906.897009] 64 bytes from 192.168.2.22: icmp_seq=578 ttl=63 time=0.621 ms
[1546511906.898001] 64 bytes from 192.168.2.22: icmp_seq=579 ttl=63 time=0.613 ms
[1546511906.898984] 64 bytes from 192.168.2.22: icmp_seq=580 ttl=63 time=0.595 ms
[1546511906.900002] 64 bytes from 192.168.2.22: icmp_seq=581 ttl=63 time=0.614 ms
[1546511906.901027] 64 bytes from 192.168.2.22: icmp_seq=582 ttl=63 time=0.638 ms
[1546511906.902000] 64 bytes from 192.168.2.22: icmp_seq=583 ttl=63 time=0.610 ms
[1546511906.903012] 64 bytes from 192.168.2.22: icmp_seq=584 ttl=63 time=0.616 ms
[1546511906.907257] 64 bytes from 192.168.2.22: icmp_seq=585 ttl=63 time=3.86 ms
[1546511906.907942] 64 bytes from 192.168.2.22: icmp_seq=586 ttl=63 time=0.636 ms
[1546511906.908946] 64 bytes from 192.168.2.22: icmp_seq=587 ttl=63 time=0.640 ms
[1546511906.909954] 64 bytes from 192.168.2.22: icmp_seq=588 ttl=63 time=0.644 ms
[1546511906.910951] 64 bytes from 192.168.2.22: icmp_seq=589 ttl=63 time=0.640 ms
[1546511906.911962] 64 bytes from 192.168.2.22: icmp_seq=590 ttl=63 time=0.651 ms
[1546511906.912951] 64 bytes from 192.168.2.22: icmp_seq=591 ttl=63 time=0.641 ms
[1546511906.913920] 64 bytes from 192.168.2.22: icmp_seq=592 ttl=63 time=0.608 ms
[1546511906.914926] 64 bytes from 192.168.2.22: icmp_seq=593 ttl=63 time=0.613 ms
[1546511906.915927] 64 bytes from 192.168.2.22: icmp_seq=594 ttl=63 time=0.615 ms
[1546511906.916935] 64 bytes from 192.168.2.22: icmp_seq=595 ttl=63 time=0.623 ms
[1546511906.917939] 64 bytes from 192.168.2.22: icmp_seq=596 ttl=63 time=0.615 ms
[1546511906.918940] 64 bytes from 192.168.2.22: icmp_seq=597 ttl=63 time=0.616 ms
[1546511906.919951] 64 bytes from 192.168.2.22: icmp_seq=598 ttl=63 time=0.618 ms
[1546511906.920928] 64 bytes from 192.168.2.22: icmp_seq=599 ttl=63 time=0.611 ms
[1546511906.921944] 64 bytes from 192.168.2.22: icmp_seq=600 ttl=63 time=0.616 ms
[1546511906.922940] 64 bytes from 192.168.2.22: icmp_seq=601 ttl=63 time=0.613 ms
[1546511906.923944] 64 bytes from 192.168.2.22: icmp_seq=602 ttl=63 time=0.615 ms
[1546511906.924944] 64 bytes from 192.168.2.22: icmp_seq=603 ttl=63 time=0.616 ms
[1546511906.925946] 64 bytes from 192.168.2.22: icmp_seq=604 ttl=63 time=0.618 ms
[1546511906.926941] 64 bytes from 192.168.2.22: icmp_seq=605 ttl=63 time=0.612 ms
[1546511906.927933] 64 bytes from 192.168.2.22: icmp_seq=606 ttl=63 time=0.609 ms
[1546511906.928929] 64 bytes from 192.168.2.22: icmp_seq=607 ttl=63 time=0.607 ms
[1546511906.929931] 64 bytes from 192.168.2.22: icmp_seq=608 ttl=63 time=0.609 ms
[1546511906.930930] 64 bytes from 192.168.2.22: icmp_seq=609 ttl=63 time=0.608 ms
[1546511906.931946] 64 bytes from 192.168.2.22: icmp_seq=610 ttl=63 time=0.613 ms
[1546511906.932954] 64 bytes from 192.168.2.22: icmp_seq=611 ttl=63 time=0.622 ms
[1546511906.933944] 64 bytes from 192.168.2.22: icmp_seq=612 ttl=63 time=0.612 ms
[1546511906.934929] 64 bytes from 192.168.2.22: icmp_seq=613 ttl=63 time=0.605 ms
[1546511906.935946] 64 bytes from 192.168.2.22: icmp_seq=614 ttl=63 time=0.611 ms
[1546511906.936959] 64 bytes from 192.168.2.22: icmp_seq=615 ttl=63 time=0.625 ms
[1546511906.937970] 64 bytes from 192.168.2.22: icmp_seq=616 ttl=63 time=0.636 ms
[1546511906.938929] 64 bytes from 192.168.2.22: icmp_seq=617 ttl=63 time=0.606 ms
[1546511906.939929] 64 bytes from 192.168.2.22: icmp_seq=618 ttl=63 time=0.605 ms
[1546511906.940943] 64 bytes from 192.168.2.22: icmp_seq=619 ttl=63 time=0.611 ms
[1546511906.941944] 64 bytes from 192.168.2.22: icmp_seq=620 ttl=63 time=0.610 ms
[1546511906.942944] 64 bytes from 192.168.2.22: icmp_seq=621 ttl=63 time=0.609 ms
[1546511906.943953] 64 bytes from 192.168.2.22: icmp_seq=622 ttl=63 time=0.617 ms
[1546511906.944927] 64 bytes from 192.168.2.22: icmp_seq=623 ttl=63 time=0.601 ms
[1546511906.945928] 64 bytes from 192.168.2.22: icmp_seq=624 ttl=63 time=0.602 ms
[1546511906.946929] 64 bytes from 192.168.2.22: icmp_seq=625 ttl=63 time=0.602 ms

If you hover your mouse over the upper right corner of an existing post where the “edit” (looks like a pencil) icon is, then a “paper clip” icon will show up. This is how you attach to an existing post (you can’t attach at the same time you create a post).

Btw, I think the ethernet controller on the TX1 goes through a USB controller, and there are some peripherals which scan at a 100Hz polling rate (though I think most of the modern world has moved to 1000Hz…and optionally 250Hz). It’s entirely a guess, but it is possible that you are seeing USB polling.

Ok, finally found out the delay in your log. Something like below one, right?

[1546511906.607268] 64 bytes from 192.168.2.22: icmp_seq=294 ttl=63 time=<b>3.86 ms</b>

I am still not convinced by the test that it is a bug or issue on cpu. As I said, maybe it is indeed an issue but using “ping” from another host may not be a correct way.

There are still some possible cause for such issue. For example, power saving feature on CPU.

@linuxdev On my custom build where I also see the issue. I have seen the issue with bluetooth, usb and wifi disabled so I don’t think it is usb.

@WayneWWW I was also wondering whether it was related to the low power cores / high power cores, as the 3ms gap appears when I increase the cpu frequency OR when I max gpu frequency and turn off railgate. Could there be a power-management / frequency scaling issue that could cause this?

@WayneWWW. Have you observed a similar result?

What linuxdev wants to say is the ethernet adapter on TX1 is based on USB driver.

There are some introduction about power management in L4T document. You could read it first.

https://developer.nvidia.com/embedded/dlc/l4t-documentation-28-2-ga

I don’t think we should call it a bug at this moment. I wonder if you could use some tools like nvidia system profiler to check per core cpu utilization.

Having read power management for TX1 devices, I have tried removing CPUIDLE from kernel and device tree file, this did not fix the issue.

The issue appears to occur if:

  1. cpu frequency is above 1122000.

OR
2. if GPU railgate = off AND gpu frequency is above 460800000.

I will continue investigating…

akmal.ali,

As I said, there are still other processes running in background if you are using L4T BSP directly from jetpack. How could you prevent such case influences your observation?

More importantly, does this 3ms delay affect/block any of your usecase?

You might be able to partially work around via increasing the priority of your process…try renicing it to “-2” (if you need to go more negative it isn’t going to help, but it might cause problems).

@linuxdev I tried renicing the process, this didn’t solve the issue.

@WayneWWW The usecase is a real-time application with low latency, so having a 3ms delay does affect our usecase. It seems that the 3ms delay also delays cuda calls despite them running on threads on other cores. I see the issue on both the L4T BSP and on our custom build which runs less unnecessary processes. The question is, that if it is a process, do you have any idea which one it could be.

The issue presents itself when gpu OR cpu clocks are increased and is affected by whether gpu railgate is set to 0 or 1. This leads me to think that it is related to the clock / power system. Unfortunately have not found the cause yet.

If this is related to some kind of cpu scheduling issue, we may need help from nv internal team.

To prove this is a real issue, you should give out a better usecase and prove that your usecase really gets influenced.
For example, elaborate what kind of real-time application you are using and better sharing your code with us.

Also, you should tell us why your usecase needs to run on single core cpu. It would be far better than using the result of “ping”.