Low Upload speeds on my Jetson Nano

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello,
I am using Nvidia Jetson Nano and an ECON camera (https://www.e-consystems.com/nvidia-cameras/jetson-nano-cameras/3mp-mipi-camera.asp) to stream to my Cloud-hosted Nginx RTMP Server. I use the Gstreamer Command line to trigger the camera and stream it to an RTMP application on my Nginx Server. I usually get glitchy video streams sometimes and I have realized that the Internet Upload speeds on my Jetson Nano are not getting better than 3Mbps. But I need at least 10 Mbps of Upload speed to get a good quality of the stream. However, the Download speeds are at least 30 Mbps most of the time and would go up to 80 Mbps on a high-speed internet network, but the Upload speed doesn’t exceed 3-4 Mpbs even then. Please suggest some solutions so I can fix it.

I also have tried to test the speeds on my laptop which is on the same network as my Jetson is, and the Laptop would get way better Upload speed than my Jetson Nano. I have tested with different Networks and the result doesn’t vary much in the case of Jetson Nano Upload speed. I even tested my Jetson Nano with an Intel WiFi Module and it didn’t make any change with the 3Mbps Upload speed.

The Network setup I have is:
A router --[connected to]–> PoE Switch, with Ethernet Cable.

PoE Switch --[connected to]–> PoE Splitter (Ethernet+Micro USB), with CAT6 Cable.

PoE Splitter’s micro USB Cable powers up the Jetson and PoE Splitter’s Ethernet Cable provides internet to Jetson.

Please help me with this,

Thank you very much in advance,
Vivek

2

On the particular Nano, after you see issues, what is the output of (you might need to install the ethtool package):

ifconfig
route
ethtool eth0

I am assuming this is eth0 wired to the camera. Adjust as needed.

3

Hello, Thank you for your response.
The output of the commands are as below:

root@nvidia-desktop:/home/nvidia# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.101  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::5dd:4a2a:8a3c:b361  prefixlen 64  scopeid 0x20<link>
        ether 00:04:4b:e6:23:96  txqueuelen 1000  (Ethernet)
        RX packets 18090632  bytes 1429461684 (1.4 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 40123527  bytes 63375172573 (63.3 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 148  base 0xe000  

l4tbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.55.1  netmask 255.255.255.0  broadcast 192.168.55.255
        inet6 fe80::1  prefixlen 128  scopeid 0x20<link>
        inet6 fe80::187d:9eff:fe60:5407  prefixlen 64  scopeid 0x20<link>
        ether ba:a1:3c:51:67:89  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 741 (741.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 884319250  bytes 107781336280 (107.7 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 884319250  bytes 107781336280 (107.7 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

rndis0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether ba:a1:3c:51:67:89  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether ba:a1:3c:51:67:8b  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@nvidia-desktop:/home/nvidia# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    100    0        0 eth0
default         _gateway        0.0.0.0         UG    32766  0        0 l4tbr0
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 l4tbr0
192.168.1.0     0.0.0.0         255.255.255.0   U     100    0        0 eth0
192.168.55.0    0.0.0.0         255.255.255.0   U     0      0        0 l4tbr0

root@nvidia-desktop:/home/nvidia# ethtool eth0
Settings for eth0:
	Supported ports: [ TP ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Supported pause frame use: No
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Full 
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: 100Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: Unknown
	Supports Wake-on: pumbg
	Wake-on: g
	Current message level: 0x00000033 (51)
			       drv probe ifdown ifup
	Link detected: yes

Yes, the camera is wired to the eth0 Interface (From a PoE Splitter). Please let me know if I can provide more information on this, so I can resolve the issue.

Thank you,

4

Everything is working correctly, but for whatever reason eth0 was only using 100Mbit/s speed (gigabit is 10 times that). Auto-negotiation is on, but even so, full speed was not enabled. Is this using a 100Mbit/s network instead of gigabit? I’m not familiar with your PoE hardware, but I’m less interested in how power is delivered, and more interested in the network capabilities of the hardware (perhaps one component is not capable of gigabit and has somehow forced other components to revert back to 100Mbit/s as well).

Incidentally, if you use “nm-connection-editor” (if you don’t have this, then “sudo apt-get install network-manager-gnome”), and go to the “Ethernet” tab of the wired connection, then you will find “Link negotiation”. This is normally “automatic”, but as a test you could set it to “manual” and force speed 100Mbit/s (and full duplex). If this still works, then there may have been an issue with auto negotiation; if the connection then fails, it may be that some component in the chain of networking does not support gigabit.

5

Hi,

The PoE Switch I use:
https://www.kjell.com/se/produkter/dator-natverk/natverk/tradburet-natverk/switchar/d-link-dgs1005-poe-gigabitswitch-5-portar-p62077?gclid=Cj0KCQiAn8nuBRCzARIsAJcdIfPc1kw2FVZHRayAbKcNcVhDKECaurvgTsaWOcClejZCJzw2aY_w4_UaAovUEALw_wcB&gclsrc=aw.ds

When I connect the camera to the Switch, it gets the yellow light for the Link/ACT. From the Led indicator yellow light means 10/100M and the green light means 1000M.

The Poe switch I use seems to support 1000M also from its led indication manual.

The PoE Splitter I use:
https://se.eetgroup.com/i/MC-POESPLITTER-MicroConnect-PoE-Splitter-5V-2.4A

Could this PoE Splitter possibly affect the Upload Speeds on the Jetson? (Download speeds are fast though)

Do you see any limitations in the network capabilities in this attached hardware? I also tried to use the Jetson with other PoE switches and also other Networks, even then It couldn’t get higher than 4Mbps Upload Speeds.

I have changed the link negotiation to Manual as you suggested and set the speed to 1Gb/s and Full duplex. After then I made a speed-test and it still shows the same Upload Speeds around 4Mbps.

But please help me understand even if the negotiation is limited to 100Mbps, then why is it having only 3-4Mbps why couldn’t it go to at least 10Mbps which is less than the 100Mpbs link it has.

If it could be the case that one of the Networking chains is limiting support for gigabit, then I have tested it with the WiFi module for the Jetson and it isn’t making a difference with the WiFi also.

Let’s think of the Internet connection to the Router or the WiFi Itself has problems to provide good speeds. In that case, my laptop which is connected to the same LAN network/Wifi is getting at least 20Mbps.

So I am confused about where is the problem laying. Please let me know your suggestions,
Thank you very much in advance,

6

I don’t have much experience with PoE switches, but the power delivery method shouldn’t matter. On the other hand, even if your Nano is capable of 1000Mbit/s, then traffic will be unable to exceed the 10/100 speed of the camera (traffic could be gigabit to the switch, and then forced back to 100 or 10Mbit/s when talking to the camera).

Better switches tend to not slow down different devices which are unrelated to each other in the network traffic, but there are some less expensive switches which might slow down several devices if any one of them runs at a lower speed. In terms of PoE I assume even a low end switch would cost more. I don’t know if this is the case for you, but consider that 10Mbit/s is actually fast if the camera itself is running 10Mbit/s, or if it is low enough resolution/frame rate that it cannot produce higher bandwidth.

How is the splitter used? If it is only for providing power, then this would probably not have any effect. If the actual USB network side is used, then this could be a problem as it is a fairly slow adapter compared to gigabit. Additionally, it would be hard to say if this adapter supports auto negotiation.

7

I notice the low upload speeds even when the camera is not being used.

I am using the camera with 1920x1080P resolution & 60 FPS, and the camera with 3 Mbps just because the nano is having only 3-4Mbps.

The PoE splitter is used to provide both power and ethernet connection for the Jetson nano.
what tests can I perform to resolve this issue?

8

What we know at this point is that the ethernet has no dropped packets, no overruns from too much data, no errors, no drops, nor any other issue. Throughput is the only issue, but we don’t know if it is because of data source/pipe/sink.

What is the IP address of the camera itself? If the camera supports ICMP protocol, then we can ping it and use traceroute on it. Adjust for your actual address, but I’ll pretend the address is “192.168.1.123”:

ping 192.168.1.123
traceroute 192.168.1.123

(if you don’t have “traceroute”, then “sudo apt-get install traceroute”)

The goal of this is to see the actual list of hops taken to get to the camera. In theory the best route would go straight to the camera without any unusual detours.

Another question is the nature of the slowest link in the chain: Source, sink, or data pipe. You can judge what you expect from the camera throughput, e.g., based on past experience with other computers, and specifications. However, whatever is consuming the data might be an issue, e.g., if we were sending data over a network capable of 10Gbit/s, and going to an SD card which can only handle 2Mbit/s, then we might confuse the limit of the SD card with the network limit. We are examining the network itself, and so we have to start with the lower limit of data source and sync. If source and sync are both above the current throughput, then we know for sure there is a network issue. Several applications may all appear to be slow throughput, but as an example, if they all use the same slow SD card, then all will be slow.

In terms of the actual network throughput, sometimes it matters if you are sending a lot of tiny packets, versus large/jumbo packets. The network may need some packet size tuning if the data is an unusually high rate of small packets, versus large data chunks all at once.

9

Hello there,

i have the exact same problem. I´m using an SanDisk Extreme microSDHC 32GB SD Card which is capable of 100/60 MB/sek speed.

However when i run the speedtest-cli ubuntu app, i get full download at 250 mbit/sec but only 3,85 mbit/sec upload speed. My DSL has 250/50. The nano is connected to a 1000 mbit/sec link and the speed is negotiated fine.

I have no problem to reach 250/50 mbit/sec with some other raspberry PI 4 and the SAME sd card.

So its clearly not my network, nor the cable, nor the sd card.

I ordered a 2nd jetson nano on amazon and it has the exact same problem.

With 3,8 mbit/sec upload speed on a 1000 mbit/sec LAN connector this thing is a total fail.

I guess its not hardware but firmware/os/software related.

Did anyone find a solution for that problem?

PS: I could reproduce the same issue on ALL connections on the nano. The wired LAN port, the INTEL Wifi m2 Module, as well as an USB LTE Stick are all never exceeding an upload speed of ~ 3,8 mbit/sec.

My OS and kernel is: Ubuntu 18.04.5 LTS (GNU/Linux 4.9.201-tegra aarch64) with

Its a plain fresh install. Tried it 6 times, with 3 different brand new SD cards, two different brand new jetson nanos and as said above even with Wifi and an LTE Stick that can do 150 mbit/sec / 50 mbit/sec.

All combinations never exceeded 3,8 mbit/sec upload speed.

KR
Daniel

UPDATE 19.02.21:

I found the reason why this is happening.

The problem lies within “speedtest-cli” under Ubuntu 18.04.

It seems that Ookla (Speedtest . net) has a buggy release for the speedtest-cli program under Ubuntu 18.04 which is being used by the Jetson Nano.

You can read about it here: networking - Slow upload speed on machine running Ubuntu Server 18.04 - Ask Ubuntu

The solution is, if you want to speedtest your Jetson Nano, that you use this Python Speedtest . net Script instead of speedtest-cli:

https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py

You can wget it and chmod+x it and then run it and you will see perfect speeds (hopefully ^^)

You can read more about the script and how to install it here too: How to run a network speed test from a headless Linux server | TechRepublic

KR
Daniel

3 Likes