Jetson TX2 unable to access GigE camera over PCIe ethernet expansion board

Hi good people!

(Noob altert)
I have for a week now be struggling to access my BlackFly GigE camera that I am connecting to the Jetson via a PCIe Expansion board (PCIe-PoE354at).

  1. Power seems to be going to both the PCIe board and the camera (at least the camera LED is blinking happily)

  2. Jetson TX2 seems to recognize an IP addresse for the PCIe board - but not the connected camera. Jetson can identify the different ports in the expansion board - and when I open for example “GNOME Network Tools” I can see tiny amounts of data being transfer for the port where the camera is connected (ethernet port number 4). If i type “ifconfig” in the terminal I can also see it is receiving data packages for that specifical port (eth4), but still no ip/inet/netmask/broadcast is specified. The best I can get is hardware addresse.

I have then tried to assign an IP addresse to the MAC addresse by using arp:

But it returns: arp: invalid hardware address even though I can definitely see the particular MAC addresse when doing ifconfig.

FLIR who made the camera suggests I use their SDK “Spinview”, but is only displaying the expansion board IP.

Any idea of which I might be doing wrong or any alternative why to find a GigE camera IP? I I’m running Ubuntu 18.04, plenty of power should be supplied to the camera.

Much Love

I won’t be able to answer specifics on cameras, but to get started here are some networking questions to post details on…

What do you see for that device under “ifconfig”? The name will probably be different on another computer, but if you have a PC you can connect this to and post the “ifconfig” output on that as well (to compare working and non-working), then this might help.

Also, post the output of the command “route” on the Jetson.

Note that you would rarely run into a circumstance where you would use the arp software to set a MAC on the actual device, although you would use MAC for a lot of other uses for outside software wanting to find or use the device. The “route” command might provide more information about why using the IP address is not working.

In some cases the hardware might also need firmware added. Without firmware some devices (not necessarily your device) won’t function (this is just a small number of devices out in the wild, but networking devices tend to do this more, at least when they are WiFi devices).

You mentioned a PCIe expansion card. What do you see from “lspci”? Each device found has a slot ID. That ID looks something like “01:00.0”, but will differ depending on device. I’m going to use “01:00.0” as an example, and the following shows a verbose reading of that device (you might want to attach this too):
sudo lspci -s 01:00.0 -vvv

You can attach files here, or if you just want to paste, add two lines with three single “back quotes”…the unshifted tilde key next to the tab key on most keyboards. Example:
```
…code…
```
(I cheated and escaped ` so it would actually print instead of causing formatting).

Hi Linuxdev! Thank you so much for taking the time.

When I do ifconfig here is what I get. As you can see eth4 has received few pieces of data from the camera :

eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 00:04:4b:f7:84:5e  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 41

eth1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 78:d0:04:28:bd:5d  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x40100000-4011ffff  

eth2: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 78:d0:04:28:bd:5e  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x40120000-4013ffff  

eth3: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 78:d0:04:28:bd:5f  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x40140000-4015ffff  

eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 78:d0:04:28:bd:60  txqueuelen 1000  (Ethernet)
        RX packets 119  bytes 39910 (39.9 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 223  bytes 39707 (39.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x40160000-4017ffff  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 1884  bytes 122074 (122.0 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1884  bytes 122074 (122.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

rndis0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 86:d4:b2:5d:47:6d  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 86:d4:b2:5d:47:6f  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.20.10.5  netmask 255.255.255.240  broadcast 172.20.10.15
        inet6 fe80::a2b7:84dc:e79d:88c0  prefixlen 64  scopeid 0x20<link>
        ether 00:04:4b:f7:84:5c  txqueuelen 1000  (Ethernet)
        RX packets 652  bytes 210419 (210.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 766  bytes 145160 (145.1 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0




When I do route I am getting:

novo_jetson@novo-jetson-desktop:~/Desktop$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    600    0        0 wlan0
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 wlan0
172.20.10.0     0.0.0.0         255.255.255.240 U     600    0        0 wlan0

When I type lspci I am getting what looks like the port on the expansion board:

 novo_jetson@novo-jetson-desktop:~/Desktop$ lspci
00:01.0 PCI bridge: NVIDIA Corporation Device 10e5 (rev a1)
01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

If I check out one of ports (where I believe the camera is attached):

sudo lspci -s 01:00.3 -vvv
01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 128 bytes
	Interrupt: pin D routed to IRQ 381
	Region 0: Memory at 40160000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at 1060 [disabled] [size=32]
	Region 3: Memory at 4018c000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <4us, L1 <32us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM L0s L1 Enabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout+ NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [140 v1] Device Serial Number 78-d0-04-ff-ff-28-bd-5d
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 03
		VF offset: 384, stride: 4, Device ID: 1520
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000480c0000 (64-bit, prefetchable)
		Region 3: Memory at 00000000480e0000 (64-bit, prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Capabilities: [1a0 v1] Transaction Processing Hints
		Device specific mode supported
		Steering table in TPH capability structure
	Capabilities: [1d0 v1] Access Control Services
		ACSCap:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
	Kernel driver in use: igb

Thanks for those sleek commands :-) -> ‘’’ ‘’’ looks really neat indeed.

I managed to find some sub-pages from the manufacture (FLIR), stating stating that their SDKs did not support my GigE camera on ARM processors :-( Does this mean I am completely screwed?

I think I’ve just made a minor break through.

By following this guide, I manage to assign IP to my camera/ethernet connection:

eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet 182.182.1.3  netmask 255.255.255.0  broadcast 182.182.1.255
        inet6 fe80::8128:1c0c:a216:cd55  prefixlen 64  scopeid 0x20<link>
        ether 78:d0:04:28:bd:5f  txqueuelen 1000  (Ethernet)
        RX packets 11  bytes 660 (660.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 370  bytes 59862 (59.8 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0x40140000-4015ffff  

It also appears when I do route:

novo_jetson@novo-jetson-desktop:~/Desktop$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    600    0        0 wlan0
default         _gateway        0.0.0.0         UG    20100  0        0 eth3
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 wlan0
172.20.10.0     0.0.0.0         255.255.255.240 U     600    0        0 wlan0
182.182.1.0     0.0.0.0         255.255.255.0   U     100    0        0 eth3

However neither broadcast or inet IP seems to work opening a cv2 video capture in python:

import cv2, sys

cap = cv2.VideoCapture('rtsp://182.182.1.255/1')
if not cap.isOpened():
    sys.exit('Failed to open test camera!')
else:
    _, img = cap.read() # grab the next image frame from camera          
    key = cv2.waitKey(10)    
    cap.release()

cv2.imwrite("img.jpg",img)

Returns

VIDEOIO ERROR: V4L: device rtsp://182.182.1.255: Unable to query number of channels
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/novo_jetson/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 890
VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:

/home/novo_jetson/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:890: error: (-2) GStreamer: unable to start pipeline
in function cvCaptureFromCAM_GStreamer

Failed to open test camera!

FYI, if you see packets on an interface which has a camera on it, then this is proper function. It does not mean that those bytes made it to any outside application, and this is the realm of the “route” as to whether that data can be reached.

So far as that URL goes, it is for RedHat, and for the most part it will be the same as Ubuntu. You might at times find minor differences, especially in file layout, so keep in mind the logic will be good, and often individual commands will be correct, but you could find differences at times.

When posting “ifconfig” output be sure to say which system it is from…Jetson versus PC. On some of the Jetson output I can see the Jetson name in the command line, but the ifconfig content is not so obvious.

On your last post, where you have eth3, is this on the Jetson for ifconfig output? I suspect it is since the jetson-desktop “route” has eth3.

This looks promising since the 182.182.1.0 subnet is separate from any other subnet. Even with a metric of 100 there will be no issue since metric only matters when two interfaces can both serve the route (previously there were two interfaces on the same route, now it is one interface on one route). Ping should work if the camera supports ICMP.

I couldn’t tell you about correct testing commands for camera testing. Does ping work to the camera’s IP address? Is the camera’s IP address something you can verify to be somewhere between 182.182.1.1 and 182.182.1.254? If ping works, then you know you can move on to camera commands since you’d know the camera is actually receiving the traffic.

The lspci shows the PCIe is working well, at full speed, and with no errors. This makes it possible for the driver to function. Seeing ifconfig without errors implies networking is also working correctly, although it doesn’t say that configuration is correct for what you want to do (it also does not say there is anything incorrect).

About this:

…this could be a problem. Not necessarily a show stopper (though it might be). If the camera functions normally without anything special from the manufacturer, and if the software not running on arm64 is just end user software which other software can replace, then you can use other software. I am totally making this up since I do not know cameras, but for example, if the camera uses RTSP and the manufacturer software just reads a standard RTSP, then perhaps gstreamer or other applications would do the job. Conversely, if that software is for setting up the camera for function in some custom way that there is no generic software for, then there is nothing more you can do without the manufacturer.

What software does this camera normally have installed when working from a desktop PC with Linux? Knowing what that software does might offer some valuable clues.

Hi again Linuxdev, Thank you so much for your input!

Yes indeed I was posting ifconfig for the Jetson. Yes I should up my practice for consistent posting, thanks for remind me :-)

With regards to my route output, this now looks like:

novo_jetson@novo-jetson-desktop:~/Desktop$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    600    0        0 wlan0
default         _gateway        0.0.0.0         UG    20100  0        0 eth3
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 wlan0
172.20.10.0     0.0.0.0         255.255.255.240 U     600    0        0 wlan0
182.182.1.0     0.0.0.0         255.255.255.0   U     100    0        0 eth3

I’m able to ping the assigned inet only (182.182.1.3) :-)!

I tried using nmap in order to scan which addresses were in use and I got that same addresse in my result:

novo_jetson@novo-jetson-desktop:~/Desktop$ nmap -sP 182.182.1.0/24

Starting Nmap 7.60 ( https://nmap.org ) at 2020-05-22 12:04 CEST
Nmap scan report for novo-jetson-desktop (182.182.1.3)
Host is up (0.00051s latency).
Nmap done: 256 IP addresses (1 host up) scanned in 3.58 seconds

With regards to lspci i’m happy to know everything seems to be working fine. xD

Yet when I am trying to open/capture it’s failing:

import cv2, sys

cap = cv2.VideoCapture('rtsp://182.182.1.3/')

if not cap.isOpened():
    sys.exit('Failed to open test camera!')
else:
    _, img = cap.read() # grab the next image frame from camera
           
    key = cv2.waitKey(10)
    cv2.imwrite("img.jpg",img)
    
    cap.release()

RETURNS

novo_jetson@novo-jetson-desktop:~/Desktop/MainScript$ python Open_IP_Cam_test.py 
[tcp @ 0x230af570] Connection to tcp://182.182.1.3:554?timeout=0 failed: Connection refused
VIDEOIO ERROR: V4L: device rtsp://182.182.1.3/: Unable to query number of channels
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/novo_jetson/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 890
VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:

/home/novo_jetson/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:890: error: (-2) GStreamer: unable to start pipeline
 in function cvCaptureFromCAM_GStreamer

Failed to open test camera!

Should I pay attention to the fact that it looks like it it is connecting as TCP?

After some more hours of research it has come to my attention that the related camera models doesn’t not support rtsp, yet I havn’t been able to confirm this for my particular camera (BF-PGE-31S4M-C) as the documentation is extremely poor.

Could it be that i need to specificy some ports?

I’ve contact the manufacturer for some more clear documentation but havn’t gotten an answer so far.

:-(

Okay, I have just gotten confirmation that it doesn’t support rtsp, but I was not provided concrete alternative. I was only suggested I could look into the pyspin module which they made for a range of their their products.

I was also looking into udp, but it looks like that was only relevant if I was streaming from remote to local (maybe I am mistaken?)

In other news the IP for camera suddenly showed up in their SDK (which allegedly should not work), and it looks like I might only have changed the static IP for the ethernet-port-3 (does that even make sense?). Screen shot attached, and it looks like the cameras IP (169.254.0.1) was never affect and is falling under the expansion board GEV INTERFACE 1. Anyhow, now that I have the correct IP that doesn’t help me since i’m missing a protocol. Right now the expansion board and camera seems to have slightly different subnets:

Camers: 255.255.0.0
Expansion Board: 255.255.255.0

Thanks for the help so far :-)!

With regard to networking, assuming “eth3” at “182.182.1.3” is all on the Jetson side, then any system which can “ping 182.182.1.3” implies basic networking is correct for that subnet.

The next question becomes “what IP address does the camera have? Can that IP address be pinged?”. Pinging the interface eth3 implies anything attached to that interface and set up for an address within the “182.182.1.x” address range will work. Presumably the camera is in that address range and attached to the eth3 PHY. If the camera has a known address, and if that device responds to ICMP protocol, then you should also be able to ping the camera address.

I do not know enough about rtsp to debug it, but if the “.255” address is for broadcast, and if the camera listens to broadcast of that protocol on that subnet, then it should work. Broadcast can at times require more support from networking setup than simple routes to a specific address. Since you verified RTSP is not supported, then there is no reason to look further at broadcast setup or requirements.

Note that in your Python I see you actually trying to talk to “182.182.1.3”. If this is the address of eth3 on the Jetson, and this is not instead the camera’s address, then this would explain why it fails. You have to talk to the camera, not the Jetson’s host address. So it comes back to knowing what the actual camera IP address is. If both the camera and the Jetson eth3 claim address “182.182.1.3”, then this is an error. What do you know about the camera’s address (presumably it is something between “182.182.1.0" and “182.182.1.255”, excluding “182.182.1.3”, and not ending with “.0” or “.255”). If you have this information, then change the Python script to use that address; also ping that address.

1 Like

Yes! All you say seems to make total sense. :-)!

After many days of I was able to find out that the strong texteth3 has an IP of 182.182.1.3 and a subnet of 255.255.255.0. However the attached camera has an IP: 169.254.0.1 subnet: 255.255.0.0. I only know how to interface and change the settings for the eth3 and not the camera, so when I attempt to change the eth3 subnet to the same as the camera the connection to the camera gets lost inside the SDK (as if nothing was connected to eth3). Any idea of how you would go about changing IP and subnet for something connect via an Ethernet port?

I should be able to open the camera if I could only get them on the same subnet.

Even more progress now.

The manufacturer let me know that their SDK also installed a file at /usr/bin/GigEConfig that allows for extremely easy modification of the connected cameras IP and subnet. It has a “force IP” function that forces the IP to work with the expansion-board/ethernetinterface IP addresse :-)!

Regarding the original matter this case have successfully been solved :-).

Thanks alot for the guidance linuxdev, you are a hero!!