Hi. I am trying to retrieve images from a network camera using RTSP which I will then perform object detection on. I have tried a number of different ways, none of which seem to work well at this stage.
To confirm that my camera (Anpvis UND950W) is OK and that the basics all see to be working, I have executed the following and it produces a nice image with no noticable lag.
$ gst-launch-1.0 rtspsrc latency=0 location=rtsp://admin:firstname.lastname@example.org/media/video2 ! rtph264depay ! h264parse ! decodebin ! nvvidconv ! xvimagesink
The output from jtop suggests it is using the GPU, Wireshark confirms UDP (and not TCP) packets, and pretty much zero noticeable lag, so all good! Now I just need to be able to get at the images instead of displaying them on the screen.
I wrote some Python in an attempt to do something similar inside a program with a call to jetson.utils.gstCamera() but I couldn’t get it to work, and the post at detectnet-video said that this is the wrong way to do it and that jetson.utils.videoOutput() is in fact the way to access RTSP. This is what is done in detectnet-camera.py. So I tried a few things for a proof of concept and the following does indeed work and displays images on the screen.
$ detectnet-camera.py --input-codec=h264 rtsp://admin:email@example.com/media/video2
The problem is this has lots of lag. About three seconds of lag and I cannot find a way to reduce the lag (what I did successfully in the gstreamer path by specifying latency=0). Is there a way to somehow specify latency=0, or to achieve the same effect? If there is, I couldn’t find it.
Another thing I tried was to use OpenCV instead. … Just the important bits follow…
net = jetson.inference.detectNet(myNetwork, sys.argv, myThreshold)
pipe = “rtspsrc latency=0 location=rtsp://admin:firstname.lastname@example.org/media/video2 ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! appsink”
cap = cv2.VideoCapture(pipe)
ret, frame = cap.read()
OK, the retrieved frame is good and I can display it using cv2.imshow(‘display’, frame) for example. The thing is I really want to do something like pass the frame to net.Detect(frame, overlay=myOverlay) but it seems the image formats differ and I get an exception that jetson.utils function wasn’t passed a valid cudaImage or cudaMemory object. Hmm, I’m not sure how to convert the image returned by cap.read() into an acceptable format. So I’m stuck there too.
So is there a way to do low-latency RTSP from a network camera while making use of the Nano GPU?
I think I’m so close to a solution, yet…