Using NvVideoEncoder for USB Camera capture frames streaming

Hello,

I’m working on the development of a software which must capture frames from an USB Camera (which can eventually be replaced by a tc358743 csi-hdmi bridge), encode the frames in H.264 and stream the encoded frames using live555 library.

My plan is to use the NvVideoEncoder class (Jetson Linux API Reference: NvVideoEncoder Class Reference | NVIDIA Docs) together with capturing the frames using V4L calls and MMAP configuration.

The resulting output, delivered on NvVideoEncoder callback, will be delivered to the live555 library.

Is that the correct approach?

The problem that I am facing now is to figure out how to get setup the v4l2_buffer structure arrived from the camera and send it to NvVideoEncoder library. As far as I understood from the example codes, the class is waiting for DMA_BUF, which uses file descriptor to identify the buffer position. But I have not found anything that uses MMAP as input to the NvVideoEncoder.

Is there any guide or example on how to send MMAP based buffers to NvVideoEncoder?

Best regards,

Flavio

Hi,
The similar reference is 12_camera_v4l2_cuda + NvVideoEncoder. Please try

  1. Try the sample to capture frame data:

/usr/src/jetson_multimedia_api/samples/12_v4l2_camera_cuda/

  1. Apply the patch to encode frames to H264 stream
    TX2 Camera convert/encode using Multimedia API issue - #17 by DaneLLL
  2. Integrate with live555 library

Hello @DaneLLL , my apologies for the late reply.

I was trying to build and execute the example, but I am having the following build issue:

 sudo make
[sudo] password for jetson: 
Linking: camera_v4l2_cuda
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegEncoder.o: In function `NvJPEGEncoder::NvJPEGEncoder(char const*)':
NvJpegEncoder.cpp:(.text+0x64): undefined reference to `jpeg_std_error'
NvJpegEncoder.cpp:(.text+0x84): undefined reference to `jpeg_CreateCompress'
NvJpegEncoder.cpp:(.text+0x94): undefined reference to `jpeg_suppress_tables'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegEncoder.o: In function `NvJPEGEncoder::~NvJPEGEncoder()':
NvJpegEncoder.cpp:(.text+0x1a0): undefined reference to `jpeg_destroy_compress'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegEncoder.o: In function `NvJPEGEncoder::encodeFromFd(int, J_COLOR_SPACE, unsigned char**, unsigned long&, int)':
NvJpegEncoder.cpp:(.text+0x4f8): undefined reference to `jpeg_mem_dest'
NvJpegEncoder.cpp:(.text+0x534): undefined reference to `jpeg_set_defaults'
NvJpegEncoder.cpp:(.text+0x548): undefined reference to `jpeg_set_quality'
NvJpegEncoder.cpp:(.text+0x570): undefined reference to `jpeg_set_hardware_acceleration_parameters_enc'
NvJpegEncoder.cpp:(.text+0x59c): undefined reference to `jpeg_start_compress'
NvJpegEncoder.cpp:(.text+0x8ac): undefined reference to `jpeg_write_raw_data'
NvJpegEncoder.cpp:(.text+0x8b8): undefined reference to `jpeg_finish_compress'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegEncoder.o: In function `NvJPEGEncoder::encodeFromBuffer(NvBuffer&, J_COLOR_SPACE, unsigned char**, unsigned long&, int)':
NvJpegEncoder.cpp:(.text+0xd04): undefined reference to `jpeg_mem_dest'
NvJpegEncoder.cpp:(.text+0x13c4): undefined reference to `jpeg_set_defaults'
NvJpegEncoder.cpp:(.text+0x13d8): undefined reference to `jpeg_set_quality'
NvJpegEncoder.cpp:(.text+0x1400): undefined reference to `jpeg_set_hardware_acceleration_parameters_enc'
NvJpegEncoder.cpp:(.text+0x142c): undefined reference to `jpeg_set_colorspace'
NvJpegEncoder.cpp:(.text+0x1758): undefined reference to `jpeg_start_compress'
NvJpegEncoder.cpp:(.text+0x1a18): undefined reference to `jpeg_write_raw_data'
NvJpegEncoder.cpp:(.text+0x1a3c): undefined reference to `jpeg_finish_compress'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o: In function `NvJPEGDecoder::NvJPEGDecoder(char const*)':
NvJpegDecoder.cpp:(.text+0x64): undefined reference to `jpeg_std_error'
NvJpegDecoder.cpp:(.text+0x84): undefined reference to `jpeg_CreateDecompress'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o: In function `NvJPEGDecoder::~NvJPEGDecoder()':
NvJpegDecoder.cpp:(.text+0x190): undefined reference to `jpeg_destroy_decompress'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o: In function `NvJPEGDecoder::decodeToFd(int&, unsigned char*, unsigned long, unsigned int&, unsigned int&, unsigned int&)':
NvJpegDecoder.cpp:(.text+0x4fc): undefined reference to `jpeg_mem_src'
NvJpegDecoder.cpp:(.text+0x518): undefined reference to `jpeg_read_header'
NvJpegDecoder.cpp:(.text+0x5b4): undefined reference to `jpeg_start_decompress'
NvJpegDecoder.cpp:(.text+0x730): undefined reference to `jpeg_read_raw_data'
NvJpegDecoder.cpp:(.text+0x73c): undefined reference to `jpeg_finish_decompress'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o: In function `NvJPEGDecoder::decodeToBuffer(NvBuffer**, unsigned char*, unsigned long, unsigned int*, unsigned int*, unsigned int*)':
NvJpegDecoder.cpp:(.text+0xca8): undefined reference to `jpeg_mem_src'
NvJpegDecoder.cpp:(.text+0xcc4): undefined reference to `jpeg_read_header'
NvJpegDecoder.cpp:(.text+0xdd4): undefined reference to `jpeg_start_decompress'
NvJpegDecoder.cpp:(.text+0xff0): undefined reference to `jpeg_finish_decompress'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o: In function `NvJPEGDecoder::decodeIndirect(NvBuffer*, unsigned int)':
NvJpegDecoder.cpp:(.text+0x152c): undefined reference to `jpeg_read_raw_data'
/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o: In function `NvJPEGDecoder::decodeDirect(NvBuffer*, unsigned int)':
NvJpegDecoder.cpp:(.text+0x1e78): undefined reference to `jpeg_read_raw_data'
collect2: error: ld returned 1 exit status
Makefile:61: recipe for target 'camera_v4l2_cuda' failed
make: *** [camera_v4l2_cuda] Error 1
jetson@jetson-draganfly:/usr/src/jetson_multimedia_api/samples/12_camera_v4l2_cuda$ 

I was not able to find a thread with the solution for this issue. That’s why it took so long.

Could you give me any advise on how to solve this link issue? I’m running Ubuntu 18.04 on Jetson Nano developer kit.

Hi,
The latest Jetpack release for Jetson Nano is 4.6.4. We would suggest re-flash the system image and install SDK Components through SDKManager. All required files will be installed.

Hello @DaneLLL ,

It took me a while to have the correct setup working. I needed to create an Ubuntu 18.04 virtual machine running the latest version of SDKMANAGER to have everything ready to setup my new SD card.

(I tried an older sdkmanager on a newer Ubuntu without success, and also the older sdkmanager on an older Ubuntu, without success either).

Then I compiled the example on my newer SD card and I got the following error:

jetson@ubuntu:/usr/src/jetson_multimedia_api/samples/12_camera_v4l2_cuda$ sudo make
make[1]: Entering directory '/usr/src/jetson_multimedia_api/samples/common/classes'
Compiling: NvJpegDecoder.cpp
NvJpegDecoder.cpp:35:10: fatal error: jpegint.h: No such file or directory
 #include "jpegint.h"
          ^~~~~~~~~~~
compilation terminated.
Makefile:40: recipe for target 'NvJpegDecoder.o' failed
make[1]: *** [NvJpegDecoder.o] Error 1
make[1]: Leaving directory '/usr/src/jetson_multimedia_api/samples/common/classes'
Makefile:48: recipe for target '/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o' failed
make: *** [/usr/src/jetson_multimedia_api/samples/common/classes/NvJpegDecoder.o] Error 2

For this error I found the solution:

sudo apt-get install libjpeg-dev

Now the code is compiling and I finally got the example running.

My next step is to apply the patch. I’ll get back when I’m done.

Best regards,

Flavio

@DaneLLL , I applied the patch and it worked.

The compilation and execution went well. And I realized that the generated file does not contain encoded frames. But I believe I can go on from here.

Now I will work with live555 integration.

Best regards,

Flavio

1 Like