TX1 Jpeg Encoder issue

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hello,

I am encoding jpeg image using Nvidia Jpeg Encoder NvJPEGEncoder::encodeFromFd() using fd received from Nvidia converter and I am seeing a strange effect. If I am encoding 1280 x 720 image then everything is perfect. However, if I encode 1920 x 1080, than a few bottom pixel rows are cut and the whole image is stretched vertically to fill 1080 height. Encoding H264 video using same fds received from Nvidia converter is fine in both resolutions, no cropping and stretching.

Do you know what is going on? Our system is TX1 R24.2.

Thanks!

2

Hi, can you try r28.2 and r28.2.1?

3

Unfortunately I cannot. We do not support R28 yet because our camera sensor has no driver for R28 yet. So we need this working on R24.2

4

Hi, do you try R24.2.3?

5

I see same thing in R24.2.1 Where can I get R24.2.3? It is not listed on nvidia archive page (JetPack Archive | NVIDIA Developer).

I also noticed some other issues related to jpeg decoding/encoding. For example decoding jpeg image 4000x3000 to fd and then encoding it from fd produces jpeg image of 4000x3008. Also decoding YUV422 to fd and then encoding from fd produces wrong colors (as if color planes are flipped vertically). Very strange.

Are you able to reproduce these issues?

6

https://developer.nvidia.com/embedded/linux-tegra-r2423

We suggest you try r28, but if your usecase is at final stage, still wish you could try r24.2.3

What is your usecase? A drone with 720p, 1080p? Or?

7

Hi,
On R24.2.3, I add below patch to 12_camera_v4l2_cuda

diff --git a/multimedia_api/ll_samples/samples/12_camera_v4l2_cuda/camera_v4l2_cuda.cpp b/multimedia_api/ll_samples/samples/12_camera_v4l2_cuda/camera_v4l2_cuda.cpp
index a3d795d..008c560 100644
--- a/multimedia_api/ll_samples/samples/12_camera_v4l2_cuda/camera_v4l2_cuda.cpp
+++ b/multimedia_api/ll_samples/samples/12_camera_v4l2_cuda/camera_v4l2_cuda.cpp
@@ -44,6 +44,7 @@
 #include "nvbuf_utils.h"
 
 #include "camera_v4l2_cuda.h"
+#include "NvJpegEncoder.h"
 
 static bool quit = false;
 
@@ -107,7 +108,7 @@ parse_cmdline(context_t * ctx, int argc, char **argv)
                     print_usage();
                     return false;
                 }
-                sprintf(ctx->cam_file, "camera.%s", optarg);
+                sprintf(ctx->cam_file, "/home/ubuntu/camera.raw");
                 break;
             case 'r':
                 ctx->fps = strtol(optarg, NULL, 10);
@@ -570,7 +571,26 @@ conv_capture_dqbuf_thread_callback(struct v4l2_buffer *v4l2_buf,
         NvDestroyEGLImage(ctx->egl_display, ctx->egl_image);
         ctx->egl_image = NULL;
     }
-
+    if (ctx->frame == ctx->save_n_frame) {
+        NvJPEGEncoder *m_JpegEncoder;
+        m_JpegEncoder = NvJPEGEncoder::createJPEGEncoder("jpenenc");
+        char filename[FILENAME_MAX];
+        sprintf(filename, "/home/ubuntu/output%03u.jpg", (unsigned) ctx->frame);
+
+        std::ofstream *outputFile = new std::ofstream(filename);
+        if (outputFile)
+        {
+            unsigned long size;
+            size = ctx->cam_w*ctx->cam_h*3/2;
+            unsigned char *m_OutputBuffer;
+            m_OutputBuffer = new unsigned char;
+            m_JpegEncoder->encodeFromFd(buffer->planes[0].fd, JCS_YCbCr, &m_OutputBuffer, size);
+            outputFile->write((char *)m_OutputBuffer, size);
+            delete outputFile;
+            delete [] m_OutputBuffer;
+        }
+        delete m_JpegEncoder;
+    }
     // Render the frame into display
     if (v4l2_buf->m.planes[0].bytesused)
         ctx->renderer->render(buffer->planes[0].fd);

And plugin USB camera which can output 1080p. And run

12_camera_v4l2_cuda$ ./camera_v4l2_cuda -d /dev/video1 -s 1920x1080 -f UYVY -n 30

With the patch, same raw frame and JPEG are stored. I don’t see obvious last-few-lines-being-cut issue.
Attach the raw frame and JPEG for reference.

output030.jpg
output030.jpg1920×1080 196 KB

camera.zip (1.99 MB)

output030.jpg
output030.jpg1920×1080 196 KB

8

This is not exactly the case. In our case the problematic fd is not from camera but either from jpeg decoder or converter. Our cases are:

  1. We take 1920x1080 rgb image in CPU memory which is fed to converter to create YUV420. Resulting fd is used for jpeg encoding. We see bottom cropped.
  2. We take 4000x3000 YUV420 jpeg image and jpeg decode to fd. Resulting fd is used for jpeg encoding. We see that 4000x3008 image is created by encoder. Another issue (not important as we can easily work around it): if we take YUV422 input jpeg image then output jpeg image has colors messed up.

The following simple code demonstrates #2 (both issues if you feed YUV422 input image). Could you please try it?

Thanks!

// Read file
    FILE* pFile = fopen("default.jpg", "r");
    fseek(pFile, 0, SEEK_END);
    unsigned long uSize = ftell(pFile);
    fseek(pFile, 0, SEEK_SET);
    unsigned char* pBuffer = (unsigned char*)malloc(uSize);
    uSize = fread(pBuffer, 1, uSize, pFile);
    fclose(pFile);
    printf("Read default.jpg (%lu)\n", uSize);
    // Decode jpg
    int      fd     = -1;
    uint32_t pixfmt = 0;
    uint32_t width  = 0;
       uint32_t height = 0;
    NvJPEGDecoder* pDecoder = NvJPEGDecoder::createJPEGDecoder("jpegdec");    
    pDecoder->decodeToFd(fd, pBuffer, uSize, pixfmt, width, height);
    printf("Decoded default.jpg (%ux%u, 0x%08X)\n", width, height, pixfmt);
    // Encode jpg
    NvJPEGEncoder* pEncoder = NvJPEGEncoder::createJPEGEncoder("jpenenc");
    pEncoder->encodeFromFd(fd, JCS_YCbCr, &pBuffer, uSize);
    printf("Encoded default.jpg (%lu)\n", uSize);
    // Write file
    pFile = fopen("default2.jpg", "wb");
    fwrite(pBuffer, 1, uSize, pFile);
    fclose(pFile);
    printf("Wrote default2.jpg (%lu)\n", uSize);
    // Clean up
    free(pBuffer);
    delete pDecoder;
    delete pEncoder;
9

Hi, so the issue is still present on r24.2.3?

The HW converter supports RGBA and BGRx. For 1920x1080 rgb image you mentioned, which one do you pick?

10

I am not able to try R24.2.3 at the moment. Any chance you can compile my code to verify it on your end?

About HW converter: I made a slight mistake - it is YUV420, not RGB. Converter setup is:

pConverter->setOuputPlaneFormat(V4L2_PIX_FMT_YUV420M, 1920, 1080, V4L2_NV_BUFFER_LAYOUT_PITCH);
pConverter->setCapturePlaneFormat(V4L2_PIX_FMT_YUV420M, 1920, 1080, V4L2_NV_BUFFER_LAYOUT_BLOCKLINEAR);
pConverter->output_plane.setupPlane(V4L2_MEMORYMMAP, 10, true, false);
pConverter->capture_plane.setupPlane(V4L2_MEMORYMMAP, 10, false, false);

Thanks!

11

Hi B00000,
You already have the product on market and cannot do software upgrade?

12

The product is not released yet but we cannot do updates because we use specific camera sensor from 3rd party and the only driver available at the moment is for R24.2.1 So upper management made a decision to go to release with that version and we (dev) have to make it work :-(

Were you able to reproduce the issue in R24.2.3 or R28.2 using the code I sent earlier?

Thanks!

13

Hi B00000,
Please try

$ ./jpeg_encode a.yuv 1920 1080 a_0.jpg <b>-crop 1920 1080</b>

The output of video converter is with width/height alignment. Please configure crop region.