Gstnvarguscamerasrc doesn't apply noise reduction post processing?

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi,

I’m using TX2NX on JP 4.6.3, but I think this question related to all Jetson platforms.

In code of gst-nvarguscamera_src package inside L4T 32.7.3 public source, I saw that the CaptureSession creates request without arguments:

    // Create capture request and enable output stream.
    src->request = UniqueObj<Request>(iCaptureSession->createRequest());
    iRequest = interface_cast<IRequest>(src->request);
    src->iRequest_ptr = iRequest;
    if (!iRequest)
      ORIGINATE_ERROR("Failed to create Request");
    iRequest->enableOutputStream(src->outputStream.get());

From createRequest() API (Jetson Linux API Reference: Argus::ICaptureSession Class Reference | NVIDIA Docs), the CAPTURE_INTENT_PREVIEW was used for this default request created.

From definition of CAPTURE_INTENT_PREVIEW,

/**
 * A CaptureIntent may be provided during capture request creation to initialize the new
 * Request with default settings that are appropriate for captures of the given intent.
 * More details regarding each intent are as follows:
 * MANUAL intent disables auto white balance and auto-focus.
 * PREVIEW intent disables noise reduction related post-processing in order to
 * reduce latency and resource usage.
 * STILL_CAPTURE intent enables Noise Reduction related post-processing in order
 * to optimize still image quality.
 * VIDEO_RECORD intent enables motion sensors related post-processing to optimize
 * the video quality.
 * Apart from above processing blocks each intent also helps in optimizing the
 * processing resource usage appropriate for that intent.
 */
DEFINE_NAMED_UUID_CLASS(CaptureIntent);
DEFINE_UUID(CaptureIntent, CAPTURE_INTENT_MANUAL,         FB3F3663,CC62,11E5,9956,62,56,62,87,07,61);
DEFINE_UUID(CaptureIntent, CAPTURE_INTENT_PREVIEW,        FB3F3664,CC62,11E5,9956,62,56,62,87,07,61);
DEFINE_UUID(CaptureIntent, CAPTURE_INTENT_STILL_CAPTURE,  FB3F3665,CC62,11E5,9956,62,56,62,87,07,61);
DEFINE_UUID(CaptureIntent, CAPTURE_INTENT_VIDEO_RECORD,   FB3F3666,CC62,11E5,9956,62,56,62,87,07,61);
DEFINE_UUID(CaptureIntent, CAPTURE_INTENT_VIDEO_SNAPSHOT, FB3F3667,CC62,11E5,9956,62,56,62,87,07,61);

The PREVIEW option will disable noise reduction related post-processing.

In gstnvarguscamerasrc plugin, has two properties for noise reduction:

  tnr-strength        : property to adjust temporal noise reduction strength
                        flags: readable, writable
                        Float. Range:              -1 -               1 Default:              -1 
  tnr-mode            : property to select temporal noise reduction mode
                        flags: readable, writable
                        Enum "GstNvArgusCamTNRMode" Default: 1, "NoiseReduction_Fast"
                           (0): NoiseReduction_Off - GST_NVCAM_NR_OFF
                           (1): NoiseReduction_Fast - GST_NVCAM_NR_FAST
                           (2): NoiseReduction_HighQuality - GST_NVCAM_NR_HIGHQUALITY

So that means the noise reduction options in gstnvarguscamerasrc plugin are not affected?

To have noise reduction in gstnvarguscamerasrc, have to change its code to use CAPTURE_INTENT_VIDEO_RECORD in CaptureRequest?

Thank you.

3

hello Hnil.DN

may I know what’s the actual use-case, there’re different capture intents.
you may try enable argus_camera application, which include user-interface for changing camera controls, and you’re able to see the changes dynamically.

4

Hi Jerry,

I’m developing camera video pipeline, so need to understand the option to have de-noise setting correctly set.

I checked argus_camera code, it has preview video on screen which uses PREVIEW capture intent, so same with gstnvarguscamerasrc, but the still capturing and video recording of argus_camera using different intents.

I think, the gstnvarguscamerasrc plugin has denoise properties to enable & change strength, but if PREVIEW intent was used, these denoise properties are not effect. So these properties are useless now and confuses for user?

5

hello Hnil.DN

here’s more in-depth explanation for capture intents.
for example,
A CaptureIntent may be provided during capture request creation to initialize the new request with default settings that are appropriate for captures of the given intent.
More details regarding each intent are as follows:

  • MANUAL intent disables auto white balance and auto-focus.
  • PREVIEW intent disables noise reduction related post-processing in order to reduce latency and resource usage.
  • STILL_CAPTURE intent enables Noise Reduction related post-processing in order to optimize still image quality.
  • VIDEO_RECORD intent enables motion sensors related post-processing to optimize the video quality.

Apart from above processing blocks each intent also helps in optimizing the processing resource usage appropriate for that intent.

6

Hi Jerry,

I saw that from the intent definition already.

The problem is the gstnvarguscamerasrc plugin makes me confuse when using it. The plugin has two properties for controlling noise reduction, tnr-mode and tnr-strength. Is this noise reduction related to “noise reduction related post-processing” in the intent definition?

The gstnvarguscamerasrc uses intent CAPTURE_INTENT_PREVIEW which means noise reduction is disabled in post-processing. But the plugin has tnr properties to control noise reduction while it is not enabled. This doesn’t make sense?

7

Hi Jerry,

Can you please confirm the gstnvarguscamerasrc in JP 4.6, its two properties tnr-strength and tnr-mode are not effect since the CAPTURE_INTENT_PREVIEW was used to read video buffer?

8

hello Hnil.DN,

I’ve already bring this question to discuss internally.

BTW, could you please help to narrow down the issue, for example, are you able to see different image quality by applying tnr-strength and tnr-mode with gst pipeline?
you may also enable Argus daemon LOGs to obtain debug logs.
for example,

$ sudo su 
# pkill nvargus-daemon
# export enableCamCoreLogs=5
# nvargus-daemon
or.. # nvargus-daemon 2>&1 | tee argus-output.log
9

Hi Jerry,

Thank you for bringing it up.

This is argus-daemon log. I did the test with below commands.

gstd-client pipeline_create cam_src_pipe nvarguscamerasrc name=argussrc bufapi-version=1 silent=1 ! \
		video/x-raw\(memory:NVMM\),width=1932,height=1090,format=NV12,framerate=30/1 ! \
		fakesink

gstd-client pipeline_play cam_src_pipe

gstd-client element_set cam_src_pipe argussrc tnr-mode 1
gstd-client element_set cam_src_pipe argussrc tnr-strength 0
gstd-client element_set cam_src_pipe argussrc tnr-strength 1.0
gstd-client element_set cam_src_pipe argussrc tnr-mode 2
gstd-client element_set cam_src_pipe argussrc tnr-strength 0.2
gstd-client element_set cam_src_pipe argussrc tnr-mode 0

gstd-client pipeline_stop cam_src_pipe
gstd-client pipeline_delete cam_src_pipe

argus-output.log.tgz (101.5 KB)

10

hello Hnil.DN,

TNR can be handled by… (1) ISP post-processing, or (2) VIC.

hence,
the PREVIEW intent disables the noise reduction at ISP post-processing.
it’s the GST plugin enables the SCF’s TNR which is related to VIC.

11

Hi Jerry.

Thank you for your confirmation.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.