Customizing sample_plugin error in LAC settings

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Hi,

I am trying to customize the LAC and histogram windows that NvMedia uses by modifying the sample_plugin.c example code. I found if I made any change to LAC settings, I would get an error when trying to run the camera.

Error I was seeing:
(NvCapture) Error InvalidState: WARNING: LAC requested and returned window count does not match. Actual 896 Expected 1024
(in /dvs/git/dirty/git-master_linux/camera/capture/nvispng/isp5/stats_hw.cpp, function NvIspLocalAverageClipDecode5(), line 441)
(NvCapture) Error InvalidState: (propagating from /dvs/git/dirty/git-master_linux/camera/capture/nvispng/isp5/stats_hw.cpp, function NvIspStatsDecode5(), line 505)
Decode stats, error 8
ISP5GetStats: Failed to parse lac[0] stats
[GetLacStatsT19x:50] Failed to get LAC[0] stats

The change I made for testing was to modify sample_plugin.c, initializeLACSettings0() from
numWindowsY = 32
to
numWindowsY = 28

I would expect the numWindowsX * numWindowsY should be the total window count, but it seems like it’s always expecting 1024 (32*32).

I was able to change histogram configuration successfully. I’m using Drive 5.1.0.2 on Xavier (not pegasus).

2

Dear gordon1zrra,

Could you share your code if possible? I will check/build it on my machine. Thanks.

3

Sure, this is the modified function

static void
initializeLACSettings0(
PluginContext *ctx,
NvMediaIPPPluginOutputStreamSettings *streamSettings,
int width,
int height)
{
int numWindowsX, numWindowsY;
int numPixels, numRemainder;
float maxLac0Range;
int i, j, index = 0;
int ROIOffset_x = 0, ROIOffset_y = 0;

if (ctx->ispVersion == NVMEDIA_IPP_ISP_VERSION_4) {
    // To compute averages of the MSB 12 bit of 14 bits (ISP percision on V4)
    maxLac0Range = powf(2.0f, 12) / powf(2.0f, 14);

    streamSettings->v4.lacSettingsValid[0] = NVMEDIA_TRUE;

    streamSettings->v4.lacSettings[0].enable = 1;
    streamSettings->v4.lacSettings[0].pixelFormat = NVMEDIA_ISP_PIXELFORMAT_QUAD;
    streamSettings->v4.lacSettings[0].rgbToYGain[0] = 0.29296875;
    streamSettings->v4.lacSettings[0].rgbToYGain[1] = 0.57421875;
    streamSettings->v4.lacSettings[0].rgbToYGain[2] = 0.132812545;
    streamSettings->v4.lacSettings[0].rgbToYGain[3] = 0;
    streamSettings->v4.lacSettings[0].range[0].low = 0.0;
    streamSettings->v4.lacSettings[0].range[0].high = maxLac0Range;
    streamSettings->v4.lacSettings[0].range[1].low = 0.0;
    streamSettings->v4.lacSettings[0].range[1].high = maxLac0Range;
    streamSettings->v4.lacSettings[0].range[2].low = 0.0;
    streamSettings->v4.lacSettings[0].range[2].high = maxLac0Range;
    streamSettings->v4.lacSettings[0].range[3].low = 0.0;
    streamSettings->v4.lacSettings[0].range[3].high = maxLac0Range;
    //Set up 2x2 ROI grid and each ROI to have 32x32 windows
    for (i = 0; i < 2; i++, ROIOffset_y += height / 2) {
        ROIOffset_x = 0;
        for (j = 0; j < 2; j++, ROIOffset_x += width / 2, index++) {
            streamSettings->v4.lacSettings[0].ROIEnable[index] = NVMEDIA_TRUE;
            numWindowsX = 32;
            numPixels = width / (numWindowsX * 2);
            numRemainder = (width / 2)- (numPixels * numWindowsX);
            streamSettings->v4.lacSettings[0].windows[index].horizontalInterval = numPixels;
            streamSettings->v4.lacSettings[0].windows[index].size.width =
                        numPixels >= 32 ? 32 :
                        numPixels >= 16 ? 16 :
                        numPixels >= 8  ? 8  :
                        numPixels >= 4  ? 4  :
                        numPixels >= 2  ? 2  : 0;
            streamSettings->v4.lacSettings[0].windows[index].startOffset.x =
                (numRemainder + (numPixels - streamSettings->v4.lacSettings[0].windows[index].size.width)) / 2 + ROIOffset_x;
            numWindowsY = 32;
            numPixels = height / (numWindowsY * 2);
            numRemainder = (height / 2) - (numPixels * numWindowsY);
            streamSettings->v4.lacSettings[0].windows[index].verticalInterval = numPixels;
            streamSettings->v4.lacSettings[0].windows[index].size.height =
                        numPixels >= 32 ? 32 :
                        numPixels >= 16 ? 16 :
                        numPixels >= 8  ? 8  :
                        numPixels >= 4  ? 4  :
                        numPixels >= 2  ? 2  : 0;
            streamSettings->v4.lacSettings[0].windows[index].startOffset.y =
                (numRemainder + (numPixels - streamSettings->v4.lacSettings[0].windows[index].size.height)) / 2 + ROIOffset_y;
            streamSettings->v4.lacSettings[0].windows[index].horizontalNum = numWindowsX;
            streamSettings->v4.lacSettings[0].windows[index].verticalNum = numWindowsY;
        }
    }
} else if (ctx->ispVersion == NVMEDIA_IPP_ISP_VERSION_5){
    // Compute average for the full range input
    maxLac0Range = 1.0f;

    streamSettings->v5.lacSettingsValid[0] = NVMEDIA_TRUE;
    streamSettings->v5.lacSettings[0].enable = NVMEDIA_TRUE;
    /* Image plane for Stats for Multi exposure mode
       HDR image is processed as one plane (combination in sensor) */
    streamSettings->v5.lacSettings[0].laneSelect = NVMEDIA_ISP_MULTI_EXP_LANE_RV;
    streamSettings->v5.lacSettings[0].R2YGain = 0.29296875;
    streamSettings->v5.lacSettings[0].G2YGain = 0.57421875;
    streamSettings->v5.lacSettings[0].B2YGain = 0.132812545;
    streamSettings->v5.lacSettings[0].R2YOffset = 0;
    streamSettings->v5.lacSettings[0].G2YOffset = 0;
    streamSettings->v5.lacSettings[0].B2YOffset = 0;
    streamSettings->v5.lacSettings[0].range[0].low = 0.0;
    streamSettings->v5.lacSettings[0].range[0].high = maxLac0Range;
    streamSettings->v5.lacSettings[0].range[1].low = 0.0;
    streamSettings->v5.lacSettings[0].range[1].high = maxLac0Range;
    streamSettings->v5.lacSettings[0].range[2].low = 0.0;
    streamSettings->v5.lacSettings[0].range[2].high = maxLac0Range;
    streamSettings->v5.lacSettings[0].range[3].low = 0.0;
    streamSettings->v5.lacSettings[0].range[3].high = maxLac0Range;
    //Set up 2x2 ROI grid and each ROI to have 32x32 windows
    for (i = 0; i < 2; i++, ROIOffset_y += height / 2) {
        ROIOffset_x = 0;
        for (j = 0; j < 2; j++, ROIOffset_x += width / 2, index++) {
            streamSettings->v5.lacSettings[0].ROIEnable[index] = NVMEDIA_TRUE;
            streamSettings->v5.lacSettings[0].radialMaskEnable[index] = NVMEDIA_FALSE;
            numWindowsX = 32;
            numPixels = width / (numWindowsX * 2);
            numPixels = (numPixels % 2) ? numPixels - 1 : numPixels;
            numRemainder = (width / 2) - (numPixels * numWindowsX);
            streamSettings->v5.lacSettings[0].windows[index].horizontalInterval = numPixels;
            streamSettings->v5.lacSettings[0].windows[index].size.width = (numPixels > 256) ?
                                                                             256 : numPixels;
            streamSettings->v5.lacSettings[0].windows[index].startOffset.x =
                (numRemainder +
                (numPixels - streamSettings->v5.lacSettings[0].windows[index].size.width)) / 2
                 + ROIOffset_x;
            //numWindowsY = 32;
            numWindowsY = 28;
            numPixels = height / (numWindowsY * 2);
            numPixels = (numPixels % 2) ? numPixels - 1 : numPixels;
            numRemainder = (height / 2) - (numPixels * numWindowsY);
            streamSettings->v5.lacSettings[0].windows[index].verticalInterval = numPixels;
            streamSettings->v5.lacSettings[0].windows[index].size.height = (numPixels > 256) ?
                                                                              256 : numPixels;
            streamSettings->v5.lacSettings[0].windows[index].startOffset.y =
                (numRemainder +
                (numPixels - streamSettings->v5.lacSettings[0].windows[index].size.height)) / 2
                 + ROIOffset_y;
            streamSettings->v5.lacSettings[0].windows[index].horizontalNum = numWindowsX;
            streamSettings->v5.lacSettings[0].windows[index].verticalNum = numWindowsY;
        }
    }
}else {
    LOG_ERR("%s: isp version not supported\n", __func__, ctx->ispVersion);
}

}

And this is the command I used to test
drive-t186ref-linux/samples/nvmedia/ipp_raw/x11/nvmipp_raw -cf ddpx-a.conf -c SF3325-CSI-A --sampleplugin

4

Dear gordon1zrra,

It looks like we have a bug forcing us to have the setting to be always 32 by 32.
Could please let me know what the issue in using 32 by 32 for LAC statistics is? Thanks.

5

On the PX2 we had reconfigured the statistic windows to prioritize regions we cared about rather than the entire image. I think we rework our code so that we just pick specific windows within the 32x32 grid. Is there any plan to fix the issue?

6

Dear gordon1zrra,

I see, will check/update it. Thanks.

7

Hi gordon1zrra

Issue will be fixing and included at future release, please stay tuned.

8

Thanks for the update. Is there any rough idea when next release will be?

9

Please wait for our announcement, or can contact with our region NVIDIA representative to get the status.

Thanks