Decoder SDK - how to access decoded frame?

Also sure wish I could cut and paste the code out of your post without picking up the 468 line numbers at the beginning of each line… :-/

Hi,
Indeed maybe buffer is not full to pass to decoder function, but I bypass NALU function, i didn’t see any information about this package.

The code doesn’t crash but don’t decode.

My code :

parser callbacks
//////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// CUDA 2.0 initialization
//
CUcontext g_cuContext = NULL;
CUdevice g_cuDevice = NULL;

static bool InitCuda()


}

static bool DeinitCuda()
{


}

int main(int argc, char *argv[])
{
    unsigned char io_buffer[16*1024];
    CUVIDPARSERPARAMS parserInitParams;
    CUstream cuStream = NULL;
    CUresult result;
    char *arg_input = NULL;
    char *arg_output = NULL;
    FILE *fInput = NULL;
    int retval = 1;
    int i, elapsed_time;
int index= 0;


printf(“CUDA 0\n”);
// Create video parser
memset(&parserInitParams, 0, sizeof(parserInitParams));
parserInitParams.CodecType = cudaVideoCodec_H264;
parserInitParams.ulMaxNumDecodeSurfaces = MAX_FRM_CNT;
parserInitParams.ulMaxDisplayDelay = 4;
parserInitParams.ulClockRate = 1000000000;
parserInitParams.pUserData = &State;
parserInitParams.pfnSequenceCallback = HandleVideoSequence;
parserInitParams.pfnDecodePicture = HandlePictureDecode2;
parserInitParams.pfnDisplayPicture = HandlePictureDisplay2;
result = cuvidCreateVideoParser(&State.cuParser, &parserInitParams);
printf(“CUDA 1\n”);

// Open input file
fopen_s(&fInput, arg_input, “rb”);
if (fInput == NULL)
{
printf(“Failed to open "%s"\n”, arg_input);
goto exit;
}
// Open output file
if (arg_output)
{
fopen_s(&State.fd_yuv, arg_output, “wb”);
if (State.fd_yuv == NULL)
{
printf(“Failed to create "%s"\n”, arg_output);
goto exit;
}
}
// Start decoding
//elapsed_time = timeGetTime();

	for (;;)
    {
    index++;
    printf("Index%d\n",index);	
        CUVIDSOURCEDATAPACKET pkt;
        int len = (int) fread(io_buffer, 1, sizeof(io_buffer), fInput);
        
        if (len <= 0)
        {
            // Flush the decoder
            pkt.flags = CUVID_PKT_ENDOFSTREAM;
            pkt.payload_size = 0;
            pkt.payload = NULL;
            pkt.timestamp = 0;
            cuvidParseVideoData(State.cuParser, &pkt);
            break;
        }
        pkt.flags = 0;
        pkt.payload_size = len;
        pkt.payload = io_buffer;
        pkt.timestamp = 0;  // not using timestamps
        if (cuvidParseVideoData(State.cuParser, &pkt) != CUDA_SUCCESS)
	    {
			volatile int g = 0;
	printf(" Error %d \n",g );
	    }

    }
printf("Number %d\n",State.DisplayQueue[State.display_pos].picture_index) ;

    // Flush display queue
    for (i=0; i<DISPLAY_DELAY; i++)
    {
        if (State.DisplayQueue[State.display_pos].picture_index >= 0)
        {
	printf(" Display 1\n");
            DisplayPicture(&State, &State.DisplayQueue[State.display_pos]);
            State.DisplayQueue[State.display_pos].picture_index = -1;
        }
        State.display_pos = (State.display_pos + 1) % DISPLAY_DELAY;
    }
    elapsed_time = 0 ; //timeGetTime() - elapsed_time;
    retval = 0;
    printf("Processed %d frames in %dms (%5.2ffps)\n",
        State.pic_cnt, elapsed_time, ((float)State.pic_cnt*1000.0/(float)elapsed_time));

    return retval;
}

You wrote:

“Indeed maybe buffer is not full to pass to decoder function, but i am huge to bypass NALU function, i didn’t see any information about this package.”

I don’t know what you mean by “huge to bypass”. That sounds like you don’t want to inject data by NALUs. If, however, you do then as I said in my first reply I can give you the code to do that.

If you just have an arbitrary data source, make a thread that just keeps injecting the data into the parser as it arrives and use the display callback to pace things.

You have posted your code without explanation or any questions. Is it not working or it is working? Did you have a question about it?

Hello,
Sorry for the lack of explanations.

This code is not working, I didn’t found many information about NALU and file nalu.h, so I bypassed it.

But I don’t understand why the function HandlePictureDecode2 callback is not called during cuvidParseVideoData, because when I add a printf I don’t see it during execution. My opinion is that’s the API function HandlePictureDecode which is called and not this one(in the file). But in this case i don’t see if the frame is decoded…So maybe you can tell me how check this.

I am aware that decoding needs severals packet to called the decode function, but i don’t see any called of previous function during all the parsing of video file.

For NALU I don’t really see the interest ? If you have some explanations ? and code ?

My aim is to use the decode function frame by frame to be able to transmit it(decompressed) over a local network.

Should you give an advise?

Thanks

for details :

    [b]//Init decoding[/b]

    parserInitParams.CodecType = cudaVideoCodec_H264;
    parserInitParams.ulMaxNumDecodeSurfaces = MAX_FRM_CNT;
	parserInitParams.ulMaxDisplayDelay = 4;
	parserInitParams.ulClockRate = 1000000000;
	parserInitParams.pUserData = &State;
    parserInitParams.pfnSequenceCallback = HandleVideoSequence2;
    parserInitParams.pfnDecodePicture = HandlePictureDecode2;
    parserInitParams.pfnDisplayPicture = HandlePictureDisplay2;
	result = cuvidCreateVideoParser(&State.cuParser, &parserInitParams);

    //parse file
int len = (int) fread(io_buffer, 1, sizeof(io_buffer), fInput);

    printf("Len %d\n",len);
    if (len <= 0)
        {
            // Flush the decoder
            pkt.flags = CUVID_PKT_ENDOFSTREAM;
            pkt.payload_size = 0;
            pkt.payload = NULL;
            pkt.timestamp = 0;
            cuvidParseVideoData(State.cuParser, &pkt);
            break;
        }
        pkt.flags = 0;

        pkt.payload_size = len;
        pkt.payload = io_buffer;

        pkt.timestamp = 0;  // not using timestamps
    printf("Parse video data \n");

        [b]//Call cuda parser and decoder[/b]

        if (cuvidParseVideoData(State.cuParser, &pkt) != CUDA_SUCCESS)
	    {
		volatile int g = 0       ;
	printf(" Error %d \n",g );
	    }


    printf("Number %d\n",State.DisplayQueue[State.display_pos].picture_index) ;
       [i] // Here it displays always the queue is NULL but never passed on error[/i].

Your init sets the pfn to HandleVideoSequence2 but the code you posted has HandleVideoSequence(). So you are not giving complete consistent code, which makes it hard to help you.

First check that you get a callback to HandleVideoSequence(). You will not get decode callbacks without it. If you do not successfully execute this first callback, then your stream may be ill-formed. There must be valid SPS and PPS in there at the start of the stream.

The API does not provide HandlePictureDecode(). That is only supplied by the user. So your theory is not the cause of the problem.

If you do not need per-NALU processing then you can ignore it. I needed it in my application because I am indexing the stream and so I have to parse on a per-NALU basis.

Sorry I try to give you more elements :

I rename 3 functions and add trace at begining in new ones:
HandleVideoSequence → HandleVideoSequence2
HandlePictureDecode → HandlePictureDecode2
HandlePictureDisplay → HandlePictureDisplay2

I fill the structure with definition of the 3 function… before called of cuvidParseVideoData.

Normally the first function created a decoder object and call HandlePictureDecode.

But for me the code you post and maybe redefinition of this 3 functions is not necessary as far as i don’t see any call of your 3 function.

You can check there is a printf at beginning of function.

My problem that’s in my case the queue is always NULL and decoding doesnt work.

As I said, if you do not get the sequence callback then your stream data being passed to the parser must be ill-formed (assuming you have init’ed everything correctly). For example, CUVID expects Annex B NALU-based elementary streams and you must have valid SPS/PPS for AVC video at the start. Are these conditions satisfied? What kind of file are you opening? This code works only with elementary streams. If you have a container you’ll have to demux it.

You do not need to rename anything.

1 Like

Re,

I finally resolved the problem. Indeed cuvidParseVideoData make precheck on the bitstream before call of HandleVideoSequence and other function

For my case, the input was not correctly formated…And the pre-check may not allow the callback to run.

Thanks for your answers.

Ben

Good news! I’m glad you are making progress. Post again if further problems arise.

1 Like

Finally got around to trying to build your code. (Although I’ll be away all next week). I have Decoder SDK 7.1.9 for Windows. Having what are probably some dumb build problems.

  1. I don’t have nvcuvid.h - only dynlink_nvcuvid.h also don’t have nalu.h
  2. cuInit() footprint does not match up - you pass 1 parameter - my libraries want 3.
  3. There seems to be a conflict in your code related to USE_NALUS related to #ifdef vs #if. You have it defined to 0 - which means defined. This results (at least in VS2013) that you fail to define io_buffer but later you fread() into the undefined io_buffer
  1. nvcuvid.h is in the CUDA toolkit include directory. If you want to revise the code to use the Video SDK includes, then go for it. Regarding nalu.h, I explained that if you needed it, I would be happy to provide it.

  2. I don’t know what you mean by “my libraries”. See here:

[url]http://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__INITIALIZE.html#axzz4igCgDbac[/url]

  1. The only effect of that typo is that an array is declared that need not be declared. If you were correct, it would not compile and, believe me, I would have noticed that. Nevertheless, I have corrected the posted code. Thank you for pointing it out.
1 Like
  1. Sorry, I should have preceded all that with ‘this is probably a dumb question but…’. I’m new to this particular toolkit. For example I didn’t realize there was a difference between the CUDA toolkit and the DECODER SDK. I’m working from the sample decoder SDK programs provided by NVIDIA which seem to have added a ‘helper’ layer to the API in the form of dynlink_nvcuvid.h. I probably screwed up my install or my paths.
  • Yes I would appreciate a copy of nalu.h.

2, By ‘my libraries’ I just meant my copy of the NVIDIA Decoder SDK and sample code as installed (apparently ineptly) by me.

  1. I was just saying I do actually get a compiler error for an undefined variable (io_buffer) when building the unmodified code. I believe your code worked for you - its an interesting mystery as to why I get an error from the compiler on that same code. I was not trying to be critical or whiney.

P.S. I actually do appreciate you taking time to help someone who is a total NVIDIA noob. I will try to do my homework and only ask intelligent, reasonable questions in the future after making an honest effort to figure things out myself.

P.P.S. I do actually have a life - 5 kids, 2 grandchildren and am headed out to Utah tomorrow for a week of whitewater kayaking on the Yampa River. I honestly hope you have an equally great life! :-)

nVidia provides these different views but either way the basic concepts work the same. If you want to run the posted code, then set your paths to the CUDA toolset include and library directories. It could be ported to the Video SDK headers; you could do that. CUDA itself seems somehow more fundamental to me.

Send me an email at donald.graft at cantab.net and I will send you what you need. NALU parsing is an additional complication beyond the subject of this thread.

The Video SDK does not provide any CUDA libraries. You have to link cuda.lib and nvcuvid.lib from the CUDA toolset.

Probably you have Visual Studio configured to treat warnings as errors. You would have gotten a warning that the array was declared but not used. You can revise it and move on. As I said, it is inconsequential.

1 Like

hello, electrodynamics:

I want use NVDecoder to decode h264 stream one by one. I use ffmpeg to read h264 data frame which can replace video source reading data from h264 file.but there is a question:
At the first time I pass frame data to  "cuvidParseVideoData" function, the decode callback HandlePictureDecode function not be called.At the seconde time, pass frame to it, the HandlePictureDecode callback will been executed.So I suspect that the NvDecoder will decode the first frame, at the second calling   "cuvidParseVideoData" function. It will has a frame decoded delay time.But I want to directly decode current frame sync, How can I do?
Thanks for replying.

See my response in the other thread you opened for this.

1 Like

Hello, electrodynamics:
I am using your “cuviddec decode sample”. Could you please provide me with the nalu.h? my email is 780540759@qq.com.
Thank you very much!

The NALU code is available here: http://rationalqm.us/misc/nalu.rar

1 Like

Hello, electrodynamics:
Thank you very much for your nalu.I have a question, If i want to decode h265 data,can i use this project?

Yes, for 8-bit HEVC just use this when creating the parser:

parserInitParams.CodecType = cudaVideoCodec_HEVC;

You’ll have to modify things if your source is 10/12 bit and you want to deliver the same depth to the output.

1 Like