Unable to retrieve SEI message from NVDEC bitstream (H264)

Gaming and Visualization Technologies Visualization Video Processing & Optical Flow
#1

Hello,

I need to retrieve SEI (USER DATA UNREGISTERED) messages from nvcuvid bitstream in the function

handlePictureDecode(CUVIDPICPARAMS *pPicParams)

I have setup a video streaming using a combination of NVENC/NVDEC and live555. This works very well except that I need to add an SEI per frame. The problem is that before giving NALUS to the decoder, I can actually parse them and retrieve my SEI messages (getting nal type by masking with 0x1F). But as soon as I give them to my decoder with cuvidParseVideoData, I am unable to retrieve them in the bitstream (pPicParams->pBitstreamData, pPicParams->nBitstreamDataLen)

Code for feeding nal to NVCUVID is :

void H264Decoder::processNewNalu(const QByteArray& nalu_h264)
{
        CUVIDSOURCEDATAPACKET packet;
	memset(&packet, 0, sizeof(CUVIDSOURCEDATAPACKET));
	packet.payload = (unsigned char*)nalu_h264.data();
	packet.payload_size = nalu_h264.size();

        __cucheck(cuCtxPushCurrent(m_cuContext));

	__cucheck(cuvidParseVideoData(m_cuVideoParser, &packet));

	__cucheck(cuCtxPopCurrent(NULL));

}

Where m_cuContext is the cuda Context created for my decoder and m_cuVideoParser is the parser created with cuvidCreateVideoParser. Code for creating video parser is :

void H264Decoder::createVideoParser()
{
	CUVIDPARSERPARAMS oVideoParserParameters;
	memset(&oVideoParserParameters, 0, sizeof(CUVIDPARSERPARAMS));
	oVideoParserParameters.CodecType = cudaVideoCodec_H264;
	oVideoParserParameters.ulMaxNumDecodeSurfaces = 1;
	oVideoParserParameters.ulMaxDisplayDelay = 0;
	
	oVideoParserParameters.pUserData = this;
	oVideoParserParameters.pfnSequenceCallback = handleVideoSequence;
	oVideoParserParameters.pfnDecodePicture = handlePictureDecode;
	oVideoParserParameters.pfnDisplayPicture = handlePictureDisplay;

	__cucheck(cuCtxPushCurrent(m_cuContext));
	__cucheck(cuvidCreateVideoParser(&m_cuVideoParser, &oVideoParserParameters));
	__cucheck(cuCtxPopCurrent(NULL));

	return true;
}

code for parsing bitstream is :

void H264Decoder::parseBitstream(const unsigned char* bitstream, unsigned int size)
{
	unsigned int offset = 0;
	while (offset < size - 4)
	{
		if (bitstream[offset] == 0x00 && bitstream[offset + 1] == 0x00 && bitstream[offset + 2] == 0x01)
		{
			printf("Header 0X00 0X00 0X01 found ! Nal Type : %d\n", bitstream[offset + 3] & 0x1F);
			offset += 3;
		}
		else if (bitstream[offset] == 0x00 && bitstream[offset + 1] == 0x00 && bitstream[offset + 2] == 0x00 && bitstream[offset + 3] == 0x01)
		{
			printf("Header 0X00 0X00 0x00 0X01 found ! Nal Type : %d\n", bitstream[offset + 4] & 0x1F);
			offset += 4;
		}
		else
			offset++;
	}
}

I only found nalus with type 1 in the bitstream (pPicParams->pBitstreamData, pPicParams->nBitstreamDataLen). I would like to find my SEI (nal type 6) in the bitstream because I actually need to output the correct SEI with the associated frame.

I use Video_Codec_SDK_9.1.23, Geforce GTX 1080, Intel® Xeon® CPU E5-1650v4 3.60 GHz, Windows 10 professional version 1703.
Current Cuda version 10.1, Nvidia Driver version 436.48
Thank you very much