video encode error

I was trying to encode video stream from onboard camera but failed.

The errors are:
[gstreamer] gstreamer mysource ERROR Internal data flow error.
[gstreamer] gstreamer Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstAppSrc:mysource:
streaming task paused, reason not-negotiated (-4)

I use the codes from one of the developers working in nvidia:https://github.com/dusty-nv/jetson-utils.

Here is my main.cpp:

#include "gstCamera.h"
#include "cudaNormalize.h"
#include "gstUtility.h"
#include "gstEncoder.h"


int main(int argc, char *argv[])
{

	gst_init(&argc, &argv);

	gstEncoder *enc = gstEncoder::Create(GST_CODEC_H265,1280,720,"1.h265");

	gstCamera* camera = gstCamera::Create(-1);


	if( !camera )
	{
		printf("\ngst-camera:  failed to initialize video device\n");
		return 0;
	}

	printf("\ngst-camera:  successfully initialized video device\n");
	printf("    width:  %u\n", camera->GetWidth());
	printf("   height:  %u\n", camera->GetHeight());
	printf("    depth:  %u (bpp)\n", camera->GetPixelDepth());

	if( !camera->Open() )
	{
		printf("\ngst-camera:  failed to open camera for streaming\n");
		return 0;
	}

	printf("\ngst-camera:  camera open for streaming\n");

	int num = 0;
	const size_t texSz = camera->GetWidth() * camera->GetHeight() * 3 / 2;


		while( 1 )
		{
			num++;
			if(num==5)
				break;

			void* imgCPU  = NULL;
			void* imgCUDA = NULL;

			// get the latest frame
			if( !camera->Capture(&imgCPU, &imgCUDA, 1000) )
				printf("\ngst-camera:  failed to capture frame\n");
			else
				printf("gst-camera:  recieved new frame  CPU=0x%p  GPU=0x%p\n", imgCPU, imgCUDA);

			// convert from YUV to RGBA
//			void* imgRGBA = NULL;

//			if( !camera->ConvertRGBA(imgCUDA, &imgRGBA, true) )
//				printf("gst-camera:  failed to convert from NV12 to RGBA\n");


			enc->EncodeFrame(imgCPU,texSz);//This function cannt work.





		}
		printf("\ngst-camera:  un-initializing video device\n");


		camera->Close();
		delete camera;
		delete enc;

	return 0;
}

[gstreamer] gstEncoder - pipeline launch string:
appsrc name=mysource ! omxh265enc ! video/x-h265 ! filesink location=1.h265
[gstreamer] gstEncoder - buffer caps string:
video/x-raw-yuv,width=1280,height=720,format=(string)I420,framerate=30/1

(gstCamera:19418): GStreamer-WARNING **: 0.10-style raw video caps are being created. Should be video/x-raw,format=(string)… now.
[gstreamer] gstEncoder - transitioning pipeline to GST_STATE_PLAYING
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and Mjstreaming[gstreamer] initialized gstreamer, version 1.8.3.0
[gstreamer] gstreamer decoder pipeline string:
nvcamerasrc fpsRange=“30.0 30.0” ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420 ! nvvidconv flip-method=2 ! video/x-raw ! appsink name=mysink
[gstreamer] gstEncoder - AppSrc requesting data (4096 bytes)

gst-camera: successfully initialized video device
width: 1280
height: 720
depth: 12 (bpp)
[gstreamer] gstreamer transitioning pipeline to GST_STATE_PLAYING

Available Sensor modes :
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter2
[gstreamer] gstreamer changed state from NULL to READY ==> nvvconv0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1
[gstreamer] gstreamer changed state from NULL to READY ==> nvcamerasrc0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline1
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter2
[gstreamer] gstreamer changed state from READY to PAUSED ==> nvvconv0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> nvcamerasrc0
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline1
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer msg new-clock ==> pipeline1
[gstreamer] gstreamer msg stream-start ==> pipeline1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter2
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> nvvconv0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> nvcamerasrc0

NvCameraSrc: Trying To Set Default Camera Resolution. Selected 1280x720 FrameRate = 30.000000 …

gst-camera: camera open for streaming
[gstreamer] gstreamer decoder onPreroll
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102800000 GPU 0x102800000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102a00000 GPU 0x102a00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102c00000 GPU 0x102c00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102e00000 GPU 0x102e00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103000000 GPU 0x103000000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103200000 GPU 0x103200000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103400000 GPU 0x103400000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103600000 GPU 0x103600000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103800000 GPU 0x103800000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103a00000 GPU 0x103a00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103c00000 GPU 0x103c00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103e00000 GPU 0x103e00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104000000 GPU 0x104000000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104200000 GPU 0x104200000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104400000 GPU 0x104400000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104600000 GPU 0x104600000
[cuda] gstreamer camera – allocated 16 ringbuffers, 1382400 bytes each
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer msg async-done ==> pipeline1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline1
gst-camera: recieved new frame CPU=0x0x104000000 GPU=0x0x104000000
[gstreamer] gstreamer changed state from NULL to READY ==> filesink0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[gstreamer] gstreamer changed state from NULL to READY ==> omxh265enc-omxh265enc0
[gstreamer] gstreamer changed state from NULL to READY ==> mysource
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[gstreamer] gstreamer changed state from READY to PAUSED ==> omxh265enc-omxh265enc0
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysource
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer msg new-clock ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> omxh265enc-omxh265enc0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysource
gst-camera: recieved new frame CPU=0x0x103c00000 GPU=0x0x103c00000
[gstreamer] gstreamer mysource ERROR Internal data flow error.
[gstreamer] gstreamer Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstAppSrc:mysource:
streaming task paused, reason not-negotiated (-4)

[gstreamer] gstreamer msg stream-start ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> filesink0
gst-camera: recieved new frame CPU=0x0x103a00000 GPU=0x0x103a00000
[gstreamer] gstEncoder - AppSrc signalling enough data
gst-camera: recieved new frame CPU=0x0x103600000 GPU=0x0x103600000
[gstreamer] gstEncoder - pipeline full, skipping frame (1382400 bytes)

gst-camera: un-initializing video device
[gstreamer] gstreamer transitioning pipeline to GST_STATE_NULL
[gstreamer] gstEncoder - shutting down pipeline, sending EOS
^C

This program can not stop running after “[gstreamer] gstEncoder - shutting down pipeline, sending EOS”, so I have to input control+C to stop it.

Thanks for your reading. I’m looking forward to your reply.

Hi,
There are some posts about gstreamer encoding for your reference:
https://devtalk.nvidia.com/default/topic/1026106/jetson-tx1/encode-opencv-output-frame-buffers-using-gstreamer-appsrc/post/5219225/#5219225
https://devtalk.nvidia.com/default/topic/1015571/jetson-tx2/what-is-maximum-video-encoding-resolution-in-pixels-/post/5253760/#5253760

I change the caps string from:

[gstreamer] gstEncoder - buffer caps string:
video/x-raw-yuv,width=1280,height=720,format=(string)I420,framerate=30/1

to

[gstreamer] gstEncoder - buffer caps string:
video/x-raw,width=1280,height=720,format=(string)I420,framerate=30/1

Now it can encode video and the output file “1.h265” is no longer 0 bytes.

However, this program still cannot stop running after “[gstreamer] gstEncoder - shutting down pipeline, sending EOS”, and the file “1.h265” can not be played by ffplay or vlc player.

[hevc @ 0x7f700092e0] Format hevc detected only with low score of 1, misdetection possible!
[hevc @ 0x7f70000c80] missing picture in access unit
[hevc @ 0x7f70000c80] No start code is found. 0KB sq= 0B f=0/0
[hevc @ 0x7f70000c80] Error splitting the input into NAL units.
[hevc @ 0x7f700092e0] decoding for stream 0 failed
[hevc @ 0x7f700092e0] Could not find codec parameters for stream 0 (Video: hevc, none): unspecified size
Consider increasing the value for the ‘analyzeduration’ and ‘probesize’ options
/home/nvidia/Documents/gstCamera/Debug/1.h265: could not find codec parameters

Now my program can stop normally, but the video file still can not be player. The output from ffplay is the same as my last reply.

Hi,
You can try to run omxh265enc ! matroskamux ! filesink to have mux’ed mkv.

Now I find the way to solve it.
Replace “omxh265enc !” to “omxh265enc ! video/x-h265,stream-format=(string)byte-stream !”.
Thanks for all replys!

Hi 708922276,

Did you use the gstEncoder found in the dev branch of the dusy-nv Jetson inference repo?

I am struggling to get this encoder to run. I get a memory error when trying to create the gstEncoder.

Could you perhaps share your gstencoder.cpp code?

Thanks