Unable to save video using manual video options with jetson inference

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
, ,
1

I am manually specifying the options to save a video stream to an mp4 file. When I rundetectnet and specify an output.mp4 the file will save and I can click on it and view it. When I specify the exact same options manually the video saves but gives me a no playable streams error. Below I’ll include the terminal output from detectnet, the terminal output from my code, and then my actual code where I specify the options and use the render function to save the video. This is the error I’m seeing

[image]  failed to save 1920x1080 image to 'output_4.mp4/0.jpg'
[image]  imageWriter -- failed to save 'output_4.mp4/0.jpg'

Terminal output from detectnet

[gstreamer] nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, framerate=30/1, format=(string)NV12 ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM) ! appsink name=mysink
[gstreamer] gstCamera successfully created device csi://0
[video]  created gstCamera from csi://0
------------------------------------------------
gstCamera video options:
------------------------------------------------
  -- URI: csi://0
     - protocol:  csi
     - location:  0
  -- deviceType: csi
  -- ioType:     input
  -- width:      1280
  -- height:     720
  -- frameRate:  30
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: rotate-180
------------------------------------------------
[gstreamer] gstEncoder -- codec not specified, defaulting to H.264
[gstreamer] gstEncoder -- pipeline launch string:
[gstreamer] appsrc name=mysource is-live=true do-timestamp=true format=3 ! omxh264enc name=encoder bitrate=4000000 ! video/x-h264 ! h264parse ! qtmux ! filesink location=output.mp4 
[video]  created gstEncoder from file:///home/crose72/Documents/GitHub/OperationSquirrel/scratch/object-detection-examples/detectnet/build/output.mp4
------------------------------------------------
gstEncoder video options:
------------------------------------------------
  -- URI: file:///home/crose72/Documents/GitHub/OperationSquirrel/scratch/object-detection-examples/detectnet/build/output.mp4
     - protocol:  file
     - location:  output.mp4
     - extension: mp4
  -- deviceType: file
  -- ioType:     output
  -- codec:      H264
  -- codecType:  omx
  -- frameRate:  30
  -- bitRate:    4000000
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------
[OpenGL] glDisplay -- X screen 0 resolution:  1920x1080
[OpenGL] glDisplay -- X window resolution:    1920x1080
[OpenGL] glDisplay -- display device initialized (1920x1080)
[video]  created glDisplay from display://0
------------------------------------------------
glDisplay video options:
------------------------------------------------
  -- URI: display://0
     - protocol:  display
     - location:  0
  -- deviceType: display
  -- ioType:     output
  -- width:      1920
  -- height:     1080
  -- frameRate:  0
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------

Terminal output from my code

[gstreamer] nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, framerate=30/1, format=(string)NV12 ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM) ! appsink name=mysink
[gstreamer] gstCamera successfully created device csi://0
[video]  created gstCamera from csi://0
------------------------------------------------
gstCamera video options:
------------------------------------------------
  -- URI: csi://0
     - protocol:  csi
     - location:  0
  -- deviceType: csi
  -- ioType:     input
  -- width:      1280
  -- height:     720
  -- frameRate:  30
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: rotate-180
------------------------------------------------
[gstreamer] gstEncoder -- pipeline launch string:
[gstreamer] appsrc name=mysource is-live=true do-timestamp=true format=3 ! omxh264enc name=encoder bitrate=4000000 ! video/x-h264 ! h264parse ! qtmux ! filesink location=video.mp4 
[video]  created gstEncoder from file:///home/crose72/Documents/GitHub/OperationSquirrel/SquirrelDefender/build/video.mp4
------------------------------------------------
gstEncoder video options:
------------------------------------------------
  -- URI: file:///home/crose72/Documents/GitHub/OperationSquirrel/SquirrelDefender/build/video.mp4
     - protocol:  file
     - location:  video.mp4
     - extension: mp4
  -- deviceType: file
  -- ioType:     output
  -- codec:      H264
  -- codecType:  omx
  -- frameRate:  30
  -- bitRate:    4000000
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------
[OpenGL] glDisplay -- X screen 0 resolution:  1920x1080
[OpenGL] glDisplay -- X window resolution:    1920x1080
[OpenGL] glDisplay -- display device initialized (1920x1080)
[video]  created glDisplay from display://0
------------------------------------------------
glDisplay video options:
------------------------------------------------
  -- URI: display://0
     - protocol:  display
     - location:  0
  -- deviceType: display
  -- ioType:     output
  -- width:      1920
  -- height:     1080
  -- frameRate:  30
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------

And then here’s my video options

bool Video::create_output_vid_stream(void)
{
    std::string baseName = "output";
    std::string extension = ".mp4";
    std::string fileName = generate_unique_file_name(baseName, extension);

    videoOptions options;

    options.resource = fileName;
    options.deviceType = videoOptions::DeviceType::DEVICE_FILE;
    options.ioType = videoOptions::IoType::OUTPUT;
    options.codec = videoOptions::Codec::CODEC_H264;
    options.codecType = videoOptions::CodecType::CODEC_OMX;
    options.width = 1280;
    options.height = 720;
    options.frameRate = 30;
    options.zeroCopy = true;

    output_vid_file = videoOutput::Create(options);

    if (!output_vid_file)
    {
        LogError("detectnet:  failed to create output_vid_file stream\n");
        return false;
    }

    return true;
}

And the render function to save the video

bool Video::save_video(void)
{
    // render output_vid_disp to the display
    if (output_vid_file != NULL)
    {
        output_vid_file->Render(image, input->GetWidth(), input->GetHeight());

        // update the status bar
        char str[256];
        sprintf(str, "TensorRT %i.%i.%i | %s | Network %.0f FPS", NV_TENSORRT_MAJOR, NV_TENSORRT_MINOR, NV_TENSORRT_PATCH, precisionTypeToStr(net->GetPrecision()), net->GetNetworkFPS());
        output_vid_file->SetStatus(str);

        // check if the user quit
        if (!output_vid_file->IsStreaming())
        {
            return false;
        }
    }

    return true;
}
3

Hi,

     - location:  0

Based on this, could you check if the output file name is correctly set?

Thanks.

4

@AastaLLL - location: 0 is related to the csi camera location, not the video output. - location: output.mp4 is the location of the video output.

I also just tried these manual options in detectnet and then recompiled to test it, and it works. So here is what I did in detectnet:

    options2.resource = "file:///home/crose72/Documents/GitHub/OperationSquirrel/scratch/object-detection-examples/detectnet/build/output.mp4";
    options2.resource.protocol = "file";
    options2.resource.location = "output.mp4";
    options2.deviceType = videoOptions::DeviceType::DEVICE_FILE;
    options2.ioType = videoOptions::IoType::OUTPUT;
    options2.codec = videoOptions::Codec::CODEC_H264;
    options2.codecType = videoOptions::CodecType::CODEC_OMX;
    options2.width = 1280;
    options2.height = 720;
    options2.frameRate = 30;
    options2.zeroCopy = true;

    videoOutput *output = videoOutput::Create(options2);

and here’s me copying those same manual options that worked in detectnet:

    options.resource = "file:///home/crose72/Documents/GitHub/OperationSquirrel/SquirrelDefender/build/output.mp4";
    options.resource.protocol = "file";
    options.resource.location = "output.mp4";
    options.deviceType = videoOptions::DeviceType::DEVICE_FILE;
    options.ioType = videoOptions::IoType::OUTPUT;
    options.codec = videoOptions::Codec::CODEC_H264;
    options.codecType = videoOptions::CodecType::CODEC_OMX;
    options.width = 1280;
    options.height = 720;
    options.frameRate = 30;
    options.zeroCopy = true;

    output_vid_file = videoOutput::Create(options);

The only difference here is the resource having a different path (both existing)

5

I figured it out! The problem was that the place where I was deleting the input/output/net was not getting called at the end of the program. I’m using a sigint to exit the code when I press ctrl + c, and I forgot to actually call that function to initialize it. This meant that when I exited the main while loop the program just ended, rather than executing SAFE_DELETE(output); and other code I put at the end of my main function. In a word, SAFE_DELETE(output) needs to run to save the video.

6

Ah, yes deleting the videoOutput interface will call videoOutput.Close() on it, which in the case of those backed by GStreamer will send EOS and gracefully stop the GStreamer pipeline - which seems necessary for MP4 files to playback correctly. I had recently read that MKV files do not require such indexing to be seekable, so perhaps those would still play even in the event of a crash. Or I recall the uncontainerized .h264/.h265 files not needing EOS…

1 Like
8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.