Video convert to gray scale with gst for visual odometry causing crash

Hello,
I wish to run visual odometry app with custom hardware, I successfully implemented a pipeline to retrieve image from my camera and split it.
I can convert the video feed to RGB with gst without issues but when I try to convert it to gray scale the application terminate unexpectedly. Could you try to spot my error please.

This is the line working :
"pipeline": "gst-launch-1.0 -v v4l2src device=/dev/video0 ! avdec_mjpeg ! videoconvert ! video/x-raw,format=RGB,height=480,framerate=30/1 ! appsink name = acquired_image "

This is what I want to do :
"pipeline": "gst-launch-1.0 -v v4l2src device=/dev/video0 ! avdec_mjpeg ! videoconvert ! video/x-raw,format=GRAY8,height=480,framerate=30/1 ! appsink name = acquired_image "

Thank you

This is the successful app :

{
  "name": "odov1",
  "modules": [
    "deepstream",
    "perception",
    "sight",
    "viewers"
  ],
  "graph": {
    "nodes": [
      {
        "name": "acquisition",
        "components": [
          {
            "name": "message_ledger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "pipeline",
            "type": "isaac::deepstream::Pipeline"
          }
        ]
      },
      {
        "name": "compute",
        "components": [
          {
            "name": "message_ledger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "operation1",
            "type": "isaac::perception::CropAndDownsampleCuda"
          },
	  {
            "name": "operation2",
            "type": "isaac::perception::CropAndDownsampleCuda"
          }
        ]
      },
      {
        "name": "publishing",
        "components": [
          {
            "name": "message_ledger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "viewer1",
            "type": "isaac::viewers::ColorCameraViewer"
          },
          {
            "name": "viewer2",
            "type": "isaac::viewers::ColorCameraViewer"
          },
	 {
            "name": "viewer3",
            "type": "isaac::viewers::ColorCameraViewer"
          },
          {
            "name": "viewer1_widget",
            "type": "isaac::sight::SightWidget"
          },
          {
            "name": "viewer2_widget",
            "type": "isaac::sight::SightWidget"
          },
          {
            "name": "viewer3_widget",
            "type": "isaac::sight::SightWidget"
          }	  
        ]
      }
    ],
    "edges": [
      {
        "source": "acquisition/pipeline/acquired_image",
        "target": "compute/operation1/input_image"
      },
      {
        "source": "acquisition/pipeline/acquired_image",
        "target": "compute/operation2/input_image"
      },
      {
        "source": "acquisition/pipeline/acquired_image",
        "target": "publishing/viewer3/color_listener"
      },
      {
        "source": "compute/operation1/output_image",
        "target": "publishing/viewer1/color_listener"
      },
      {
       "source": "compute/operation2/output_image",
        "target": "publishing/viewer2/color_listener"
      }
    ]
  },
  "config": {
    "acquisition": {
      "pipeline": {
        "pipeline": "gst-launch-1.0 -v v4l2src device=/dev/video0  ! avdec_mjpeg ! videoconvert ! video/x-raw,format=RGB,height=480,framerate=30/1 ! appsink name = acquired_image "
       }
      },
      "compute": {
      "operation1": {
        "crop_start": [0, 0],
        "crop_size": [480, 640]
       },
      "operation2": {
        "crop_start": [0, 640],
        "crop_size": [480, 640]
       }
      },
      "publishing": {
      "viewer1_widget": {
        "title": "Viewer: Video Left",
        "type": "2d",
        "channels": [
          {
            "name": "publishing/viewer1/Color"
          }
        ]
      },
      "viewer2_widget": {
        "title": "Viewer: Video Right",
        "type": "2d",
        "channels": [
          {
            "name": "publishing/viewer2/Color"
          }
        ]
      },
       "viewer3_widget": {
        "title": "Viewer: Acquis ",
        "type": "2d",
        "channels": [
          {
            "name": "publishing/viewer3/Color"
          }
        ]
      }
    }
  }
}

I saw two other ways to do it but I don’t know what to do :
using Dali
using CUDA

Are you trying to use deepstream codelet to output grayscale (instead of rgb)? Is this where you crashing? before VIO?

deepstream codelet current do not support grayscale

1 Like

Thank you @atorabi
Yes, I’m using deepstream gstreamer and it’s not crashing anymore because I specified the video wight but it is very frustrating because I can get an output but it is unusable due to what seems like a synchronization issue.
I get a static image composed of many images, screenshot

same with GRAY16_XL

Hello @atorabi
Do you think that gray scale conversion with deepstream and Gstreamer will be supported in the next updates of isaac ?
Do you think that I could make it work by interfacing Dali or CUDA with isaac ?
Thanks a ton for your support