Having problems display gray scale images from MIPI camera with Jetson-util

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
1

I am trying to setup a pipeline with the mipi camera and use CUDA code to do image processing. While I can get the camera to produce a color image, I can’t seem to get it to render a gray scale image. I am having problems with the conversion algorithm not being able to handle uint8 arrays.

I have tried several versions with no luck. I modified the basic camera render and added the colorconversion code as a final attempt. (My stand alone file input/output works fine).

Here is my example code:

#include <jetson-utils/videoSource.h>
#include <jetson-utils/videoOutput.h>
#include <jetson-utils/cudaColorspace.h>
#include <jetson-utils/cudaMappedMemory.h>
#include <jetson-utils/imageIO.h>

int main( int argc, char** argv )
{
    // create input/output streams
    videoSource* input = videoSource::Create(argc, argv, ARG_POSITION(0));
    videoOutput* output = videoOutput::Create(argc, argv, ARG_POSITION(1));
   

    if( !input )
        return 0;

    // capture/display loop
    while( true )
    {
        uint8_t* imgOutput = NULL;  // output is gray8 (uint8)
        uint8_t* image = NULL;  // can be uchar3, uchar4, float3, float4
        int status = 0;        // see videoSource::Status (OK, TIMEOUT, EOS, ERROR)
		int width = 0;
        int height = 0;

        if( !input->Capture(&image, 1000, &status) )  // 1000ms timeout (default)
        {
            if( status == videoSource::TIMEOUT )
                continue;
				
            break; // EOS
        }

        width = input->GetWidth();
        height = input->GetHeight();

        // allocate the output as gray8 (uint8), with the same width/height
        if( !cudaAllocMapped(&imgOutput, width, height,IMAGE_GRAY8) )
	        return false;

        // convert from rgb8 to gray8
        if( CUDA_FAILED(cudaConvertColor(image, IMAGE_RGB8, imgOutput, IMAGE_GRAY8, width, height)) )
	        return false;	// an error or unsupported conversion occurred

        if( output != NULL )
        {
            output->Render(imgOutput, input->GetWidth(), input->GetHeight(),IMAGE_GRAY8);

            if( !output->IsStreaming() )  // check if the user quit
                break;
        }
    }

    // destroy resources
    SAFE_DELETE(input);
    SAFE_DELETE(output);
}
3

Hi,

Based on the source below, the conversion from RGB8 to GRAY8 should work.
Could you share the error and console log with us?

Thanks.

4

I keep getting an error in invalid image format type:

$ nvcc -o jetsoncamera jetsoncamera.cu -I/usr/local/include/jetson-utils -I/usr/local/cuda/include -L/usr/local/lib -ljetson-utils -lopencv_core -lopencv_highgui -lopencv_videoio -lopencv_imgproc -lopencv_imgcodecs -lcuda -lcudart
/usr/local/include/jetson-utils/imageFormat.inl(234): error: static assertion failed with “invalid image format type - supported types are uchar3, uchar4, float3, float4”
static_assert(__image_format_assert_false::value, “invalid image format type - supported types are uchar3, uchar4, float3, float4”);
^
detected during:
instantiation of “imageFormat imageFormatFromType() [with T=uint8_t]” at line 242 of /usr/local/include/jetson-utils/videoSource.h
instantiation of “__nv_bool videoSource::Capture(T **, uint64_t, int *, cudaStream_t) [with T=uint8_t]” at line 25 of jetsoncamera.cu

1 error detected in the compilation of “jetsoncamera.cu”.

5

Hi,

We will give it a try and share more info with you.
Thanks.

6

AsstaLL, did you try to compile the routine? I have my class using file oriented I/O since I could not get the camera to display in greyscale.
Nick

7

Hi,

Sorry that doesn’t have time to try yet.

But just confirm that you got stuck in the building step before testing, right?
If yes, could you share the building command/Makefile file with us as well?

Thanks.

8

Here is the Makefile for the routine that uses the jetson camera and converts it to gray scale image, displaying it on the monitor:

Paths to CUDA and Jetson Utils

CUDA_DIR := /usr/local/cuda
JETSON_UTILS_DIR := /usr/local/include/jetson-utils

Compiler and linker

CXX := $(CUDA_DIR)/bin/nvcc
CXXFLAGS := -O3 -I$(CUDA_DIR)/include -I$(JETSON_UTILS_DIR)
LDFLAGS := -L$(CUDA_DIR)/lib64 -lcudart -L$(JETSON_UTILS_DIR)/lib -ljetson-utils

Target executable

TARGET := jetsoncamera
SRC := jetsoncamera.cu

Build rules

all: $(TARGET)

$(TARGET): $(SRC)
$(CXX) $(CXXFLAGS) $(SRC) -o $(TARGET) $(LDFLAGS)

clean:
rm -f $(TARGET)

Here is the error message I get when compiling:
$ make
/usr/local/cuda/bin/nvcc -O3 -I/usr/local/cuda/include -I/usr/local/include/jetson-utils jetsoncamera.cu -o jetsoncamera -L/usr/local/cuda/lib64 -lcudart -L/usr/local/include/jetson-utils/lib -ljetson-utils
/usr/local/include/jetson-utils/imageFormat.inl(234): error: static assertion failed with “invalid image format type - supported types are uchar3, uchar4, float3, float4”
static_assert(__image_format_assert_false::value, “invalid image format type - supported types are uchar3, uchar4, float3, float4”);
^
detected during:
instantiation of “imageFormat imageFormatFromType() [with T=uint8_t]” at line 242 of /usr/local/include/jetson-utils/videoSource.h
instantiation of “__nv_bool videoSource::Capture(T **, uint64_t, int *, cudaStream_t) [with T=uint8_t]” at line 25 of jetsoncamera.cu

1 error detected in the compilation of “jetsoncamera.cu”.
make: *** [Makefile:18: jetsoncamera] Error 1

10

Hi,

We recently got a similar issue and the user has a fix to make it work.
Could you check if the fix also solves your use case as well?

Thanks.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.