Custom MIPI-CSI Camera Integration

kumar81 · March 22, 2018, 4:32pm

Hi,

We are working on a custom camera outputting RGB888 (24 bits) format over MIPI-CSI on 2 lanes connected to Jetson TX1 running 28.1.

While we could make it work over v4l2src plugin, when we try running with nvcamerasrc, it is not working and the nvcamera daemon outputs following error.

tV4LPixelType: pixel type 0x1000 invalid                                      
setActiveBufferMemory: 2                                                        
INPUT: Width 768 Height 576 pixelformat RG10                                    
writeFrameRate: INPUT frameLength:625, frameRate:50.000000                      
writeGain:      INPUT gainCtrl:100 analogGain:1.000000                          
writeExposure:  INPUT coarseTime:615, expTime:0.019680                          
updateOutputSettings:   OUTPUT frameLength:625, frameRate:50.000000             
updateOutputSettings:   OUTPUT analogGain:1.000000                              
updateOutputSettings:   OUTPUT coarseTime:615, expTime:0.019680                 
apply:-----------------------                                                   
NvPclSettingsApply: Reading PCL settings                                        
PowerServiceUtils:calculateReqClock: entered                                    
PowerServiceHw:addRequest: table size: before: 0, after:1                       
        request table for VI 0:                                                 
        req[0]: guID=0, stageID=SensorCapture                                   
        req[0]: inW=768, inH=576, inBpp = 10, fps=50                            
        req[0]: outW=768, outH=576, outBpp=10                                   
        req[0]: out1W=0, out1H=0, out1Bpp=0                                     
        req[0]: out2W=0, out2H=0, out2Bpp=0                                     
        req[0]: clock=32450000, pixelRate=29500000, timeout=900                 
        req[0]: isoBw=40562, timeout=900                                        
        req[0]: non_isoBw=0, timeout=900                                        
PowerServiceHw:addRequest: table size: before: 0, after:1                       
        request table for CSI 0:                                                
        req[0]: guID=0, stageID=SensorCapture                                   
        req[0]: inW=768, inH=576, inBpp = 10, fps=50                            
        req[0]: outW=768, outH=576, outBpp=10                                   
        req[0]: out1W=0, out1H=0, out1Bpp=0                                     
        req[0]: out2W=0, out2H=0, out2Bpp=0                                     
        req[0]: clock=0, pixelRate=29500000, timeout=900                        
        req[0]: isoBw=0, timeout=900                                            
        req[0]: non_isoBw=0, timeout=900                                        
PowerServiceHwVi:setIso: m_bwVal_Iso=40562                                      
PowerServiceHw:setClock: PowerServiceHw[1]: requested_clock_Hz=32450000         
PowerServiceCore:setCameraBw: totalIsoBw=40562                                  
NvPclSettingsUpdate: Sending Updated Settings through PCL                       
NvPclSettingsApply: Applying last settings through PCL                          
apply:+++++++++++++++++++++++                                                   
apply:-----------------------                                                   
NvPclSettingsApply: Reading PCL settings                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0                                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0                                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0                                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0                                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0                                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0                                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0                                                        
SCF: Error NotImplemented: Unsupported sensor type (in src/components/CaptureSet
upEngineImpl.cpp, function chooseGenInstFunc(), line 126)                       
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngineI
mpl.cpp, function doGetInstructions(), line 1798)                               
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function getInstructionList(), line 262)                                   
SCF: Error NotImplemented:  (propagating from src/components/CaptureSetupEngine.
cpp, function setupCC(), line 186)                                              
SCF: Error NotImplemented:  (propagating from src/api/Session.cpp, function capt
ure(), line 811)                                                                
disposing CC 0 Session 0

Further in another mode, we will need to extract only 8 bit green channel of the incoming data 24 bit data and output it as MEDIA_BUS_FMT_Y8_1X8. Is it possible to use the VI for this purpose? If so, how?

Thanks in advance.

ShaneCCC · March 23, 2018, 3:02am

@kumar81
Sorry to tell nvcamerasrc didn’t support RGB888 yet.

kumar81 · March 23, 2018, 7:36am

Thank you ShaneCCC.

Is there a way to leverage the “8-bit raw or DPCM 14-8-14/12-8-12/10-8-10” conversion mentioned in the “Supported CSI to VI Data Formats” of the TX1 manual? We could align our intensity byte to the necessary location from the camera.

And when we run using v4l2src as below, we see occasional break in the video flow. What could be the reason?

gst-launch-1.0 -v v4l2src device=/dev/video0 ! ‘video/x-raw,framerate=(fraction)50/1,format=RGB’ ! videoconvert ! xvimagesink

ShaneCCC · March 23, 2018, 8:55am

What do you mean “we see occasional break in the video flow”

kumar81 · March 23, 2018, 9:16am

Well, there were jumps in the stream. It got solved by running TX1 with maximum performance.

Now we are really interested in extracting only 8 bits of our input stream. Any pointers in that regard will be very helpful.

ShaneCCC · March 26, 2018, 5:14am

@kumar81
Sorry, I don’t understand what your mean? Could you have more detail information.

kumar81 · March 26, 2018, 8:18am

We are receiving MIPI packets in RGB888 format. We would like to know if the VI engine can extract only the Green byte and store it in system memory.

ShaneCCC · March 26, 2018, 8:31am

@kumar81
VI engine didn’t support it. But you can do post processing.

kumar81 · March 26, 2018, 8:45am

Can you provide pointers for the same ShaneCCC?

ShaneCCC · March 26, 2018, 8:50am

You can write gstreamer element to do what you want after you receive the frame data.

kumar81 · March 26, 2018, 9:00am

OK Thank you ShaneCCC, we are hoping to avoid the software manipulation.

BTW I have another question. As mentioned earlier, we are interested only in 8-bit intensity data from our camera. If we can make the camera to output RAW 8-bit MIPI packets, will it possible to get just these WxH bytes over v4l2src? Is this feature available today?

Reason I am asking is that in the vi_video_formats array, the RAW8 inputs are manipulated as Bayer format. But we will send only the Y, and the same has to be stored in memory without any processing.

ShaneCCC · March 26, 2018, 9:18am

@kumar81
It’s could possible but you may need to modify the vi driver to support it. Please check the TRM YUV Pixel Data “YUV Pixel Data Formatting”.

kumar81 · March 27, 2018, 1:04pm

Thank you Shane, we will look in to it.

Topic		Replies	Views
V4l2-ctl works, however, nvarguscamerasrc fails Jetson TX2 camera , device-tree	15	1234	July 19, 2023
Not working custom camera driver Jetson AGX Orin camera , kernel	5	216	July 30, 2024
JetPack 5.10.104 vi_capture_control_message: NULL VI channel received Jetson Xavier NX camera	2	76	July 19, 2024
Mipi-driver : frame start syncpt timeout!0 Jetson Nano camera , kernel , gstreamer	50	3705	October 15, 2021
How to capture 4 chans video data through one csi interface based ov5693_i2c driver on jetson tx1 board Jetson TX1	12	2139	October 18, 2021
Camera driver Jetson Nano camera	21	2105	April 12, 2023
V4L2 Driver Timeout Issues with Xilinx FPGA and Orin Nano DevKit Jetson Orin Nano camera , fpga	21	329	August 1, 2024
Using MIPI to transfer RAW image data Jetson TX2	7	1310	October 18, 2021
Vi error “no reply from camera processor” for custom camera driver Jetson Xavier NX camera , board-design	5	580	July 5, 2022
Problem of getting data from MIPI CSI Jetson TX2	8	1459	October 18, 2021

Custom MIPI-CSI Camera Integration

Related topics