Issues with MAX96716A deserializer + 4‑lane GMSL: clock / pix_clk / serdes_pix_clk mismatch

Robotics & Edge Computing Jetson Systems Jetson Orin NX
,
1

Hi, I’m using a 4-lane GMSL camera with MAX96716A deserializer. When running at 1500 enough for 1 camera but not for 2, but when increasing to 1900 Mbit/lane, image is corrupted.
Device tree:

pix_clk_hz = "360000000";
serdes_pix_clk_hz = "600000000";

/sys/kernel/debug/bpmp/debug/clk/nvcsi/rate shows 98907692.
If I manually increase it just by 1 Hz (98907693), the image is perfect.

Changing the clocks in the device tree doesn’t affect the rate, it stays at 98907692.

Has anyone seen this rounding / clock issue? Is there a way to set the nvcsi clock higher permanently?

echo 98907692 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

image
image810×505 141 KB

echo 98907693 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

image
image817×496 66.1 KB

r36.4.3
orin nx 16gb

3

*** Please note that this reply is generated by LLM automatically ***

The issue you’re experiencing with the 4-lane GMSL camera and MAX96716A deserializer on the Jetson Orin NX seems to be related to the nvcsi clock rate. You’ve observed that when the clock rate is set to 98907692 Hz, the image is corrupted, but when you manually increase it by 1 Hz to 98907693 Hz, the image becomes perfect.

This behavior suggests that there might be a rounding issue with the clock rate, which is causing the corruption. The fact that changing the clocks in the device tree doesn’t affect the rate, and it stays at 98907692 Hz, indicates that there might be a limitation or a constraint in the system that’s preventing the clock rate from being set to the desired value.

To address this issue, you can try the following:

  1. Verify the device tree configuration: Double-check the device tree configuration to ensure that the clock rates are set correctly. Make sure that the nvcsi clock rate is set to the desired value, and that there are no other clock rates that might be affecting the system.
  2. Use the bpmpdebug tool: The bpmpdebug tool can be used to set the clock rates manually. You can try setting the nvcsi clock rate to 98907693 Hz using this tool to see if it resolves the issue.
  3. Check the clock rate limitations: Investigate the clock rate limitations of the Jetson Orin NX and the MAX96716A deserializer. It’s possible that there are limitations or constraints that are preventing the clock rate from being set to the desired value.
  4. Contact NVIDIA support: If none of the above steps resolve the issue, you may want to contact NVIDIA support for further assistance. They may be able to provide more information on the clock rate limitations and help you resolve the issue.

In comparison to other devices, the Jetson Orin NX has a unique clocking system, and the nvcsi clock rate is critical for the proper functioning of the camera. The MAX96716A deserializer also has its own clocking requirements, and it’s possible that there are specific clock rate requirements for this device that need to be met.

In terms of the camera API matrix, the Jetson Orin NX supports the libargus and GStreamer (GST-nvarguscamerasrc) APIs for camera access. However, the issue you’re experiencing seems to be related to the clock rate and not the camera API.

To summarize, the issue you’re experiencing with the 4-lane GMSL camera and MAX96716A deserializer on the Jetson Orin NX seems to be related to the nvcsi clock rate. You can try verifying the device tree configuration, using the bpmpdebug tool, checking the clock rate limitations, and contacting NVIDIA support for further assistance.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

5

hello daniel.banar,

could you please double check developer guide for the Sensor Pixel Clock section to evaluate the pixel clock rate,
FYI, I usually using sensor CSI lane output rate to calculate the clock rate,
for instance, pixel_clk_hz = sensor data rate per lane (Mbps) * number of lanes / bits per pixel

you may configure the kernel driver to set the clock rate during system boot-up.
for instance,
$public_sources/kernel_src/kernel/nvidia-oot/drivers/video/tegra/camera/tegra_camera_platform.c

diff --git a/drivers/video/tegra/camera/tegra_camera_platform.c 

@@ -1091,15 +1091,18 @@ static int calculate_and_set_device_clock(struct tegra_camera_info *info,
 		dr = bus_width * 100;
 		if (dr == 0)
 			return -EINVAL;
+		clk_rate = 400000000;
 		break;
 	case HWTYPE_VI:
 		nr = final_pr * overhead;
 		dr = 100 * (ppc / ppc_divider);
+		clk_rate = 998400000;
 		break;
 	case HWTYPE_ISPA:
 	case HWTYPE_ISPB:
 		nr = final_pr * overhead;
 		dr = 100 * ppc;
+		clk_rate = (nr + dr) / dr;
 		break;
 	case HWTYPE_SLVSEC:
 		if (info->sensor_type != SENSORTYPE_SLVSEC)
@@ -1108,13 +1111,14 @@ static int calculate_and_set_device_clock(struct tegra_camera_info *info,
 		dr = bus_width * 100;
 		if (dr == 0)
 			return -EINVAL;
+		clk_rate = (nr + dr) / dr;
 		break;
 	default:
 		return -EINVAL;
 	}
 
 	/* avoid rounding errors by adding dr to nr */
-	clk_rate = (nr + dr) / dr;
+	//clk_rate = (nr + dr) / dr;
6

Fixed it by increasing serdes_pix_clk_hz = “760000000”;

However when I switched to quad channel deserializer MAX96724 with 4 cameras runnning If i set it to anything higher then serdes_pix_clk_hz = "600000000"; the streams wont even start


gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! “video/x-raw(memory:NVMM), width=1920, height=1200, format=NV12, framerate=60/1” ! nvvidconv ! nvv4l2h264enc bitrate=5000000 maxperf-enable=true ! h264parse ! rtph264pay config-interval=1 ! udpsink host=192.168.55.100 port=2221 sync=false async=false
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Redistribute latency…
NvMMLiteOpen : Block : BlockType = 4
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1200 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.001152, max 15.995580; Exposure Range min 28000, max 83333000;

GST_ARGUS: Running with following settings:
Camera index = 1
Camera mode  = 0
Output Stream W = 1920 H = 1200
seconds to Run    = 0
Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadExecute:694 NvBufSurfaceFromFd Failed.
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:247 (propagating)
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: CANCELLED
Additional debug info:
Argus Error Status
Execution ended after 0:00:02.711798080
Setting pipeline to NULL …
GST_ARGUS: Cleaning up
Freeing pipeline …

cat /sys/kernel/debug/tracing/trace gives me bunch of these errors

     kworker/3:1-97      [003] ....   211.136839: rtcpu_nvcsi_intr: tstamp:7175974301 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
     kworker/3:1-97      [003] ....   211.136839: rtcpu_nvcsi_intr: tstamp:7175974301 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
     kworker/3:1-97      [003] ....   211.136839: rtcpu_nvcsi_intr: tstamp:7175974301 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:0 st:0 vc:0 status:0x00000e00
     kworker/3:1-97      [003] ....   211.136839: rtcpu_nvcsi_intr: tstamp:7175974301 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:1 st:0 vc:0 status:0x00000600
     kworker/3:1-97      [003] ....   211.192853: rtcpu_nvcsi_intr: tstamp:7176996396 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
     kworker/3:1-97      [003] ....   211.192855: rtcpu_nvcsi_intr: tstamp:7176996396 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
     kworker/3:1-97      [003] ....   211.192855: rtcpu_nvcsi_intr: tstamp:7176996396 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:0 st:0 vc:0 status:0x00000e00
     kworker/3:1-97      [003] ....   211.192856: rtcpu_nvcsi_intr: tstamp:7176996396 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:1 st:0 vc:0 status:0x00000600

 kworker/3:1-97      [003] ....   211.416849: rtcpu_isp_falcon_task_start: tstamp:2889659681 ch:0 task:HANDLE_EVENT
 kworker/3:1-97      [003] ....   211.416853: rtcpu_isp_falcon_task_end: tstamp:2889659722 task:HANDLE_EVENT

But again at 1500Mbit/lane @ 4lanes is okay, 1700 and 1900 work fine with the increased serdes_pix_clk_hz on dual channel MAX96716A but not on quad MAX96724

Sometimes I can get frames using v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1000 but only when I start the stream and reenable the csi ouput register of the deserialiser

7

hello daniel.banar,

as mentioned by previous post.
could you please double check Sensor Pixel Clock section, using sensor CSI lane output rate to evaluate clock rate,
for instance, pixel_clk_hz = sensor data rate per lane (Mbps) * number of lanes / bits per pixel

8

Yes the cameras itself uses 900Mbit/s, pix_clk_hz = “360000000” is correct and works, standalone and with dual channel deserialiser, but not 4 channel, while the deserializer i have configured for 1900Mbit/s which corresponds to serdes_pix_clk_hz = “760000000”; But this doesnt work for the quad deserialiser which i have to manually increase the csi clock to allow anything above 1500Mbit/s

9

hello daniel.banar,

may I know the sensor resolution and the output frame-rate?
please give it a try to lower the frame-rate for testing?
besides.. may I also know which CSI brick you’re used, is it possible to give it a try with other CSI brick?

10

Sure its a 1920x1200, but i noticed a strange anomaly with framerate using v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=100 it reports 60fps, but when i start a gstreamer stream gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! “video/x-raw(memory:NVMM), width=1920, height=1200, format=NV12, framerate=60/1” ! nvvidconv ! nvv4l2h264enc bitrate=5000000 maxperf-enable=true ! h264parse ! rtph264pay config-interval=1 ! udpsink host=192.168.55.100 port=2222 sync=false async=false Im only getting 30fps.

The deserialiser is connected to CSI0/1 port and i cant use any other port

11

hello daniel.banar,

you may also test with display disabled to exclude other factors for checking sensor output frame-rate.
for instance,
$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1200, framerate=60/1, format=NV12' ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_0 video-sink=fakesink sync=0 -v

12

Hi jerry, when i run it with serdes_pix_clk_hz = “600000000”; im getting like 22fps, after setting the maxrates for nvcsi im getting 30fps, but this framerate issue im not concerned about right now, its probably just wrong register map for this camera

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1200 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.001152, max 15.995580; Exposure Range min 28000, max 83333000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 1920 H = 1200 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 17, dropped: 0, current: 32.56, average: 32.56
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 33, dropped: 0, current: 30.57, average: 31.56
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 49, dropped: 0, current: 30.58, average: 31.23
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 65, dropped: 0, current: 30.57, average: 31.07
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 81, dropped: 0, current: 30.58, average: 30.97
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 97, dropped: 0, current: 30.57, average: 30.90
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 113, dropped: 0, current: 30.58, average: 30.86
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 129, dropped: 0, current: 30.58, average: 30.82
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 145, dropped: 0, current: 30.57, average: 30.79
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 161, dropped: 0, current: 30.57, average: 30.77
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 177, dropped: 0, current: 30.58, average: 30.75
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 193, dropped: 0, current: 30.57, average: 30.74
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 209, dropped: 0, current: 30.58, average: 30.73
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 225, dropped: 0, current: 30.57, average: 30.72
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_0: last-message = rendered: 241, dropped: 0, current: 30.57, average: 30.71
13

hello daniel.banar,

actually.. you should revise that. the frame-rate should be consistency if you’ve correct config.

14

I just tried using 4xIMX219 cameras and its the same issue - serdes_pix_clk_hz = “600000000”; video errors, after manually boosting clocks works fine,
but serdes_pix_clk_hz = “760000000”; the streams wont even start.

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 3280 H = 2464 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadExecute:694 NvBufSurfaceFromFd Failed.
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:247 (propagating)
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: CANCELLED
Additional debug info:
Argus Error Status
Execution ended after 0:00:02.662886336
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
Freeing pipeline ...
15

hello daniel.banar,

you may refer to SerDes Pixel Clock section to review your clock rate settings.

16

Hi yes, this should be correct serdes_pix_clk_hz = “760000000”;

Im setting the MAX96724 to 1900 Mbps/lane.

// This is to set predefined (coarse) CSI output frequency
// CSI Phy 1 is 1900 Mbps/lane.
0x04,0x9C,0x1D,0x00,0xF4, // (Default) 
0x04,0x9C,0x04,0x18,0x33,

serdes_pix_clk_hz = (deserializer output data rate in hertz) * (number of CSI lanes) / (bits per pixel).

serdes_pix_clk_hz = (1900000000) * (4) / (10) = 760000000
But it doesnt work i have to set it to 600000000 which corresponds to 1500Mbps/lane
The Image of the cameras is broken but after maxing the nvcsi clock.

cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

The system works fine.

17

hello daniel.banar,

may I know the test results by setting serdes_pix_clk_hz = “760000000”; and also maxing the nvcsi clock?

18

Maxing the clocks or not doesnt make a difference
nvarguscam just fails with

GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadExecute:694 NvBufSurfaceFromFd Failed.
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:247 (propagating)
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: CANCELLED
Additional debug info:
Argus Error Status
Execution ended after 0:00:02.662886336
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
Freeing pipeline ...
19

hello daniel.banar,

you may see-also Topic 326038 to apply camera firmware with debug flag enabled for gathering more details.
BTW, although Topic326038_Mar26.7z it’s the debug version rce-fw for JP6.2/r36.4.3, it may also applies to r36.4.4 public release version for debugging.

21

Im sorry I realized Im not using R36.4.3 but R35.4.1
and i still havent solved the problem

22

hello daniel.banar,

there’re some known camera bugs,
for instance,
Topic 268833: JP-5.1.2 camera firmware to update deskew algorithm, and also stability fixes.
Topic 276217: Argus fixes for set_mode_delay_ms and infinite timeout support
Topic 309780: RCE General error queue is out of sync with frame queue.

or..
is it possible for moving to the latest JP-5 release version for verification?

23

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.