Questions Regarding iToF Camera (MLX75027) Driver Development on Orin NX

[Star_sea]

Hello,

Our team is currently developing a 3D iToF camera based on the Melexis MLX75027 sensor. We plan to connect this camera to the NVIDIA Orin NX development board via a MIPI CSI-2 interface for depth data acquisition, which will be used in our highway vehicle detection system.

I have some experience writing drivers and device tree configurations for traditional image sensors on the Orin NX (e.g., IMX327), and have followed NVIDIA’s camera driver documentation to create a preliminary driver for the MLX75027. So far:

• The sensor is successfully registered as a V4L2 sub-device.
• I2C communication is working correctly.
• A register configuration table has been written, and I’ve attempted to start the stream by setting relevant registers.
  
  

However, the camera has not yet successfully output any data stream, which may be due to the current driver being incomplete. Since we are new to ToF camera driver development, we are seeking guidance and have the following questions:

1. If the ToF camera is able to output RAW12 depth data via the MIPI interface, can the ISP on the Orin NX process this into a depth image?
   If yes, what command-line tools or configurations should be used?
   If not, what is the recommended way to access the RAW data stream and convert it into usable depth information?

2. The MLX75027 sensor uses a 4-phase method to calculate depth, and the datasheet indicates it may output four phase shift values.
   I assume the driver needs to include a function that converts these phase values into depth data. However, based on the camera driver framework on Orin NX, I couldn’t find an example of where such data processing typically occurs.
   Should I implement a custom function within the driver to perform this conversion, and is there a way to send the processed depth data back through the MIPI interface?
   

   

3. How can I access the four phase shift values from the sensor within the driver, and what is the recommended way to process them into depth values?

4. Are there any reference materials, sample drivers, or resources available to assist with iToF camera driver development on the Orin NX platform?

We would really appreciate any advice, documentation, or examples that could help us move forward with this project.

Thank you!

[ShaneCCC]

Current only able to receive 12 bit RAW data. For the depth data you need to parse it by yourself.

You may conider intel realsense camera alternative.

Thanks

[Star_sea]

This camera outputs data in RAW12 format.

1）Are you saying that the ISP on the Orin NX does not provide the functionality to convert phase information into depth information? Can such functionality be customized within the Orin NX ISP?

2）If not, for iTOF depth calculation, should it be handled in the driver or kernel, or should we directly acquire RAW12 frames and use a custom algorithm in user space to obtain the depth image?

[ShaneCCC]

Yes, ISP don’t support convert but only demosaicing from the RAW data.
You need to implement your algorithm to obtain the depth information from the RAW data.

Thanks