Ser/des driver for multi csi inputs with gstreamer

Hi,
I wrote a driver for a FPD Link deserializer which connected to one serializer from one side and to the Orin NX CSI port from the other side.
Gstreamer works and shows the data via v4l2src=/dev/video0

My question is what are the changes required for the Gstreamer to display the data of both the first serializer and a second serializer connected to the same deserializer that is now connected to both CSI ports of the Orin NX ?
The deserializer outputs the first serializer data via mipi CSI port 0 and outputs the second serializer data via mipi CSI port 1.

Do I need:

  1. make changes only in the Gstreamer pipe?
  2. make changes in my current driver which register the deserializer as a v4l2 device?
  3. Add an entirely new driver to support the additional serializer output?

Its my first experience so any help will be much appreciated.
Thanks!

Need make change to register video node for each input.

Thanks

Thanks @ShaneCCC ,
Is there a reference from which I can learn how to do it?

  • Is it requires changes in driver code only or in device tree as well?

Thanks

Have reference to below document for virtual channel implement.

https://docs.nvidia.com/jetson/archives/r35.5.0/DeveloperGuide/SD/CameraDevelopment/JetsonVirtualChannelWithGmslCameraFramework.html

Thanks @ShaneCCC ,
I apologies but I think I didn’t explain myself well:
My setup is:

  • 2 serializers: ser1 , ser2
  • 1 deserializer: des1 with two separate RX ports (1 for each ser) and two separate TX ports (1 for each orin csi RX ports)
  • Orin nx with two separate CSI RX ports

The connections are:
Ser1->Des1_RX0 Des1_TX0->Orin nx csi port 0
Ser2->Des1_RX1 Des1_TX1->Orin nx csi port 1

The serializes connected to separate ports all the way to separate ports on the orin nx, hence I don’t need to use virtual channels. (Am I wrong?)

So I need a little help and maybe an example of how to create 1 driver to control two identical (but distinct) serializers connected via two separate ports without using virtual channels.

Thanks again

Hello @bsp_dev,

We believe you are correct, there should not be a need of using virtual channels.

We think you would need the following:

  1. Add the extra node to the device tree.
  2. Create a dummy driver for tegracam to be happy.
  3. Modify your SERDES driver to account for the propper configuration.

Don’t get us wrong, it is easier said than done. However, it should be possible and more importantly a good bit of fun.

Let us know if you need a hand.

regards,
Andrew
support@proventusnova.com

Hi @proventusnova ,
Thank you for your reply.
It’s my first time experiencing with these stuff (kernel drivers, v4l2, ser/des, etc…)

I don’t know where to start from.

  1. Regarding the device tree - I found an example (camera-rbpcv3-imx477)
  2. Regarding the driver modifications - I don’t know what to do. Currently I create and register a v4l2 device for my deserializer and not for my serializer. It expose the /dev/video0 file. How I expose additional dev file: /dev/video1 that will control the same des but will transfer a specific serializer data ?

@bsp_dev,

Got it.

I would first focus on setting up the two device nodes in the DTB, along with the dummy camera driver for capturing.

And, I would run all SERDES configuration using i2c set commands on a script.

That would at least remove thar part of complexity from your system.

Once you get it capturing from both CSI ports. You can start working on the SERDES driver.

regards,
Andrew
support@proventusnova.com

  1. What do you mean by a “dummy camera driver”?
  • I followed the sensor programing chapter in the dev guide to write my current driver.
  1. why exporting the SERDES configuration to a script? this way I’ll not be able to change its behavior during runtime via setting different v4l2 device mode (mode 0, mode 1, etc…) Am I wrong?
  1. That is a great question. What I mean by dummy camera driver, ia a driver tha will only configure the Jetson capture subsystem. Usually a camera driver also configures the camera itself, which usually changes when you have to do it through a SERDES connection.
  2. The suggestion to using a script to configure the SERDES is only to simplify your problem set while you are on this devempment stage, so that you can only focus for now on configuring the capture subsystem. However, if you feel more confortable working directly with the driver, go for it.

Regards,
Andrew
support@proventusnova.com

@proventusnova
First of all thank you very much Andrew for your help and patience.
I appreciate it very much.

I apologies but I don’t understand your answers entirely since I’m not familiar enough with these subjects.
By your permission I’ll try to break the driver modification job into smaller points to try and understand.

  1. Current driver:
    The DES is the v4l2 device. The driver exposes the /dev/video0 file. Each operation on this file results in reading/writing to the DES regs. The gstreamer source is the /dev/video0 file meaning its source is the DES?

  2. “New” driver

  • The DES is still the v4l2 device or now I need to register the serializers as v4l2 devices?

  • If that’s the case, how the app will be able to change its behavior? (now the sers are the v4l2 devices, hence they support the mode0, mode1, etc). Do I need 3 v4l2 devices now?

  • Maybe there is a sample driver that I can see how they did it?

@bsp_dev,

No worries, its a conplex topic and we know how it feels to be in your position. It is a pleasure to help, or at least try.

Let’s try the following…

Would you be down to schedule a quick call so we can talk about it and for me to try to better explain myself?

We can schelule maybe tomorrow.

Regards,
Andrew
support@proventusnova.com

@proventusnova I sent you a private message to try and schedule.

Sounds good, looking forward to talking with you.

Regards,
Andrew
support@proventusnova.com