FSYNC-Group Clarification

[dbennington1]

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Hello,

I am interested in using heterogeneous triggering using fsync groups, and I think I understand most of it, but there are some gaps in the documentation I would like to clarify. Namely:

• According to this page, there are 3 multiplexors that take 3 different FSYNC signals and route them to the deserializers using MFP2 and MFP4, but this page talks about 4 different fsync generators, by default grouped into 2 FSYNC groups (generators 0, 2, and 3 on fsync-group0 and generator 1 on fsync-group1). Can NVIDIA clarify which FSYNC outputs correspond to which generator? For instance, does FSYNC1, 2 and 3 correspond to Generator 0, 2, and 3, and Generator 1 is just not connected?

• Using the nvsipl_camera sample, there is a -D flag which appears to take two different types of argument, either a 16-element list of 2 and 4 or a 3-element list of 1-3 (though I believe 0 may also be available). The documentation doesn’t specify which is which or if both are needed or if it’s one at a time - I suspect if you supply the 16-element list, you are assigning each camera channel on a per-deserializer basis whether it’s forwarding MFP2 or 4 as fsync to that particular camera, and if you provide the 3-element list you are assigning the multiplexor index to multiplexors 1, 2, and 3, but it seems odd to me to that these are defined together - can you use the same flag twice and use a different argument list if you want to specify the MFP and the multiplexor index? Can only one be defined at once? Am I misinterpreting it entirely?

Thanks

[SivaRamaKrishnaNV]

Dear @dbennington1,
Thank you for bringing to our attention. I will check internally and get back to you.

[dbennington1]

To answer my own second question, it appears that the multiplexor indicies are set via the -U flag, though the help text in the nvsipl_camera application appears to mix them up slightly.

...
-D or --syncGPIOIdxDes 'gpios'             :GPIO index of the deserializer for FRSYNC signal. Applicable only for the specific board
                                           :GPIO index is a list of GPIOs for each deserializer.
                                           :Eg: '2 2 2 2 2 2 2 2 2 2 2 2 2 4 2 4' use GPIO 4 for the link 1 and 3 on CSI-GH interface and GPIO 2 is used for other links
                                           :Eg: '1 2 3' selects the selection index 1 for the multiplxer on CSI-CD, the selection index 2 for the multiplxer on CSI-EF, the selection index 3 for the multiplxer on CSI-GH,
...
-U or --syncMuxSelDes 'Indexes'            :Selection index to the multiplexer. Applicable only for the specific board
                                           :Index is a list of selection index for each deserializer.
...

I believe that second example in the -D flag may be meant to be associated with the -U helptext.

I am still uncertain on the FSYNC to generator mapping, however.

[dbennington1]

An additional piece of information, it would appear as though -D and -U have absolutely no effect on my system - I set it to things that should work, things that shouldn’t work, and things that should have thrown an error on the command line, the command executes just fine and all cameras trigger as according to generator0 (I split the fsync groups in the DT). I have triple-checked, the flashing script detects a p3710-10-s05 in a 940-63710-0010-200, according to the documentation it should absolutely be listening to these flags. Any ideas?

[collin.day]

We have similar issues. I am trying to set the frequency and duty cycle of different generators by altering the dtsi files mentioned in the documentation. It is not clear at all which fsync generators correlate to which pin. How would a person go about saying “1 want generator 0 to be connected to MFP2 of the deserializer”, etc.?

[dbennington1]

Hello,

Any updates? This has been up for a week with no movement from NVIDIA.

[VickNV]

Could you please provide detailed information about your setup, any changes made, and the specific steps you followed during testing? This will help us thoroughly investigate the issue and provide an accurate resolution.

[dbennington1]

The setup is a single Quanta IMX728 camera connected to Port 0 of the Group A deserializer ports on a Drive Orin AGX.

The DTSI at /drive/drive-linux/kernel/source/hardware/nvidia/platform/t23x/automotive/kernel-dts/p3710/common/tegra234-p3710-0010.dtsi has been modified with the following:

 fsync-groups {
	status = "disabled";
	fsync-group@0 {
		id = <0>;
		status = "okay";
		generators = <&gen0>, <&gen2>, <&gen3>;
	};
	fsync-group@1 {
		id = <1>;
		status = "okay";
		generators = <&gen1>;
	};
};

to

 fsync-groups {
	status = "okay";
	fsync-group@0 {
		id = <0>;
		status = "okay";
		generators = <&gen0>;
	};
	fsync-group@1 {
		id = <1>;
		status = "okay";
		generators = <&gen1>;
	};
	fsync-group@2 {
		id = <2>;
		status = "okay";
		generators = <&gen2>;
	};
	fsync-group@3 {
		id = <3>;
		status = "okay";
		generators = <&gen3>;
	};
};

which was necessary to enable fsync-groups and to separate the different generators, a supported configuration as of 6.0.8.1 as per this page.

Using the command

sudo ./nvsipl_camera -c HZKJ_IMX728_ES2_V2_070FOV_CPHY_x4 -m "1 0 0 0" -sR -v 4 -D '4 4 4 4 2 2 2 2 2 2 2 2 2 2 2 2' -F '0 <TSC_TIME>'

I would expect that it would start generator0, but select from MFP4 on Deserializer Group A for all 4 camera ports, which should be FSYNC3. Since I know from other experimentation that FSYNC1 is Generator0, I would expect that this would result in no or asynchronous triggering, however, it operates exactly as it would with Generator0.

Using the command

sudo ./nvsipl_camera -c HZKJ_IMX728_ES2_V2_070FOV_CPHY_x4 -m "1 0 0 0" -sR -v 4 -D '40 a e 4 2 5 2 2 2 2 2 2 2 2 2 2' -F '0 <TSC_TIME>'

just to do a sanity check, the command runs exactly the same way, triggering as though from Generator0 and showing no errors or instances in the outputs of the -D flag being parsed at all, erroneously or not.

Connecting a camera to the Group C Deserializer Group, Port 0, and doing the same thing results in the same result:

sudo ./nvsipl_camera -c HZKJ_IMX728_ES2_V2_070FOV_CPHY_x4 -m "0 0 1 0" -sR -v 4 -D '2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2' -F '0 <TSC_TIME>' -U '0 1 0'

This likewise results in the system running fine, as according to Generator0, but I would expect this to be triggered from FSYNC2, which unless FSYNC1, 2, and 3 are all Generator0, I would not expect to work, and since I have not started any fsync-group but group 0 (generator0), I would expect this to either not trigger or run asynchronously.

Finally, I issue

sudo ./nvsipl_camera -c HZKJ_IMX728_ES2_V2_070FOV_CPHY_x4 -m "0 0 1 0" -sR -v 4 -D '2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2' -F '0 <TSC_TIME>' -U '10 a c'

Which as before runs the same way, triggering as though from generator0 and showing no errors instances int he outputs of the -U flag being parsed at all.

I used the DriveAGX SDK to insert printf’s into the nvsipl_camera code, on line 2034 of main.cpp it appears as though upMaster->GetMAX96712FusaNVCustomInterface(d) returns a nullptr, which silently fails and resumes execution. On a lark, I tried using generic drivers instead of the Quanta ones:

nvsipl_camera -c V1SIM728S3RU4070HC10_CPHY_x4 -m "1 0 0 0" -sR -r 5 --disableSubframe -v4 -D '40 a e 4 2 5 2 2 2 2 2 2 2 2 2 2' -F '0 <TSC_TIME>'

Same issue. The -D flag and -U flags are not being parsed at all. I am not sure what the custom interface is, it is not in any documentation I have seen.

[dbennington1]

Any further details necessary?

[VickNV]

Which camera module do you use to run with the ‘V1SIM728S3RU4070HC10_CPHY_x4’ configuration? Does ‘Heterogeneous Frame Synchronization’ work with it? Could you please share the complete command and its output? Have you observed the ‘MAX96712 Fusa NV custom interface found’ message?

[dbennington1]

Hello,

I use the same Quanta IMX728 camera with the V1SIM728S3RU4070HC10_CPHY_x4 configuration, it’s not quite right anecdotally the modules are similar enough that using it without loading the Quanta drivers initializes the camera properly. It was really just a sanity check, and in any case, the heterogeneous frame synchronization doesn’t appear to work with that either - I have never seen the ‘MAX96712 Fusa NV custom interface found’ message in any of the outputs of any of the commands shown above. The complete command is shown in the post above, I will see about getting the full output.

In a more concrete sense, what modules has NVIDIA confirmed as working with heterogeneous frame synchronization? Does heterogeneous frame synchronization require vendor support with camera drivers, or is it an NVIDIA-side configuration? Can you share an example of a command that does take the -U and -D flags and applies them accurately?

[VickNV]

AFAIK it’s working with some Valeo or Smartlead IMX728 and IMX623 modules listed on https://developer.nvidia.com/drive/ecosystem-hw-sw#cameras and https://developer.nvidia.com/drive/ecosystem-orin#camera.

If you are using a Quanta camera module listed on those pages, I recommend reaching out to Quanta directly, as mentioned on the page. They closely collaborate with our camera team and have in-depth knowledge of the support for their modules. Quanta can provide specific information and support related to Quanta camera modules.

[dbennington1]

Hello,

Thanks for confirming. I spoke with Quanta, it does appear as though the driver I was using does not support heterogeneous frame sync, so nvsipl_camera not working does make sense - thanks for working with me on that.

The first question, however, as well as the question Collin seems to have, has not yet been answered. What is the mapping between fsync generators and FSYNC outputs?

[VickNV]

The following is the mapping between fsync generators and FSYNC outputs:

FSYNC1 corresponds to gen0: generator@380
FSYNC2 corresponds to gen3: generator@500
FSYNC3 corresponds to gen2: generator@480

[dbennington1]

Wonderful, thank you!