If a camera is a USB device (and not a host), then it is a violation of standards to use a type-A connector. It really needs a type-B. Looking closer at your wiring diagram it shows pin #2 for both D+ and D- of plug #2, and I assume that is a typographic error (it says D+ and D- are shorted). Can you verify though if it is D+ of the left connector to D+ of the right connector? If it is D+ to D- I suspect it won’t work.
In the case of the USB 3 wires I don’t know if that would work or not, although likely it would work. What does your USB 3 logic analyzer say (do you have an analyzer, it would be quite helpful)? Is there activity on the USB 3 wiring, or is the activity all on the USB 2 wiring? If the former, then the controller is probably configured correctly, and something else is telling it to fall back; however, if it is the latter, then perhaps either wiring or controller is in the wrong configuration.
As a test, assuming the Jetson has a standard USB 3 type-A host port, why not try plugging in other USB 3 devices to see if they work at USB 3 speed? Is it just one device failing, or is it all USB 3 devices?
Incidentally, you should never connect a type-A host port to another type-A host port (in theory you are connecting a type-A host to a type-A device, but Jetson to laptop type-A to type-A is a bad idea). This is not full duplex hardware. One end must be a master. In the case of USB 3 type-C it is a bit of an illusion that the cable is full duplex. When USB 3.1 gen. 2+ is used there are extra data wires, and each half is one-way (with a host/master and device/slave) despite being able to configure each half to become host or device (sorry, my USB 3 is not so great, “I’m a USB 2 guy”). It isn’t full duplex despite having two USB lines if that makes sense (not sure I’m describing it correctly, but two independent USB data paths are still independent balanced pairs and not full duplex with TX/RX just because they are in the same cable).
The device tree does change what controller is available. One controller handles legacy (USB2 or older), while another controller handles USB3+. If USB3 is enabled, it implies certain conditions of the connected device will cause routing to the USB3 controller when it is enabled, but failure of those conditions will route instead to the USB2 controller. Failing either the device tree configuration or failing the other wiring states will cause routing to a USB2 controller. Thus I asked if any of the USB3 devices work as USB3.
You might also extract your device tree and post that:
dtc -I fs -O dts -o extracted_tree.dts /proc/device-tree
(then attach extracted_tree.dts
; you could bzip/gzip it if you want)
I won’t be able to answer what device tree changes are needed, but I can guarantee that if the wiring is wrong, then it won’t allow it even if the device tree is correct. I strongly suggest checking with a logic analyzer and also never using a type-A to type-A cable between two hosts.