Jetson Orin NX CAN Only Works in Loopback

robert-osoelectronics · April 24, 2024, 11:37pm

Hello!
I’ve been struggling to get CAN working on my Orin NX. I’ve found that transmit only works when I enable CAN loopback mode, and I have not been able to get CAN receive to work at all. I’ve seen that a few other folks have reported similar issues, but that the threads ended without any real resolution (thread1, thread2).

Hardware Setup

I’m using a Jetson Orin NX Dev Kit with a CAN transceiver hooked up to the J17 header. I’m using the recommended Waveshare transceiver board, and have verified that it is wired correctly.

For a link partner, I’m using a Peak Systems PCAN-USB. I’ve verified that the CAN bus is properly terminated with 120Ω on each end (measured 60Ω between CAN_H/CAN_L). and verified that the PCAN-USB can send messages onto the bus as expected (although they are not ACK’d by the Jetson).

Here’s what I see on the oscilloscope (probing channel1=CAN_H, channel2=CAN_L, probed at the header on the Jetson’s CAN transceiver board):

As you can see, the bus signal integrity looks great, and the CAN packet is properly formed, although not being ACK’d by the Jetson.

Software Setup

The Jetson has been freshly flashed from SDKManager. I’ve tried various Jetpack versions to see if that made any difference (I didn’t have any luck with any of them), but the results reported here are with a fresh flash with Jetpack 5.1.1.

Here’s the report in JetsonHacks jtop:

I followed the directions in the Developer Guide for setting up CAN:

jetson@ubuntu:~$ sudo busybox devmem 0xc303018 w 0xc458
jetson@ubuntu:~$ sudo busybox devmem 0xc303010 w 0xc400
jetson@ubuntu:~$ sudo modprobe can
jetson@ubuntu:~$ sudo modprobe can_raw
jetson@ubuntu:~$ sudo modprobe mttcan
jetson@ubuntu:~$ sudo ip link set can0 up type can bitrate 500000 dbitrate 1000000 berr-reporting on fd on
jetson@ubuntu:~$ ip -d link show can0
9: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 72 qdisc pfifo_fast state UP mode DEFAULT
group default qlen 10
    link/can  promiscuity 0 minmtu 0 maxmtu 0
    can <BERR-REPORTING,FD> state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0
          bitrate 500000 sample-point 0.870
          tq 20 prop-seg 43 phase-seg1 43 phase-seg2 13 sjw 1
          mttcan: tseg1 2..255 tseg2 0..127 sjw 1..127 brp 1..511 brp-inc 1
          dbitrate 1000000 dsample-point 0.720
          dtq 40 dprop-seg 8 dphase-seg1 9 dphase-seg2 7 dsjw 1
          mttcan: dtseg1 1..31 dtseg2 0..15 dsjw 1..15 dbrp 1..15 dbrp-inc 1
          clock 50000000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
jetson@ubuntu:~$ candump -x any &
[1] 3812

FAILURE - Non-loopback Packet Transmission

At this point, if I send a packet with cansend can0 123#abcdabcd, the following occurs:

The link partner (PCAN-USB) does not report receiving the packet
candump does not report a transmitted packet
On the oscilloscope (same probing setup as above), I see some confusing activity but not a valid packet transmission:

Here’s another scope capture from a different run zoomed out:

If I then try to run cansend again, I get no activity on the bus.

The system reports state: BUS-OFF and I see some bit errors in dmesg:

jetson@ubuntu:~$ ip -d link show can0
9: can0: <NO-CARRIER,NOARP,UP,ECHO> mtu 72 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 10
    link/can  promiscuity 0 minmtu 0 maxmtu 0
    can <BERR-REPORTING,FD> state BUS-OFF (berr-counter tx 248 rx 0) restart-ms 0
          bitrate 500000 sample-point 0.870
          tq 20 prop-seg 43 phase-seg1 43 phase-seg2 13 sjw 1
          mttcan: tseg1 2..255 tseg2 0..127 sjw 1..127 brp 1..511 brp-inc 1
          dbitrate 1000000 dsample-point 0.720
          dtq 40 dprop-seg 8 dphase-seg1 9 dphase-seg2 7 dsjw 1
          mttcan: dtseg1 1..31 dtseg2 0..15 dsjw 1..15 dbrp 1..15 dbrp-inc 1
          clock 50000000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
jetson@ubuntu:~$ sudo dmesg | tail
[  539.644100] mttcan_controller_config: ctrlmode 30
[  539.644121] mttcan c310000.mttcan can0: Bitrate set
[  539.644231] IPv6: ADDRCONF(NETDEV_CHANGE): can0: link becomes ready
[  969.014314] mttcan c310000.mttcan can0: Bit0 Error Detected
[  969.020081] mttcan c310000.mttcan can0: IR 0x8010000 PSR 0x71d
[  969.026178] mttcan c310000.mttcan can0: entered error warning state
[  969.032686] mttcan c310000.mttcan can0: entered error passive state
[  969.039183] mttcan c310000.mttcan can0: entered bus off state
[  969.045118] mttcan c310000.mttcan can0: Bit0 Error Detected
[  969.050895] mttcan c310000.mttcan can0: IR 0xb800000 PSR 0x7e5

This state seems unrecoverable. I’ve tried setting the link down and then back up again to get out of the BUS-OFF state (which appears to succeed), but I never see any more activity on the bus with further can-send commands.

jetson@ubuntu:~$ sudo ip link set can0 down
jetson@ubuntu:~$ sudo ip link set can0 up
jetson@ubuntu:~$ ip -d link show can0
9: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 72 qdisc pfifo_fast state UP mode DEFAULT group default qlen 10
    link/can  promiscuity 0 minmtu 0 maxmtu 0
    can <BERR-REPORTING,FD> state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0
          bitrate 500000 sample-point 0.870
          tq 20 prop-seg 43 phase-seg1 43 phase-seg2 13 sjw 1
          mttcan: tseg1 2..255 tseg2 0..127 sjw 1..127 brp 1..511 brp-inc 1
          dbitrate 1000000 dsample-point 0.720
          dtq 40 dprop-seg 8 dphase-seg1 9 dphase-seg2 7 dsjw 1
          mttcan: dtseg1 1..31 dtseg2 0..15 dsjw 1..15 dbrp 1..15 dbrp-inc 1
          clock 50000000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
jetson@ubuntu:~$ cansend can0 123#abcdabcd
jetson@ubuntu:~$ ip -d link show can0
9: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 72 qdisc pfifo_fast state UP mode DEFAULT group default qlen 10
    link/can  promiscuity 0 minmtu 0 maxmtu 0
    can <BERR-REPORTING,FD> state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0
          bitrate 500000 sample-point 0.870
          tq 20 prop-seg 43 phase-seg1 43 phase-seg2 13 sjw 1
          mttcan: tseg1 2..255 tseg2 0..127 sjw 1..127 brp 1..511 brp-inc 1
          dbitrate 1000000 dsample-point 0.720
          dtq 40 dprop-seg 8 dphase-seg1 9 dphase-seg2 7 dsjw 1
          mttcan: dtseg1 1..31 dtseg2 0..15 dsjw 1..15 dbrp 1..15 dbrp-inc 1
          clock 50000000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
jetson@ubuntu:~$ sudo dmesg | tail
[  969.032686] mttcan c310000.mttcan can0: entered error passive state
[  969.039183] mttcan c310000.mttcan can0: entered bus off state
[  969.045118] mttcan c310000.mttcan can0: Bit0 Error Detected
[  969.050895] mttcan c310000.mttcan can0: IR 0xb800000 PSR 0x7e5
[ 1556.229103] mttcan_controller_config: ctrlmode 30
[ 1556.229131] mttcan c310000.mttcan can0: Bitrate set
[ 1556.229140] mttcan c310000.mttcan can0: wait for bus off seq
[ 1556.241226] mttcan c310000.mttcan can0: Bit0 Error Detected
[ 1556.241268] IPv6: ADDRCONF(NETDEV_CHANGE): can0: link becomes ready
[ 1556.246995] mttcan c310000.mttcan can0: IR 0xa000000 PSR 0x70d

I’ve only been able to get the bus to show activity (the same incorrect activity in the scope shot above, but activity nonetheless) after a system reboot.

SUCCESS? - Loopback Packet Transmission

After a ton of trial and error, I found that I could get packets to reliably transmit correctly if I enable loopback on the CAN peripheral. With all the other settings and steps from above unchanged, if I do:

jetson@ubuntu:~$ sudo ip link set can0 up type can bitrate 500000 dbitrate 1000000 berr-reporting on fd on loopback on
jetson@ubuntu:~$ ip -d link show can0
9: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 72 qdisc pfifo_fast state UP mode DEFAULT group default qlen 10
    link/can  promiscuity 0 minmtu 0 maxmtu 0
    can <LOOPBACK,BERR-REPORTING,FD> state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0
          bitrate 500000 sample-point 0.870
          tq 20 prop-seg 43 phase-seg1 43 phase-seg2 13 sjw 1
          mttcan: tseg1 2..255 tseg2 0..127 sjw 1..127 brp 1..511 brp-inc 1
          dbitrate 1000000 dsample-point 0.720
          dtq 40 dprop-seg 8 dphase-seg1 9 dphase-seg2 7 dsjw 1
          mttcan: dtseg1 1..31 dtseg2 0..15 dsjw 1..15 dbrp 1..15 dbrp-inc 1
          clock 50000000 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
jetson@ubuntu:~$ candump -x any &
[1] 3042
jetson@ubuntu:~$ cansend can0 123#abcdabcd
jetson@ubuntu:~$   can0  RX - -  123   [4]  AB CD AB CD
  can0  TX - -  123   [4]  AB CD AB CD

I see the CAN packet transmitted correctly to the bus and reported by candump. Here’s a scope shot:

And the link partner (PCAN-USB) correctly receives the CAN packet as well. I can continue sending packets, and they are correctly sent to the bus. However, as you’d expect I cannot receive any packets on the Jetson due to loopback being enabled.

Questions

Can someone help me debug why messages are not transmitted correctly unless loopback mode is on?
Is there a way to recover from BUS-OFF without rebooting the system?

KevinFFF · April 25, 2024, 8:14am

Hi robert-osoelectronics,

Since you are using the devkit, could you update to the latest JP5.1.3(R35.5.0)?

I think the BUS-OFF state is caused from wrong CAN connection.

Could you share the block diagram of your connection for CAN?

robert-osoelectronics · April 25, 2024, 5:04pm

Hi Kevin - thanks for the quick reply!

Since you are using the devkit, could you update to the latest JP5.1.3(R35.5.0)?

I’ve updated to 5.1.3:

And I re-ran the steps from my first post to reproduce the issue. The behavior is all exactly as before.

I think the BUS-OFF state is caused from wrong CAN connection.

I am confident that I have the CAN bus connected correctly for the following reasons:

When I transmit from the Jetson in loopback mode, the CAN packets are successfully received by the Peak PCAN-USB module (so my TX pin, power, and GND to the CAN transceiver are hooked up correctly)
When I transmit from the PCAN-USB module and use the oscilloscope to probe J17 Pin 3 on the bottom of the Jetson, I see the CAN message (so my RX pin is hooked up correctly)
BUS-OFF only occurs after I attempt to transmit from the Jetson not in loopback mode
Oscilloscope measurements (see previous post) of the CAN bus show correct polarity and voltage levels for CAN transmissions from the PCAN-USB as well as from the Jetson

Could you share the block diagram of your connection for CAN?

Here you go:

And here’s a description of the setup:
I’m using a Jetson Orin NX Dev Kit with a CAN transceiver hooked up to the J17 header. I’m using the recommended Waveshare transceiver board, and have verified that it is wired correctly.

For a link partner, I’m using a Peak Systems PCAN-USB. I’ve verified that the CAN bus is properly terminated with 120Ω on each end (measured 60Ω between CAN_H/CAN_L). and verified that the PCAN-USB can send messages onto the bus as expected (although they are not ACK’d by the Jetson).

KevinFFF · April 26, 2024, 7:14am

Actually, the loopback on should means internal loopback. (i.e. you don’t need to connect any external cable and it should still work).

Thanks for sharing the connection to check, it looks good to me.

Do you have another Orin NX devkit and Waveshare CAN Transceiver to verify CAN communication between 2 Orin NX devkits?

Have you tried other values for bitrate and dbitrate?

Do you disable the BRS bit (Bit Rate Switch) from your PCAN?

robert-osoelectronics · April 26, 2024, 4:53pm

Actually, the loopback on should means internal loopback. (i.e. you don’t need to connect any external cable and it should still work).

Understood. Per my scope measurements above, it looks like the CAN peripheral still transmits when internal loopback mode is enabled, however.

I’m beginning to wonder if the issue is something in the receive path. I believe the CAN peripheral monitors transmitted bits on its receive line, to see if another device is “stepping” on its transmission for bus arbitration. It would make sense that transmit would work during loopback mode, when the receive path is internal to the CAN peripheral, but not during non-loopback operation?

Working on this theory, is there anything else I should check beyond the pin multiplexing? I have verified the physical connection from the CAN transceiver to J17 pin 3 (CAN_RX). As mentioned in my last post, if I transmit a CAN packet from my PCAN-USB and probe CAN_RX at J17 on the bottom side of the Dev Kit, I see the packet as expected.

Do you have another Orin NX devkit and Waveshare CAN Transceiver to verify CAN communication between 2 Orin NX devkits?

I do not have another Orin NX devkit unfortunately. I have tried with several different CAN transceiver boards with identical results.

Have you tried other values for bitrate and dbitrate?

I have tried 250kbps baud rate with identical results, and have tried disabling CAN-FD by running:

sudo ip link set can0 up type can bitrate 250000 berr-reporting on

Do you disable the BRS bit (Bit Rate Switch) from your PCAN?

Unfortunately the option to disable BRS is greyed out in my PCAN-View software. However, my understanding is that BRS is a CAN FD feature? I am not transmitting CAN FD packets, so hopefully this isn’t an issue?

In any case, I would expect the Jetson to transmit a packet correctly as long as the bus is properly configured and not busy, whether or not the link partner correctly ACKs or receives the packet. Currently it is not transmitting the packet.

Thank you again for your help!

KevinFFF · April 30, 2024, 9:58am

Sorry that I don’t have PCAN so that I can’t reproduce your issue locally.
Normally, we would get 2 devkits to transmit CAN to verify the functionality.
If the loopback test working, then it means the pinmux/configurations are correct.
The issue might be caused from PCAN side.

system · May 22, 2024, 5:49am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.