Jetson TX2 mmc_rx_fifo_overflow errors

We are using Jetson TX2 with tegra-l4t-r32.3.1 kernel.

Test setup:
Router ------------ Netgear Switch ---------------- Nvidia Jetson TX2

In our test scenario, Jetson TX2 gets the network IP address over DHCP or static IP.

Switching between dhcp or static ip is done using systemd-networkd, and works fine most of the time.

But occasionally after switching dhcp to static / static to dhcp , we noticed Jetson TX2 is unable to receive any packets but able to send packets out.

When this issue happens , I could see ethtool reporting mmc_rx_fifo_overflow errors continuously.

eth0 interface TX packet count increase with egress packets , but RX packet count remains constant and don’t even see RX interrupt count changing.

Could you please help us understand the triggers for mmc_rx_fifo_overflow ? I am unable to find any documents detailing the ethernet MAC registers. Is there any known issues Jetson TX2 MAC?

jetson-tx2-dev:/sys/kernel/debug# cat /proc/interrupts | grep qos
35: 10 0 0 0 0 0 GICv2 226 Level ether_qos.common_irq
37: 3067 0 0 0 0 0 GICv2 222 Level 2490000.ether_qos.rx0 <<<<<<<< No change
38: 10180 0 0 0 0 0 GICv2 218 Level 2490000.ether_qos.tx0 <<<<<<< change with outgoing packets

------ ethtool dump
jetson-tx2-dev:/sys/kernel/debug# ethtool -i eth0
driver: eqos
version:
firmware-version:
expansion-rom-version:
bus-info: 2490000.ether_qos
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

jetson-tx2-dev:/sys/kernel/debug# ethtool -S eth0
NIC statistics:
mmc_tx_octetcount_gb: 14760907
mmc_tx_framecount_gb: 229948
mmc_tx_broadcastframe_g: 7787
mmc_tx_multicastframe_g: 221607
mmc_tx_64_octets_gb: 229353
mmc_tx_65_to_127_octets_gb: 378
mmc_tx_128_to_255_octets_gb: 176
mmc_tx_256_to_511_octets_gb: 36
mmc_tx_512_to_1023_octets_gb: 5
mmc_tx_1024_to_max_octets_gb: 0
mmc_tx_unicast_gb: 554
mmc_tx_multicast_gb: 221607
mmc_tx_broadcast_gb: 7787
mmc_tx_underflow_error: 0
mmc_tx_singlecol_g: 0
mmc_tx_multicol_g: 0
mmc_tx_deferred: 0
mmc_tx_latecol: 0
mmc_tx_exesscol: 0
mmc_tx_carrier_error: 0
mmc_tx_octetcount_g: 14760907
mmc_tx_framecount_g: 229948
mmc_tx_excessdef: 0
mmc_tx_pause_frame: 221561
mmc_tx_vlan_frame_g: 0
mmc_rx_framecount_gb: 17971
mmc_rx_octetcount_gb: 1608718
mmc_rx_octetcount_g: 1608718
mmc_rx_broadcastframe_g: 4837
mmc_rx_multicastframe_g: 5172
mmc_rx_crc_error: 0
mmc_rx_align_error: 0
mmc_rx_run_error: 0
mmc_rx_jabber_error: 0
mmc_rx_undersize_g: 0
mmc_rx_oversize_g: 0
mmc_rx_64_octets_gb: 16360
mmc_rx_65_to_127_octets_gb: 136
mmc_rx_128_to_255_octets_gb: 55
mmc_rx_256_to_511_octets_gb: 1412
mmc_rx_512_to_1023_octets_gb: 2
mmc_rx_1024_to_max_octets_gb: 6
mmc_rx_unicast_g: 7962
mmc_rx_length_error: 0
mmc_rx_outofrangetype: 0
mmc_rx_pause_frames: 0
mmc_rx_fifo_overflow: 17263 <<<<<<<<<<<<< increase with every input packet
mmc_rx_vlan_frames_gb: 0
mmc_rx_watchdog_error: 0
mmc_rx_ipc_intr_mask: 24694931433
mmc_rx_ipc_intr: 0
mmc_rx_ipv4_gd: 1871
mmc_rx_ipv4_hderr: 0
mmc_rx_ipv4_nopay: 0
mmc_rx_ipv4_frag: 0
mmc_rx_ipv4_udsbl: 0
mmc_rx_ipv6_gd_octets: 0
mmc_rx_ipv6_hderr_octets: 0
mmc_rx_ipv6_nopay_octets: 0
mmc_rx_udp_gd: 1734
mmc_rx_udp_err: 0
mmc_rx_tcp_gd: 129
mmc_rx_tcp_err: 0
mmc_rx_icmp_gd: 8
mmc_rx_icmp_err: 0
mmc_rx_ipv4_gd_octets: 543600
mmc_rx_ipv4_hderr_octets: 0
mmc_rx_ipv4_nopay_octets: 0
mmc_rx_ipv4_frag_octets: 0
mmc_rx_ipv4_udsbl_octets: 0
mmc_rx_ipv6_gd: 0
mmc_rx_ipv6_hderr: 0
mmc_rx_ipv6_nopay: 0
mmc_rx_udp_gd_octets: 487097
mmc_rx_udp_err_octets: 0
mmc_rx_tcp_gd_octets: 18571
mmc_rx_tcp_err_octets: 0
mmc_rx_icmp_gd_octets: 512
mmc_rx_icmp_err_octets: 0
q_re_alloc_rx_buf_failed[0]: 0
q_re_alloc_rx_buf_failed[1]: 0
q_re_alloc_rx_buf_failed[2]: 0
q_re_alloc_rx_buf_failed[3]: 0
q_re_alloc_rx_buf_failed[4]: 0
q_re_alloc_rx_buf_failed[5]: 0
q_re_alloc_rx_buf_failed[6]: 0
q_re_alloc_rx_buf_failed[7]: 0
tx_process_stopped_irq_n[0]: 0
tx_process_stopped_irq_n[1]: 0
tx_process_stopped_irq_n[2]: 0
tx_process_stopped_irq_n[3]: 0
tx_process_stopped_irq_n[4]: 0
tx_process_stopped_irq_n[5]: 0
tx_process_stopped_irq_n[6]: 0
tx_process_stopped_irq_n[7]: 0
rx_process_stopped_irq_n[0]: 0
rx_process_stopped_irq_n[1]: 0
rx_process_stopped_irq_n[2]: 0
rx_process_stopped_irq_n[3]: 0
rx_process_stopped_irq_n[4]: 0
rx_process_stopped_irq_n[5]: 0
rx_process_stopped_irq_n[6]: 0
rx_process_stopped_irq_n[7]: 0
tx_buf_unavailable_irq_n[0]: 0
tx_buf_unavailable_irq_n[1]: 0
tx_buf_unavailable_irq_n[2]: 0
tx_buf_unavailable_irq_n[3]: 0
tx_buf_unavailable_irq_n[4]: 0
tx_buf_unavailable_irq_n[5]: 0
tx_buf_unavailable_irq_n[6]: 0
tx_buf_unavailable_irq_n[7]: 0
rx_buf_unavailable_irq_n[0]: 0
rx_buf_unavailable_irq_n[1]: 0
rx_buf_unavailable_irq_n[2]: 0
rx_buf_unavailable_irq_n[3]: 0
rx_buf_unavailable_irq_n[4]: 0
rx_buf_unavailable_irq_n[5]: 0
rx_buf_unavailable_irq_n[6]: 0
rx_buf_unavailable_irq_n[7]: 0
rx_watchdog_irq_n: 0
fatal_bus_error_irq_n: 0
pmt_irq_n: 0
tx_normal_irq_n[0]: 10010
tx_normal_irq_n[1]: 0
tx_normal_irq_n[2]: 0
tx_normal_irq_n[3]: 0
tx_normal_irq_n[4]: 0
tx_normal_irq_n[5]: 0
tx_normal_irq_n[6]: 0
tx_normal_irq_n[7]: 0
rx_normal_irq_n[0]: 3067
rx_normal_irq_n[1]: 0
rx_normal_irq_n[2]: 0
rx_normal_irq_n[3]: 0
rx_normal_irq_n[4]: 0
rx_normal_irq_n[5]: 0
rx_normal_irq_n[6]: 0
rx_normal_irq_n[7]: 0
napi_poll_n: 0
tx_clean_n[0]: 10010
tx_clean_n[1]: 0
tx_clean_n[2]: 0
tx_clean_n[3]: 0
tx_clean_n[4]: 0
tx_clean_n[5]: 0
tx_clean_n[6]: 0
tx_clean_n[7]: 0
tx_path_in_lpi_mode_irq_n: 0
tx_path_exit_lpi_mode_irq_n: 0
rx_path_in_lpi_mode_irq_n: 0
rx_path_exit_lpi_mode_irq_n: 0
tx_pkt_n: 10015
rx_pkt_n: 3147
tx_vlan_pkt_n: 0
rx_vlan_pkt_n: 0
tx_timestamp_captured_n: 0
rx_timestamp_captured_n: 0
tx_tso_pkt_n: 6
q_tx_pkt_n[0]: 10015
q_tx_pkt_n[1]: 0
q_tx_pkt_n[2]: 0
q_tx_pkt_n[3]: 0
q_tx_pkt_n[4]: 0
q_tx_pkt_n[5]: 0
q_tx_pkt_n[6]: 0
q_tx_pkt_n[7]: 0
q_rx_pkt_n[0]: 3147
q_rx_pkt_n[1]: 0
q_rx_pkt_n[2]: 0
q_rx_pkt_n[3]: 0
q_rx_pkt_n[4]: 0
q_rx_pkt_n[5]: 0
q_rx_pkt_n[6]: 0
q_rx_pkt_n[7]: 0
link_disconnect_count: 2
link_connect_count: 8
jetson-tx2-dev:/sys/kernel/debug#

Please help validate your issue with NV devkit and rel-32.6.

Also. share the full log when error happens.

@WayneWWW thanks for response. Disabling the LLDP in systemd-networkd seems to be working, do you know if there are any known issues with eqos driver related to LLDP?

1 Like

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

We do have some user says LLDP affect the performance when device is directly connected to other device end to end.

It is not like your setup.

Do you have any log to share so that we can analyze what is going on?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.