Eth1 and eth0 have same mac address in our custom board

[848145724]

Dear all:
1 : With JetPack4.4_R32.4.3, RTL8111G NIC conneted to jetson nano’s pcie0 in custom board.
2: lspci -vvv looks normal

00:01.0 PCI bridge: NVIDIA Corporation Device 0fae (rev a1) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 83
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00001000-00001fff
        Memory behind bridge: 13000000-130fffff
        Prefetchable memory behind bridge: 0000000020000000-00000000200fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: <access denied>
        Kernel driver in use: pcieport

00:02.0 PCI bridge: NVIDIA Corporation Device 0faf (rev a1) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 83
        Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
        I/O behind bridge: 00002000-00002fff
        Memory behind bridge: fff00000-000fffff
        Prefetchable memory behind bridge: 0000000020100000-00000000201fffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: <access denied>
        Kernel driver in use: pcieport

01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
        Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 406
        Region 0: I/O ports at 1000 [size=256]
        Region 2: Memory at 13000000 (64-bit, non-prefetchable) [size=4K]
        Region 4: Memory at 20000000 (64-bit, prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: r8168

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 19)
        Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 407
        Region 0: I/O ports at 2000 [size=256]
        Region 2: Memory at 20104000 (64-bit, prefetchable) [size=4K]
        Region 4: Memory at 20100000 (64-bit, prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: r8168

3: Both eth0 and eth1 have the same mac, and eth0 or eth1 can obtain an IP address independently and access the Internet after execution command. when I change eth1 mac, eth0 change too.

sudo /etc/init.d/network-manager restart

eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 48:b0:2d:3e:4f:ad  txqueuelen 1000  (Ethernet)
        RX packets 98  bytes 14675 (14.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 155  bytes 14626 (14.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 150  base 0xd000  

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.113  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::709c:ed3c:5299:65f  prefixlen 64  scopeid 0x20<link>
        ether 48:b0:2d:3e:4f:ad  txqueuelen 1000  (Ethernet)
        RX packets 2558  bytes 1667895 (1.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 796  bytes 84987 (84.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 151  base 0xa000 

4: Because eth1 and eth0 both in ethernet@1003000 as follow, so i guess must be some properitys are missing.

        pcie@1003000 {
                /delete-property/ iommus;
                /delete-property/ iommu-map;
                /delete-property/ iommu-map-mask;
                nvidia,wake-gpio = <&gpio TEGRA_GPIO(A, 2) 0>;
                nvidia,pmc-wakeup = <&tegra_pmc
                                PMC_WAKE_TYPE_EVENT 0 PMC_TRIGGER_TYPE_LOW>;
                avdd-pll-uerefe-supply = <&max77620_ldo7>;
                hvddio-pex-supply = <&max77620_sd3>;
                dvddio-pex-supply = <&max77620_ldo1>;
                dvdd-pex-pll-supply = <&max77620_ldo1>;
                hvdd-pex-pll-e-supply = <&max77620_sd3>;
                vddio-pex-ctl-supply = <&max77620_sd3>;
                status = "okay";

                pci@1,0 {
#if TEGRA_XUSB_PADCONTROL_VERSION >= DT_VERSION_2
                        phys = <&{/xusb_padctl@7009f000/pads/pcie/lanes/pcie-1}>,
                                <&{/xusb_padctl@7009f000/pads/pcie/lanes/pcie-2}>,
                                <&{/xusb_padctl@7009f000/pads/pcie/lanes/pcie-3}>,
                                <&{/xusb_padctl@7009f000/pads/pcie/lanes/pcie-4}>;
                        phy-names = "pcie-0", "pcie-1", "pcie-2", "pcie-3";
#endif
                        nvidia,num-lanes = <4>;
                        //nvidia,plat-gpios = <&gpio TEGRA_GPIO(A, 0) GPIO_ACTIVE_HIGH>;
                        status = "okay";

                        ethernet@0,0 {
                                reg = <0x000000 0 0 0 0>;
                        };
                };

                pci@2,0 {
#if TEGRA_XUSB_PADCONTROL_VERSION >= DT_VERSION_2
                        phys = <&{/xusb_padctl@7009f000/pads/pcie/lanes/pcie-0}>;
                        phy-names = "pcie-0";
#endif
                        nvidia,num-lanes = <1>;
                        nvidia,plat-gpios = <&gpio TEGRA_GPIO(X, 3) GPIO_ACTIVE_HIGH>;
                        status = "okay";

                        ethernet@0,0 {
                                reg = <0x000000 0 0 0 0>;
                        };
                };
        };

Could you give me some suggestions. thanks very much.

[848145724]

1: When modify tegra210-porg-plugin-manager.dtsi refer to JetPack4.6_R32…6.1, Both eth0 and eth1 can access to internet, so it works.

2: But, there is a little bug, the mac of eth0 and eth1 are same with each other after flash jetson nano. I have to change eth0 mac address to randow value which‘s differ from eth1 mac address.

[WayneWWW]

Can you confirm that even the extra ethernet interface is reading the mac address from the eeprom?

[848145724]

the extra ethernet interface is reading the mac address from the eeprom

[    1.131549] r8168 0000:01:00.0: enabling device (0000 -> 0003)
[    1.131587] r8168 Gigabit Ethernet driver 8.045.08-NAPI loaded
[    1.131690] tegradc tegradc.1: dpd enable lookup fail:-19
[    1.145598] r8168 0000:01:00.0 (unnamed net_device) (uninitialized): Invalid ethernet address 00:00:00:00:00:00, trying device tree node
[    1.145764] r8168 0000:01:00.0 (unnamed net_device) (uninitialized): Found valid ethernet address 48:b0:2d:3e:4f:ad from device tree
[    1.146366] r8168: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[    1.146432] r8168  Copyright (C) 2017  Realtek NIC software team <nicfae@realtek.com>
                This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>.
                This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>.
[    1.146590] r8168 0000:02:00.0: enabling device (0000 -> 0003)
[    1.146613] r8168 Gigabit Ethernet driver 8.045.08-NAPI loaded
[    1.161491] tegradc tegradc.0: nominal-pclk:148351648 parent:148350781 div:1.0 pclk:148350781 146868084~161703244
[    1.162994] tegradc tegradc.1: probed
[    1.167827] r8168 0000:02:00.0 (unnamed net_device) (uninitialized): Invalid ethernet address 00:00:00:00:00:00, trying device tree node
[    1.167898] r8168 0000:02:00.0 (unnamed net_device) (uninitialized): Found valid ethernet address 48:b0:2d:3e:4f:ad from device tree
[    1.168308] r8168: This product is covered by one or more of the following patents: US6,570,884, 

And the whole dmesg in dmesg.log file
dmesg.log (66.2 KB)

[WayneWWW]

Then it has nothing wrong. Eth0 is reading the mac from eeprom too.
Maybe you can let eth1 use a random one.

[848145724]

So, how to distinguish eth0 from eth1 in drivers.
1: In dmesg log, hard mac address of eth0 and eth1 are same with each other
2: when I set eth0 soft mac address use a random value in /etc/network/interfaces.d/eth0, then eth0 frequently can not get IP address or get ip address very slowly.

So, How to resolve this problem? That means : how to distinguish eth0 from eth1 in r8168 drivers when eth0 and eth1 share same pcie controller pcie@1003000?

[WayneWWW]

I would suggest you can contact realtek and ask for their suggestion first.

Actually, there is firmware installed on the on-board ethernet port. And such tool is also provided by Realtek.

Jetson Nano Developer Kit Ethernet Firmware Update

Hi all, we’ve released a utility that will update the Ethernet firmware programmed on Jetson Nano Developer Kits manufacturer prior to 5/15 (week 21 of 2019). The updated firmware corrects the following behaviors with the Ethernet port: Enable the RJ45 activity/link LED's Use the board's unique MAC address The utility determines whether your system needs it’s firmware updated and applies new firmware only if required. The week of manufacture can be seen in characters 4-7 of your developer …

So maybe that is not a good option to arbitrarily change the on-board ethernet mac address.

[848145724]

Hi , WayneWWW:
First, the NIC 8111G hasn’t eeprom, so the firmware you provided can’t be used.
Second, random mac address just for test in local network.
Finally, we will contact realtek for some experiences about similar problems.
Thank you very much.