ENC28J60 Ethernet Module Interfacing with Jetson Nano

[mustafa.ozcelikors]

I would like to interface with a ENC28J60 module (ENC28J60 Ethernet Module Communication Interface Adaptor :Elecrow bazaar, Make your making Electronic modules projects easy) using SPI1 interface on the Jetson Nano.

So far, I have done the following:

1. Selected GPIO11 / PZ0 as my interrupt pin, don’t know how to configure a pin as interrupt though.

2. Added TEGRA_GPIO(Z, 0) to gpio-input section on gpio-default devicetree.

3. Configured following on pinmux:

      pz0 {
    		nvidia,pins = "pz0";
    		nvidia,function = "rsvd1";
    		nvidia,pull = <TEGRA_PIN_PULL_DOWN>; // <TEGRA_PIN_PULL_UP>;
    		nvidia,tristate = <TEGRA_PIN_DISABLE>;
    		nvidia,enable-input = <TEGRA_PIN_ENABLE>;
    	};
   
               spi1_mosi_pc0 {
    		nvidia,pins = "spi1_mosi_pc0";
    		nvidia,function = "spi1";
    		nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
    		nvidia,tristate = <TEGRA_PIN_DISABLE>;
    		nvidia,enable-input = <TEGRA_PIN_ENABLE>;
    	};
   
    	spi1_miso_pc1 {
    		nvidia,pins = "spi1_miso_pc1";
    		nvidia,function = "spi1";
    		nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
    		nvidia,tristate = <TEGRA_PIN_DISABLE>;
    		nvidia,enable-input = <TEGRA_PIN_ENABLE>;
    	};
   
    	spi1_sck_pc2 {
    		nvidia,pins = "spi1_sck_pc2";
    		nvidia,function = "spi1";
    		nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
    		nvidia,tristate = <TEGRA_PIN_DISABLE>;
    		nvidia,enable-input = <TEGRA_PIN_ENABLE>;
    	};
   
    	spi1_cs0_pc3 {
    		nvidia,pins = "spi1_cs0_pc3";
    		nvidia,function = "spi1";
    		nvidia,pull = <TEGRA_PIN_PULL_UP>;
    		nvidia,tristate = <TEGRA_PIN_DISABLE>;
    		nvidia,enable-input = <TEGRA_PIN_ENABLE>;
    	};
   
    	spi1_cs1_pc4 {
    		nvidia,pins = "spi1_cs1_pc4";
    		nvidia,function = "spi1";
    		nvidia,pull = <TEGRA_PIN_PULL_UP>;
    		nvidia,tristate = <TEGRA_PIN_DISABLE>;
    		nvidia,enable-input = <TEGRA_PIN_ENABLE>;
    	};
   

4. Added enc28j60 devicetree as follows, similar to mcp2515 example (which works for me):

      spi@7000d400 { /* SPI 1 to 40 pin header */
    	status = "okay";
    	enc28j60: spi@0 {
    		compatible = "microchip,enc28j60";
    		reg = <0>;
    		interrupt-parent = <&gpio>;
        		interrupts = <TEGRA_GPIO(Z, 0) IRQ_TYPE_EDGE_FALLING>;  //GPIO11 corresponds to PZ0
    		nvidia,enable-hw-based-cs;
    		nvidia,rx-clk-tap-delay = <0x7>;
    		spi-max-frequency = <10000000>;
    		controller-data {
    			nvidia,cs-setup-clk-count = <0x1e>;
    			nvidia,cs-hold-clk-count = <0x1e>;
    			nvidia,rx-clk-tap-delay = <0x1f>;
    			nvidia,tx-clk-tap-delay = <0x0>;
    		};
            	};
    };
   

5. Made the module connections: (1st col. ENC28J60 module, 2nd col Jetson Nano Dev. Kit 40-pin header)

SI         19
SO         21
SCK        23
CS         24
INT        31
WOL        NC
RESET      NC
CLKOUT     NC
GND        GND
VCC        3V3

6. Enabled ENC28J60 in kernel config
   CONFIG_ENC28J60=y

NOTE: My 5V/3A supply didnt arrive yet, so I am powering Jetson Nano from a USB/0.5A.

However, I am getting the following error:

[    4.176139] enc28j60 spi0.0: enc28j60 Ethernet driver 1.02 loaded                
[    4.185134] enc28j60 spi0.0: enc28j60 chip not found                             
[    4.190129] enc28j60: probe of spi0.0 failed with error -5 

I am referencing on how this is done with a Raspberry Pi, and Jetson Nano MCP2515 interfacing example.
Is there anything I am missing that would result in the aforementioned error.
Any guidance is appreciated. Thanks.

[ShaneCCC]

Have a reference to below link to set the pin by cfg file to try.

SPI1 not work on Xavier

Please modify the pinmux for spi1 like spi2 does. file name: tegra19x-mb1-pinmux-p2888-0000-a04-p2822-0000-b01.cfg SPI1 pinmux.0x0243d040 = 0x00000055; # spi1_sck_pz3: rsvd1, pull-down, tristate-enable, input-enable, lpdr-disable pinmux.0x0243d020 = 0x00000055; # spi1_miso_pz4: rsvd1, pull-down, tristate-enable, input-enable, lpdr-disable pinmux.0x0243d058 = 0x00000055; # spi1_mosi_pz5: rsvd1, pull-down, tristate-enable, input-enable, lpdr-disable pinmux.0x0243d010 = 0x00000059; # spi1_cs0_pz6:…

[mustafa.ozcelikors]

@ShaneCCC

by cfg file to

What is the difference between setting the pins via cfg file or via devicetree?
I am able to run other SPI devices with my devicetree configuration. Via devicetree the configuration is more readable.
I shared my spi1 pin conf above.
Ccould you please tell me where might be the problem? Thanks.

[ShaneCCC]

If you can run others device that tell the pin configure without problem.
You may need to check if the ENC28J60 power status.

[mustafa.ozcelikors]

It turned out it was a power problem. I fixed it by supplying 3V3 to the module.

However I’m now getting kernel panic about in 2-3 seconds after I plug in ethernet cable to the module.
Anybody knows why this might be happening? I am attaching the driver I modified to use interrupts - based on MCP2515 example in this forum to make interrupts work - enc28j60.c (45.6 KB) The module does not work without it.

[   48.329864] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 0   
[   48.337123] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.9.140-l4t-r32.4.2+ge3c8d1
[   48.345400] Hardware name: NVIDIA Jetson Nano
[   48.353150] Call trace:                                                          
[   48.355621] [<ffffff800808c0a8>] dump_backtrace+0x0/0x178                        
[   48.361037] [<ffffff800808c244>] show_stack+0x24/0x30                            
[   48.366102] [<ffffff800844e758>] dump_stack+0x98/0xc0                            
[   48.371168] [<ffffff80081bc5e0>] panic+0x118/0x27c                               
[   48.375975] [<ffffff800817eeb8>] watchdog_nmi_enable+0x0/0x60                    
[   48.381734] [<ffffff800817e064>] watchdog_timer_fn+0x8c/0x2a0                    
[   48.387494] [<ffffff8008136c78>] __hrtimer_run_queues+0xf8/0x350                 
[   48.393512] [<ffffff80081375b0>] hrtimer_interrupt+0xa8/0x1d8                    
[   48.399273] [<ffffff8008bc79c0>] tegra210_timer_isr+0x38/0x48                    
[   48.405033] [<ffffff800811fd98>] __handle_irq_event_percpu+0x60/0x280            
[   48.411487] [<ffffff800811ffe0>] handle_irq_event_percpu+0x28/0x70               
[   48.417679] [<ffffff8008120078>] handle_irq_event+0x50/0x80                      
[   48.423265] [<ffffff8008123c4c>] handle_fasteoi_irq+0xc4/0x1a0                   
[   48.429109] [<ffffff800811edcc>] generic_handle_irq+0x34/0x50                    
[   48.434867] [<ffffff800811f49c>] __handle_domain_irq+0x6c/0xc0                   
[   48.440711] [<ffffff8008080d2c>] gic_handle_irq+0x54/0xa8                        
[   48.446121] [<ffffff8008082c28>] el1_irq+0xe8/0x194                              
[   48.451013] [<ffffff8008b70274>] cpuidle_enter_state+0xec/0x380                  
[   48.456945] [<ffffff8008b7057c>] cpuidle_enter+0x34/0x48                         
[   48.462269] [<ffffff800810ff3c>] call_cpuidle+0x44/0x68                          
[   48.467506] [<ffffff800811027c>] cpu_startup_entry+0x18c/0x210                   
[   48.473354] [<ffffff8008092844>] secondary_start_kernel+0x13c/0x160              
[   48.479631] [<0000000080f271a4>] 0x80f271a4                                      
[   48.483827] SMP: stopping secondary CPUs                                         
[   48.487870] Kernel Offset: disabled                                              
[   48.491374] Memory Limit: none                                                   
[   48.517301] Rebooting in 5 seconds.. 

Here is how dmesg looks:

root@jetson-nano-qspi-sd:~# dmesg | grep eth0                                       
[    6.344103] net eth0: enc28j60 driver registered                                 
[   16.793892] net eth0: link down                                                  
[   16.797052] net eth0: multicast mode                                             
[   16.800750] net eth0: multicast mode                                             
[   16.804384] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready                   
[   16.843407] net eth0: multicast mode                                             
[   16.847004] net eth0: multicast mode                                             
[   16.850574] net eth0: multicast mode

Here is what I modified in enc28j60.c:

diff --git a/drivers/net/ethernet/microchip/enc28j60.c b/drivers/net/ethernet/microchip/enc28j60.c
index 0a26b11ca8f6..bbfb9236693d 100644
--- a/drivers/net/ethernet/microchip/enc28j60.c
+++ b/drivers/net/ethernet/microchip/enc28j60.c
@@ -30,6 +30,10 @@
 #include <linux/spi/spi.h>
 #include <linux/of_net.h>
 
+#include <linux/of_gpio.h>
+#include <asm-generic/gpio.h>
+
 #include "enc28j60_hw.h"
 
 #define DRV_NAME	"enc28j60"
@@ -1555,6 +1559,10 @@ static int enc28j60_probe(struct spi_device *spi)
 	const void *macaddr;
 	int ret = 0;
 
+	int gpio;
+	struct device_node *np = spi->dev.of_node;
+
 	if (netif_msg_drv(&debug))
 		dev_info(&spi->dev, DRV_NAME " Ethernet driver %s loaded\n",
 			DRV_VERSION);
@@ -1566,6 +1574,42 @@ static int enc28j60_probe(struct spi_device *spi)
 	}
 	priv = netdev_priv(dev);
 
+	/* If GPIO based interrupt is passed in device tree */
+	gpio = of_get_named_gpio(np, "interrupts", 0);
+	if (gpio < 0) {
+		dev_err(&spi->dev, "failed to get GPIO from device tree\n");
+	}
+
+	else {
+		ret = gpio_request(gpio, "enc28j60-gpio");
+		if (ret) {
+			dev_err(&spi->dev, "failed to request GPIO %d\n", gpio);
+			return ret;
+		}
+
+		ret = gpio_export(gpio, false);
+		if (ret) {
+			dev_err(&spi->dev, "failed to export GPIO %d\n", gpio);
+			return ret;
+		}
+
+		ret = gpio_direction_input(gpio);
+		if (ret) {
+			dev_err(&spi->dev, "failed to set pin to input state\n");
+			return -EINVAL;
+		}
+
+		/* IRQ setup */
+		ret = gpio_to_irq(gpio);
+		if (ret < 0) {
+			dev_err(&spi->dev, "failed to map GPIO to IRQ: %d\n", ret);
+			return -EINVAL;
+		}
+
+		spi->irq = ret;
+	}
+
 	priv->netdev = dev;	/* priv to netdev reference */
 	priv->spi = spi;	/* priv to spi reference */
 	priv->msg_enable = netif_msg_init(debug.msg_enable,

More info:

root@jetson-nano-qspi-sd:~# cat /proc/interrupts  | grep enc                        
299:          0          0          0          0      GPIO 200 Edge      enc28j60

Ifconfig (eth0 is the ENC28J60):

root@jetson-nano-qspi-sd:~# ifconfig                                                
eth0      Link encap:Ethernet  HWaddr 96:5F:7B:84:22:60                             
          UP BROADCAST MULTICAST  MTU:1500  Metric:1                                
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0                        
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0                      
          collisions:0 txqueuelen:1000                                              
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)                                    
          Interrupt:43                                                              
                                                                                    
eth1      Link encap:Ethernet  HWaddr 00:04:4B:E7:25:42                             
          inet addr:192.168.1.24  Bcast:192.168.1.255  Mask:255.255.255.0           
          inet6 addr: fe80::204:4bff:fee7:2542/64 Scope:Link                        
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1                        
          RX packets:79 errors:0 dropped:0 overruns:0 frame:0                       
          TX packets:57 errors:0 dropped:0 overruns:0 carrier:0                     
          collisions:0 txqueuelen:1000                                              
          RX bytes:10114 (9.8 KiB)  TX bytes:6871 (6.7 KiB)                         
          Interrupt:150 Base address:0xe000

I checked netdev: enc28j60 kernel panic fix. - Patchwork , no use.

[ShaneCCC]

Looks like system hang and trigger the watchdog.
Could you disable the watchdog to confirm it.

[mustafa.ozcelikors]

How do I do that?
In my devicetree, I have:

    watchdog@60005100 {
		status = "okay";
	};

	tegra_wdt: watchdog@60005100 {
		status = "disabled";
	};

Disabling these does not work.
also,

root@jetson-nano-qspi-sd:~# zcat /proc/config.gz | grep CONFIG_TEGRA_WATCHDOG       
# CONFIG_TEGRA_WATCHDOG_LEGACY is not set                                           
# CONFIG_TEGRA_WATCHDOG is not set

and there is no /dev/watchdog available.

[mustafa.ozcelikors]

Even when I disable nmi_watchdog, I get the same thing:
root@jetson-nano-qspi-sd:~# echo '1' > /proc/sys/kernel/nmi_watchdog

root@jetson-nano-qspi-sd:~# [  324.331077] Kernel panic - not syncing: Watchdog detected hard LOCKU0
[  324.338519] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.9.140-l4t-r32.4.2+ge3c8d3e9030a #1
[  324.346910] Hardware name: Moz Jetson Carrier Board based on NVIDIA Jetson Nano (DT)
[  324.354761] Call trace:
[  324.357354] [<ffffff800808c0a8>] dump_backtrace+0x0/0x178
[  324.362877] [<ffffff800808c244>] show_stack+0x24/0x30
[  324.368043] [<ffffff800844e758>] dump_stack+0x98/0xc0
[  324.373214] [<ffffff80081bc5e0>] panic+0x118/0x27c
[  324.378129] [<ffffff800817eeb8>] watchdog_nmi_enable+0x0/0x60
[  324.383995] [<ffffff800817e064>] watchdog_timer_fn+0x8c/0x2a0
[  324.389862] [<ffffff8008136c78>] __hrtimer_run_queues+0xf8/0x350
[  324.395981] [<ffffff80081375b0>] hrtimer_interrupt+0xa8/0x1d8
[  324.401852] [<ffffff8008bc79d0>] tegra210_timer_isr+0x38/0x48
[  324.407724] [<ffffff800811fd98>] __handle_irq_event_percpu+0x60/0x280
[  324.414279] [<ffffff800811ffe0>] handle_irq_event_percpu+0x28/0x70
[  324.420571] [<ffffff8008120078>] handle_irq_event+0x50/0x80
[  324.426260] [<ffffff8008123c4c>] handle_fasteoi_irq+0xc4/0x1a0
[  324.432204] [<ffffff800811edcc>] generic_handle_irq+0x34/0x50
[  324.438057] [<ffffff800811f49c>] __handle_domain_irq+0x6c/0xc0                                   
[  324.444002] [<ffffff8008080d2c>] gic_handle_irq+0x54/0xa8                                        
[  324.449505] [<ffffff8008082c28>] el1_irq+0xe8/0x194                                              
[  324.454505] [<ffffff8008b70250>] cpuidle_enter_state+0xb8/0x380                                  
[  324.460541] [<ffffff8008b7058c>] cpuidle_enter+0x34/0x48                                         
[  324.465963] [<ffffff800810ff3c>] call_cpuidle+0x44/0x68                                          
[  324.471292] [<ffffff800811027c>] cpu_startup_entry+0x18c/0x210                                   
[  324.477243] [<ffffff8008092844>] secondary_start_kernel+0x13c/0x160                              
[  324.483615] [<0000000080f271a4>] 0x80f271a4

How do I proceed here? The same driver seems to work with Raspberry Pi.

[ShaneCCC]

It could be the enc28j60 driver cause the kernel panic. Any further message before the watchdog detect the CPU hang?

root@jetson-nano-qspi-sd:~# [ 324.331077] Kernel panic - not syncing: Watchdog detected hard LOCKU0

[mustafa.ozcelikors]

No message at all, unfortunately.

Do you have a guess why this driver stalls kernel operation in Jetson Nano, whereas it works for Raspberry Pi?

The following happens when I disable hard lockup in watchdog.

root@jetson-nano-qspi-sd:~# [   91.435082] INFO: rcu_sched detected stalls on CPUs/:
[   91.440801]  0-...: (5 GPs behind) idle=ff3/2/0 softirq=3408/3448 fqs=2480       
[   91.447876]  (detected by 2, t=5252 jiffies, g=-12, c=-13, q=8)                  
[   91.453952] Task dump for CPU 0:                                                 
[   91.457269] swapper/0       R  running task        0     0      0 0x00000002     
[   91.464507] Call trace:                                                          
[   91.467078] [<ffffff8008086334>] __switch_to+0x9c/0xb8                           
[   91.472336] [<ffffff8008b6f2b8>] cpuidle_enter_state+0xa0/0x380                  
[   91.478365] [<ffffff8008b6f60c>] cpuidle_enter+0x34/0x48                         
[   91.483783] [<ffffff800810ff3c>] call_cpuidle+0x44/0x68                          
[   91.489111] [<ffffff800811027c>] cpu_startup_entry+0x18c/0x210                   
[   91.495069] [<ffffff8008f1df1c>] rest_init+0x84/0x90                             
[   91.500151] [<ffffff80098e0b38>] start_kernel+0x370/0x388                        
[   91.505659] [<ffffff80098e0204>] __primary_switched+0x80/0x94

The following makes me think it is interrupt configuration / handling related:

[  324.395981] [<ffffff80081375b0>] hrtimer_interrupt+0xa8/0x1d8
[  324.401852] [<ffffff8008bc79d0>] tegra210_timer_isr+0x38/0x48
[  324.407724] [<ffffff800811fd98>] __handle_irq_event_percpu+0x60/0x280
[  324.414279] [<ffffff800811ffe0>] handle_irq_event_percpu+0x28/0x70
[  324.420571] [<ffffff8008120078>] handle_irq_event+0x50/0x80
[  324.426260] [<ffffff8008123c4c>] handle_fasteoi_irq+0xc4/0x1a0
[  324.432204] [<ffffff800811edcc>] generic_handle_irq+0x34/0x50
[  324.438057] [<ffffff800811f49c>] __handle_domain_irq+0x6c/0xc0                                   
[  324.444002] [<ffffff8008080d2c>] gic_handle_irq+0x54/0xa8                                        
[  324.449505] [<ffffff8008082c28>] el1_irq+0xe8/0x194 

I even tried replacing the driver with the driver from latest upstream kernel (5.7), still the same result. This also makes me think the problem is Jetson Nano interrupt configuration related. In the final stage, I have following configuration:

TEGRA_GPIO(Z, 0) in gpio-input section.

In pinmux dts,

enc28j60_int_pin: pz0 {
				nvidia,pins = "pz0";
				nvidia,function = "rsvd1";
				nvidia,pull = <TEGRA_PIN_PULL_NONE>;/*<TEGRA_PIN_PULL_UP>;*/
				nvidia,tristate = <TEGRA_PIN_DISABLE>;
				nvidia,enable-input = <TEGRA_PIN_ENABLE>;
};

In SPI devicetree,

enc28j60: spi@0 {
			compatible = "microchip,enc28j60";
			reg = <0>;
        	interrupts = <&gpio TEGRA_GPIO(Z, 0) IRQ_TYPE_EDGE_FALLING>;
			nvidia,enable-hw-based-cs;
			nvidia,rx-clk-tap-delay = <0x7>;
			spi-max-frequency = <10000000>;
			pinctrl-names = "default";
            pinctrl-0 = <&enc28j60_int_pin>;
			controller-data {
				nvidia,cs-setup-clk-count = <0x1e>;
				nvidia,cs-hold-clk-count = <0x1e>;
				nvidia,rx-clk-tap-delay = <0x1f>;
				nvidia,tx-clk-tap-delay = <0x0>;
			};
         };

Suggested devicetree (For i.MX) is given as follows in kernel docs (https://www.kernel.org/doc/Documentation/devicetree/bindings/net/microchip%2Cenc28j60.txt):

        ssp2: ssp@80014000 {
                compatible = "fsl,imx28-spi";
                pinctrl-names = "default";
                pinctrl-0 = <&spi2_pins_b &spi2_sck_cfg>;

                enc28j60: ethernet@0 {
                        compatible = "microchip,enc28j60";
                        pinctrl-names = "default";
                        pinctrl-0 = <&enc28j60_pins>;
                        reg = <0>;
                        interrupt-parent = <&gpio3>;
                        interrupts = <3 IRQ_TYPE_EDGE_FALLING>;
                        spi-max-frequency = <12000000>;
                };
        };

        pinctrl@80018000 {
                enc28j60_pins: enc28j60_pins@0 {
                        reg = <0>;
                        fsl,pinmux-ids = <
                                MX28_PAD_AUART0_RTS__GPIO_3_3    /* Interrupt */
                        >;
                        fsl,drive-strength = <MXS_DRIVE_4mA>;
                        fsl,voltage = <MXS_VOLTAGE_HIGH>;
                        fsl,pull-up = <MXS_PULL_DISABLE>;
                };
        };

Any ideas to proceed are appreciated and would be beneficial to the community as well.

[WayneWWW]

Hi,

Sorry to jump in.

For current status, we could only suggest you to

1. check from vendor what is the requirement for this driver
2. if you cannot see any other error log from driver, maybe you could add some debug print in driver by yourself to see which line has error.

Do you have a guess why this driver stalls kernel operation in Jetson Nano, whereas it works for Raspberry Pi?

I think this comparison may not be fully correct since we don’t know what kernel environment is running on Pi.

[mozcelikors]

I have identified that the problem results in enc28j60_irq function being constantly triggered, which creates deferred work (schedule_work) on each entry. This eventually drains system of resources and causes hrtimer based panic. enc28j60_irq is being entered constantly is the issue. I still think this might be caused by devicetree irq configuration.

I have observed the INT pin with oscilloscope, I observe following signal:

However I think the noise present in the line is not enough to cause any repeated interrupts.