M.2 / PCIe Google Coral in Ubuntu?

Do I need a special driver to use the PCIe (single lane on dev board) in Ubuntu 16.04?

I have a Google Coral TPU on the M.2 port, and am having install issues with this:

https://coral.ai/docs/m2/get-started/

Hi bpmosie,

We haven’t tried Google Coral TPU with Jetson Nano, not sure if can work, besides, the current L4T SW for Jetson Nano is Ubuntu 18.04, may I know which version you’re using on device?

The usb Coral does work. I haven’t tested the PCI module. Based on those instructions it should. What issues are you having?

Hello bpmosie,

Do you have any updates?

In my case, the coral M.2 pci worked as follows.

sudo apt install pkg-config usbutils

sudo apt upgrade -y libstdc++6

sudo apt --reinstall install gasket-dkms libedgetpu1-std

sudo reboot

and enjoy ^^

python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg

1 Like

Thanks, mazzingkaizer – I will try. The USB version works well, and it would be great to have the M.2 version working.

Hi mazzing, following your instructions I get the following error with the M.2 coral

ValueError: Failed to load delegate from libedgetpu.so.1

any idea?
thanks

Same issue for me. It doesn’t look like the kernel modules are loading properly for me. ls /dev/apex_0 returns nothing, and I followed all the steps in the documentation.

dmesg:

[    5.907707] gasket: loading out-of-tree module taints kernel.
[    5.907789] gasket: module verification failed: signature and/or required key missing - tainting kernel
[    5.927630] apex 0000:0b:00.0: Page table init timed out
[    5.927661] apex 0000:0b:00.0: MSI-X table init timed out
[    5.928349] apex: probe of 0000:0b:00.0 failed with error -110

Is it too late to reply? I’m sorry.

I thought Gasket was an M.2 driver. Gasket installation seems wrong.

In my case, /dev/apex_0 was well verified by “lsusb” command after installing the gasket.

Install gaskets in the following order in a clean environment:

1,
echo “deb https://packages.cloud.google.com/apt coral-edgetpu-stable main” | tee /etc/apt/sources.list.d/coral-edgetpu.list

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

apt install gasket-dkms

sh -c “echo ‘SUBSYSTEM==“apex”, MODE=“0660”, GROUP=“apex”’ >> /etc/udev/rules.d/65-apex.rules”

groupadd apex

adduser $USER apex

apt-get update

apt-get install libedgetpu1-std

Good Luck.

Hi,

I am still having issues connecting the M.2 accelerator to the Jetson Nano DevKit.
I followed each instruction, here is the output for some commands :

lspci
0:02.0 PCI bridge: NVIDIA Corporation Device 0faf (rev a1)
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

lsusb (not really relevant, but hey, more info is better right?)
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 046a:000c Cherry GmbH
Bus 001 Device 003: ID 046d:c31c Logitech, Inc. Keyboard K120
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

lsmod
Module Size Used by
bnep 16562 2
fuse 103841 2
overlay 48691 0
zram 26166 4
spidev 13282 0
nvgpu 1575721 18
bluedroid_pm 13912 0
ip_tables 19441 0
x_tables 28951 1 ip_tables

modinfo gasket
filename: /lib/modules/4.9.140-tegra/updates/dkms/gasket.ko
author: Rob Springer rspringer@google.com
license: GPL v2
version: 1.1.3
description: Google Gasket driver framework
srcversion: 069B6D0F6AE12073F4EAF5D
depends:
vermagic: 4.9.140-tegra SMP preempt mod_unload modversions aarch64
parm: dma_bit_mask:int

modinfo
filename: /lib/modules/4.9.140-tegra/updates/dkms/apex.ko
author: John Joseph jnjoseph@google.com
license: GPL v2
version: 1.1
description: Google Apex driver
srcversion: 508A8A34D57322CEA287D17
alias: pci:v00001AC1d0000089Asvsdbcsci*
depends: gasket
vermagic: 4.9.140-tegra SMP preempt mod_unload modversions aarch64
parm: allow_power_save:int
parm: allow_sw_clock_gating:int
parm: allow_hw_clock_gating:int
parm: bypass_top_level:int
parm: trip_point0_temp:int
parm: trip_point1_temp:int
parm: trip_point2_temp:int
parm: hw_temp_warn1:int
parm: hw_temp_warn2:int
parm: hw_temp_warn1_en:bool
parm: hw_temp_warn2_en:bool
parm: temp_poll_interval:int

dpkg -l | grep gasket
ii gasket-dkms 1.0-10 all DKMS source for the gasket driver

uname -a
Linux jetson-desktop 4.9.140-tegra #1 SMP PREEMPT Mon Dec 9 22:47:42 PST 2019 aarch64 aarch64 aarch64 GNU/Linux

dmesg | grep pci
[ 0.967687] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.3, lane = pcie-0, function = pcie-x1
[ 0.967783] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.4, lane = pcie-1, function = pcie-x4
[ 0.967870] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.5, lane = pcie-2, function = pcie-x4
[ 0.967967] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.6, lane = pcie-3, function = pcie-x4
[ 0.968054] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.7, lane = pcie-4, function = pcie-x4
[ 0.968142] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.8, lane = pcie-5, function = xusb
[ 0.968225] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.9, lane = pcie-6, function = xusb
[ 0.977971] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
[ 0.979254] tegra-pcie 1003000.pcie: PCIE: Enable power rails
[ 0.979641] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
[ 0.983636] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
[ 1.071883] ehci-pci: EHCI PCI platform driver
[ 1.071939] ohci-pci: OHCI PCI platform driver
[ 1.406312] tegra-pcie 1003000.pcie: link 0 down, retrying
[ 1.818326] tegra-pcie 1003000.pcie: link 0 down, retrying
[ 2.231469] tegra-pcie 1003000.pcie: link 0 down, retrying
[ 2.233537] tegra-pcie 1003000.pcie: link 0 down, ignoring
[ 2.337986] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
[ 2.337992] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 2.337995] pci_bus 0000:00: root bus resource [mem 0x13000000-0x1fffffff]
[ 2.337999] pci_bus 0000:00: root bus resource [mem 0x20000000-0x3fffffff pref]
[ 2.338003] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 2.338023] pci 0000:00:02.0: [10de:0faf] type 01 class 0x060400
[ 2.338085] pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 2.338227] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 2.338328] pci 0000:01:00.0: [10ec:8168] type 00 class 0x020000
[ 2.338351] pci 0000:01:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 2.338381] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[ 2.338401] pci 0000:01:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit]
[ 2.338532] pci 0000:01:00.0: supports D1 D2
[ 2.338535] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 2.349910] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 2.349936] pci 0000:00:02.0: BAR 14: assigned [mem 0x13000000-0x130fffff]
[ 2.349940] pci 0000:00:02.0: BAR 13: assigned [io 0x1000-0x1fff]
[ 2.349946] pci 0000:01:00.0: BAR 4: assigned [mem 0x13000000-0x13003fff 64bit]
[ 2.349963] pci 0000:01:00.0: BAR 2: assigned [mem 0x13004000-0x13004fff 64bit]
[ 2.349979] pci 0000:01:00.0: BAR 0: assigned [io 0x1000-0x10ff]
[ 2.349987] pci 0000:00:02.0: PCI bridge to [bus 01]
[ 2.349990] pci 0000:00:02.0: bridge window [io 0x1000-0x1fff]
[ 2.349996] pci 0000:00:02.0: bridge window [mem 0x13000000-0x130fffff]
[ 2.350212] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt
[ 2.350215] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ 2.350220] pcie_pme 0000:00:02.0:pcie001: service driver pcie_pme loaded
[ 2.350299] aer 0000:00:02.0:pcie002: service driver aer loaded

dmesg | grep gasket yields nothing…

Does anyone have any idea how I could resolve this? Seems the EdgeTPU is not even powered up…

I think you’re right that it’s not powered up for whatever reason. It should show up in lspci even if the module didn’t exist, and it looks like it does. You could try restarting the nano with fewer peripherals connected or another power supply and see if that fixes it. Given others in the thread have reported it working, you could compare configurations (eg. model number of your nano and carrier board).

Hi~ sam

Did you install L4T on Ubuntu Linux version 16.04 or 18.04 ? I am sure you did.
If not, try installing on Ubuntu Linux version 16.04 or 18.04
I did it at Ubuntu 18.04 from 1 to 8.

It’ll be all right. ^^

Hi,

The problem has been solved.
I swapped out Seeed Studio carrier boards, and the accelerator instantly got recognised and worked.
So it seems there is a problem with the carrier board, I will try to investigate further wat the problem is is Seeed Studio does not agree with replacing it.

Thanks for the help,
Greetings.

Hello everyone, I got 10 brand new Jetson Nano developer kit and installed on each one an M.2 / PCIe Google Coral. What I found is that 3 of them have a faulty PCIe.
I tested every Nano Devkit using the same SD card without modifications between the tests and the same Coral. It’s obvious that there is an hardware fault on a high percentage of the boards.
I need either a way to fix the board, or a way to purchase the devkits being sure to get the working ones.

We are equipped for professional electronics jobs so I’d prefer to find a way to fix the boards.
Any help would be appreciated.

Thanks

Hi @mazzingkaizer I have the M.2 B+E key and the dual edge M.2 E key which both fit in the Jetson nano’s M.2 slot (B01 4gig).

I have been banging my head with this for a few days!

I am using the latest SD image (https://developer.nvidia.com/jetson-nano-sd-card-image)

uname -a
Linux nano 4.9.201-tegra #1 SMP PREEMPT Fri Jan 15 14:41:02 PST 2021 aarch64 aarch64 aarch64 GNU/Linux

lspci -x | grep 089a
01:00.0 System peripheral: Device 1ac1:089als

ls /dev/apex_0
/dev/apex_0

The above does seem to look good. (following Get started with the M.2 or Mini PCIe Accelerator | Coral)

Running the tflite/python/examples/classification example from https://github.com/google-coral/tflite:

python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg

ERROR
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py”, line 152, in load_delegate
delegate = Delegate(library, options)
File “/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py”, line 111, in init
raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “classify_image.py”, line 122, in
main()
File “classify_image.py”, line 99, in main
interpreter = make_interpreter(args.model)
File “classify_image.py”, line 73, in make_interpreter
{‘device’: device[0]} if device else {})
File “/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py”, line 155, in load_delegate
library, str(e)))
ValueError: Failed to load delegate from libedgetpu.so.1

when running the example listed on the Coral getting started

$ python3 examples/classify_image.py \
> --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
> --labels test_data/inat_bird_labels.txt \
> --input test_data/parrot.jpg

ERROR

Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py”, line 152, in load_delegate
delegate = Delegate(library, options)
File “/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py”, line 111, in init
raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “examples/classify_image.py”, line 84, in
main()
File “examples/classify_image.py”, line 61, in main
interpreter = make_interpreter(*args.model.split(’@’))
File “/usr/lib/python3/dist-packages/pycoral/utils/edgetpu.py”, line 66, in make_interpreter
delegates = [load_edgetpu_delegate({‘device’: device} if device else {})]
File “/usr/lib/python3/dist-packages/pycoral/utils/edgetpu.py”, line 42, in load_edgetpu_delegate
return tflite.load_delegate(_EDGETPU_SHARED_LIB, options or {})
File “/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py”, line 155, in load_delegate
library, str(e)))
ValueError: Failed to load delegate from libedgetpu.so.1

I did follow you’re setups also:

sudo apt install pkg-config usbutils

sudo apt upgrade -y libstdc++6

sudo apt --reinstall install gasket-dkms libedgetpu1-std

If ANYONE gets an M.2 Coral working on the Jetson, please do let us know. It seems Google has put out a product with very little end to end testing on SBC that have an M.2 port!

@sam.sterckval I purchased my Coral M.2 TPU’s from Seeed as well, what do you mean by carrier board?

1 Like

Take a look at my reply to @mazzingkaizer .

Hi @Gavan1 - I’m in the same situation as you with nearly the same errors. I started with a dual M.2 E Key, which didn’t even show up when I ran lspci -nn. I then ordered the single M.2 B+E key and while that shows up in the system, I get the same exact Failed to load delegate from libedgetpu.so.1 error when trying to run the same example you listed.

No real solution here, but it’s not just you.

Edit about an hour later:

I started looking digging deeper at the issues where other people had seen this issue pop up. It looks like the solution was in GitHub issue google-coral/edgetpu#96. In short, you want to add pcie_aspm=off to the APPEND line in /boot/extlinux/extlinux.conf.

My dual TPU is still not showing up. But this lets my single TPU work well.

2 Likes

Dual m2 also not even showing up for me

I filed an issue on the google-coral/edgetpu project, where they shared that the E Key slot on the NVIDIA Jetson Nano is for wireless only.

That message comes from Google, not from me, but it seems to be a logical explanation for why the Dual TPU M.2 E Key Coral accelerator isn’t visible when running lspci.

It doesn’t make sense on how a M.2 E Key slot could be for wireless only. That’s something I’ll have to poke at as I thought the E Key slot was supposed to support 2× PCIe ×1, USB 2.0, I2C, SDIO, UART and PCM - which should be enough.

1 Like

I’ve just managed to get my Dual TPU to show-up and work on my Jetson Nano 4GB.

I’ve basically followed the steps from Get started with the M.2 or Mini PCIe Accelerator | Coral

I’ve also had to add apply this Get started with the M.2 or Mini PCIe Accelerator | Coral - basically edit the /boot/extlinux/extlinux.conf and add pcie_aspm=off but I’ve aslo added gasket.dma_bit_mask=32 on the same line.

Update: even though I’ve managed to run inference I don’t seem to be able to use the “dual” TPU, meaning the second TPU.