Jetson TK1 as SPI Slave

Okay, up front, I do understand that this is not the most straightforward thing to do in a linux environment. I’ve read many forum and mailing list threads bemoaning that a generalized SPI slave solution is essentially impossible given the way the linux kernel is put together (or at least that’s what they seemed to be implying, perhaps I misunderstood). My hope is that my particular situation is constrained/simple enough that I may be able to kludge something together that works in this special case. Any advice whatsoever will be greatly appreciated, I promise.

The K1 TRM, in Section 35 (SPI), indicates that receiving in slave mode is indeed possible, and even makes reference to associated FIFO buffers. I embarrassingly must admit near total ignorance as to how one might begin attempting to use/find them in linux, though.

I have a very simple device which is periodically spitting out a short bitstream on an SPI port. I would like to listen to it. The following are true:

  • The device emits 16 bytes (128 bits) at 15 MHz, along with appropriate CS and CLK signals.
  • That burst happens at about 100 kHz (though I can adjust this if necessary).

The goal is to get these bytes into something like a callback handler, preferably in user space. I’m not against getting my hands dirty and writing a kernel module, but it’s been a long time since I took that class, so I’d need a little advice in that regard as well, if that turns out to be the way to go.

Many thanks in advance,

{c}

Some minor updates on my struggle, slogging through the process…

(Posted here in the vain hope that perhaps some kind soul will notice me, and deign to take pity on my ineptitude…)

Thanks to the wonderfully detailed description provided by NeuroBots ( https://devtalk.nvidia.com/default/topic/818922/?comment=4485967 ), I was able to both learn many things, and also to successfully set up the existing master mode driver, which I have been scrutinizing extensively.

One interesting (troubling?) occurrence that I’ve noticed, though it doesn’t seem to prevent it from working, is that unloading and reloading spi-tegra114.ko logs a warning:

Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662666] ------------[ cut here ]------------
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662682] WARNING: at drivers/base/dd.c:272 driver_probe_device+0x23c/0x384()
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662688] Modules linked in: spi_tegra114(O+) spidev(O) dm_crypt bnep rfcomm rtc_as3722 bluetooth rfkill parport_pc parport nvhost_vi
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662730] CPU: 0 PID: 1726 Comm: insmod Tainted: G           O 3.10.24-aeye-19.3.8 #3
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662752] [<c001625c>] (unwind_backtrace+0x0/0x13c) from [<c0012b04>] (show_stack+0x18/0x1c)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662766] [<c0012b04>] (show_stack+0x18/0x1c) from [<c0061564>] (warn_slowpath_common+0x5c/0x74)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662778] [<c0061564>] (warn_slowpath_common+0x5c/0x74) from [<c0061630>] (warn_slowpath_null+0x24/0x2c)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662789] [<c0061630>] (warn_slowpath_null+0x24/0x2c) from [<c03d8e40>] (driver_probe_device+0x23c/0x384)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662801] [<c03d8e40>] (driver_probe_device+0x23c/0x384) from [<c03d6eec>] (bus_for_each_drv+0x60/0x94)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662811] [<c03d6eec>] (bus_for_each_drv+0x60/0x94) from [<c03d8b94>] (device_attach+0x7c/0x90)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662821] [<c03d8b94>] (device_attach+0x7c/0x90) from [<c03d7fb4>] (bus_probe_device+0x8c/0xb0)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662831] [<c03d7fb4>] (bus_probe_device+0x8c/0xb0) from [<c03d6374>] (device_add+0x524/0x610)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662843] [<c03d6374>] (device_add+0x524/0x610) from [<c0442c90>] (spi_add_device+0xcc/0x17c)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662857] [<c0442c90>] (spi_add_device+0xcc/0x17c) from [<c0443c60>] (spi_register_master+0x328/0x658)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662874] [<c0443c60>] (spi_register_master+0x328/0x658) from [<be90e4c4>] (tegra_spi_probe+0x62c/0x6e8 [spi_tegra114])
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662898] [<be90e4c4>] (tegra_spi_probe+0x62c/0x6e8 [spi_tegra114]) from [<c03d8d44>] (driver_probe_device+0x140/0x384)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662909] [<c03d8d44>] (driver_probe_device+0x140/0x384) from [<c03d9068>] (__driver_attach+0x94/0x98)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662919] [<c03d9068>] (__driver_attach+0x94/0x98) from [<c03d6e3c>] (bus_for_each_dev+0x68/0x9c)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662929] [<c03d6e3c>] (bus_for_each_dev+0x68/0x9c) from [<c03d82f4>] (bus_add_driver+0x1dc/0x280)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662940] [<c03d82f4>] (bus_add_driver+0x1dc/0x280) from [<c03d997c>] (driver_register+0x80/0x148)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662949] [<c03d997c>] (driver_register+0x80/0x148) from [<c0008650>] (do_one_initcall+0xdc/0x188)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662962] [<c0008650>] (do_one_initcall+0xdc/0x188) from [<c00bd510>] (load_module+0xcf8/0x10c8)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662975] [<c00bd510>] (load_module+0xcf8/0x10c8) from [<c00bdb54>] (SyS_finit_module+0x15c/0x198)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662987] [<c00bdb54>] (SyS_finit_module+0x15c/0x198) from [<c000ec40>] (ret_fast_syscall+0x0/0x30)
Feb 22 04:34:38 tegra-ubuntu kernel: [  381.662993] ---[ end trace 969bf576b7b748d3 ]---

Dutifully consulting drivers/base/dd.c:272, we find that during a probing phase there’s a device list with nothing in it to be probed.

WARN_ON(!list_empty(&dev->devres_head));

Sure, no big deal, seems like a reasonable non-fatal warning. However, somewhat more troubling is that the kernel has apparently been flagged as “Tainted.”

Since I am quite certain that I haven’t actually loaded any proprietary modules, I did a bit of reading, and confirmed that

MODULE_LICENSE("GPL v2");

(the last line in spi-tegra114.c) is indeed supposed to be one of the recognized acceptable license strings (see http://lxr.free-electrons.com/source/include/linux/module.h#L181 ). But I also noted that, when tainted, the message is suppose to include the offending license string, presumably to assign shame and blame.

But in this case, the string reads:

Tainted: G           O

Now, that “G” looks familiar (remember it was supposed to be “GPL v2”), but what happened to the rest of it?!

The only explanation I’ve yet been able to think of is that someone might be stomping on memory somewhere, which is a rather terrifying thought… (Clearly, I’d be happy to be proven wrong, please. No one wants memory stomping.)

{c}

Hello, CC:
That’s interesting.
It seems that you’ve made spi-tegra114 a loadable module.
Have you ever tried the following test?

  1. Repeat insmod spi-tegra114.ko and rmmod spi-tegra114 (without any SPI operation), and see whether the kernel prints those error information? (I’ve tried with my local building and it seems OK.)
  2. insmod xxx, do some SPI operations, and rmmod xxx, and insmod again. Then the upper message appears? That may indicate memory corruption issues.

br
ChenJian

Indeed. I reproduced it just now with the following sequence:

  1. Reboot.

  2. dmesg (excerpted, clearly showing “Not tainted”)

[   10.125560] spi-tegra114 spi-tegra114.0: registered master spi0
[   10.142453] spi spi0.0: setup 8 bpw, ~cpol, cpha, 25000000Hz
[   10.142552] spi spi0.0: setup mode 1, 8 bits/w, 25000000 Hz max --> 0
[   10.142713] ------------[ cut here ]------------
[   10.142745] WARNING: at drivers/base/dd.c:272 driver_probe_device+0x23c/0x384()
[   10.142756] Modules linked in: spidev spi_tegra114(+) parport_pc parport nvhost_vi
[   10.142804] CPU: 3 PID: 325 Comm: systemd-udevd Not tainted 3.10.24 #4
[   10.142841] [<c001625c>] (unwind_backtrace+0x0/0x13c) from [<c0012b04>] (show_stack+0x18/0x1c)
[   10.142866] [<c0012b04>] (show_stack+0x18/0x1c) from [<c0061540>] (warn_slowpath_common+0x5c/0x74)
[   10.142888] [<c0061540>] (warn_slowpath_common+0x5c/0x74) from [<c006160c>] (warn_slowpath_null+0x24/0x2c)
[   10.142907] [<c006160c>] (warn_slowpath_null+0x24/0x2c) from [<c03d8994>] (driver_probe_device+0x23c/0x384)
[   10.142931] [<c03d8994>] (driver_probe_device+0x23c/0x384) from [<c03d6a40>] (bus_for_each_drv+0x60/0x94)
[   10.142950] [<c03d6a40>] (bus_for_each_drv+0x60/0x94) from [<c03d86e8>] (device_attach+0x7c/0x90)
[   10.142967] [<c03d86e8>] (device_attach+0x7c/0x90) from [<c03d7b08>] (bus_probe_device+0x8c/0xb0)
[   10.142983] [<c03d7b08>] (bus_probe_device+0x8c/0xb0) from [<c03d5ec8>] (device_add+0x524/0x610)
[   10.143004] [<c03d5ec8>] (device_add+0x524/0x610) from [<c04427e4>] (spi_add_device+0xcc/0x17c)
[   10.143027] [<c04427e4>] (spi_add_device+0xcc/0x17c) from [<c04437b4>] (spi_register_master+0x328/0x658)
[   10.143056] [<c04437b4>] (spi_register_master+0x328/0x658) from [<be81de94>] (tegra_spi_probe+0x338/0x524 [spi_tegra114])
[   10.143128] [<be81de94>] (tegra_spi_probe+0x338/0x524 [spi_tegra114]) from [<c03d8898>] (driver_probe_device+0x140/0x384)
[   10.143147] [<c03d8898>] (driver_probe_device+0x140/0x384) from [<c03d8bbc>] (__driver_attach+0x94/0x98)
[   10.143163] [<c03d8bbc>] (__driver_attach+0x94/0x98) from [<c03d6990>] (bus_for_each_dev+0x68/0x9c)
[   10.143178] [<c03d6990>] (bus_for_each_dev+0x68/0x9c) from [<c03d7e48>] (bus_add_driver+0x1dc/0x280)
[   10.143195] [<c03d7e48>] (bus_add_driver+0x1dc/0x280) from [<c03d94d0>] (driver_register+0x80/0x148)
[   10.143214] [<c03d94d0>] (driver_register+0x80/0x148) from [<c0008650>] (do_one_initcall+0xdc/0x188)
[   10.143237] [<c0008650>] (do_one_initcall+0xdc/0x188) from [<c00bd4b4>] (load_module+0xcf8/0x10c8)
[   10.143260] [<c00bd4b4>] (load_module+0xcf8/0x10c8) from [<c00bdaf8>] (SyS_finit_module+0x15c/0x198)
[   10.143279] [<c00bdaf8>] (SyS_finit_module+0x15c/0x198) from [<c000ec40>] (ret_fast_syscall+0x0/0x30)
[   10.143292] ---[ end trace 2557fdfe4ef84043 ]---
[   10.143628] spi-tegra114 spi-tegra114.0: registered child spi0.0
  1. sudo rmmod spi_tegra114

  2. sudo insmod /lib/modules/3.10.24/kernel/drivers/spi/spi-tegra114.ko
    (and yes, uname -r does yield “3.10.24”)

  3. dmesg (excerpted, this time with “Tainted: G W”)

[  106.312614] spi-tegra114 spi-tegra114.0: registered master spi0
[  106.326682] spi spi0.0: setup 8 bpw, ~cpol, cpha, 25000000Hz
[  106.326792] spi spi0.0: setup mode 1, 8 bits/w, 25000000 Hz max --> 0
[  106.326990] ------------[ cut here ]------------
[  106.327021] WARNING: at drivers/base/dd.c:272 driver_probe_device+0x23c/0x384()
[  106.327037] Modules linked in: spi_tegra114(+) dm_crypt bnep rfcomm bluetooth rfkill rtc_as3722 spidev parport_pc parport nvhost_vi [last unloaded: spi_tegra114]
[  106.327160] CPU: 2 PID: 2184 Comm: insmod Tainted: G        W    3.10.24 #4
[  106.327191] [<c001625c>] (unwind_backtrace+0x0/0x13c) from [<c0012b04>] (show_stack+0x18/0x1c)
[  106.327216] [<c0012b04>] (show_stack+0x18/0x1c) from [<c0061540>] (warn_slowpath_common+0x5c/0x74)
[  106.327231] [<c0061540>] (warn_slowpath_common+0x5c/0x74) from [<c006160c>] (warn_slowpath_null+0x24/0x2c)
[  106.327244] [<c006160c>] (warn_slowpath_null+0x24/0x2c) from [<c03d8994>] (driver_probe_device+0x23c/0x384)
[  106.327261] [<c03d8994>] (driver_probe_device+0x23c/0x384) from [<c03d6a40>] (bus_for_each_drv+0x60/0x94)
[  106.327272] [<c03d6a40>] (bus_for_each_drv+0x60/0x94) from [<c03d86e8>] (device_attach+0x7c/0x90)
[  106.327280] [<c03d86e8>] (device_attach+0x7c/0x90) from [<c03d7b08>] (bus_probe_device+0x8c/0xb0)
[  106.327297] [<c03d7b08>] (bus_probe_device+0x8c/0xb0) from [<c03d5ec8>] (device_add+0x524/0x610)
[  106.327312] [<c03d5ec8>] (device_add+0x524/0x610) from [<c04427e4>] (spi_add_device+0xcc/0x17c)
[  106.327329] [<c04427e4>] (spi_add_device+0xcc/0x17c) from [<c04437b4>] (spi_register_master+0x328/0x658)
[  106.327358] [<c04437b4>] (spi_register_master+0x328/0x658) from [<be914e94>] (tegra_spi_probe+0x338/0x524 [spi_tegra114])
[  106.327394] [<be914e94>] (tegra_spi_probe+0x338/0x524 [spi_tegra114]) from [<c03d8898>] (driver_probe_device+0x140/0x384)
[  106.327407] [<c03d8898>] (driver_probe_device+0x140/0x384) from [<c03d8bbc>] (__driver_attach+0x94/0x98)
[  106.327419] [<c03d8bbc>] (__driver_attach+0x94/0x98) from [<c03d6990>] (bus_for_each_dev+0x68/0x9c)
[  106.327432] [<c03d6990>] (bus_for_each_dev+0x68/0x9c) from [<c03d7e48>] (bus_add_driver+0x1dc/0x280)
[  106.327442] [<c03d7e48>] (bus_add_driver+0x1dc/0x280) from [<c03d94d0>] (driver_register+0x80/0x148)
[  106.327458] [<c03d94d0>] (driver_register+0x80/0x148) from [<c0008650>] (do_one_initcall+0xdc/0x188)
[  106.327505] [<c0008650>] (do_one_initcall+0xdc/0x188) from [<c00bd4b4>] (load_module+0xcf8/0x10c8)
[  106.327530] [<c00bd4b4>] (load_module+0xcf8/0x10c8) from [<c00bdaf8>] (SyS_finit_module+0x15c/0x198)
[  106.327552] [<c00bdaf8>] (SyS_finit_module+0x15c/0x198) from [<c000ec40>] (ret_fast_syscall+0x0/0x30)
[  106.327568] ---[ end trace 2557fdfe4ef84044 ]---
[  106.328155] spi-tegra114 spi-tegra114.0: registered child spi0.0
  1. Scratch head in puzzlement…

To be absolutely clear: No SPI operations were performed. Just boot, then rmmod, then insmod, that’s it.

Hello, CC:
What’s SDK version you are using?
Any change you’ve made with native SDK?
I tested insmod/rmmod for several times in my local building and no such issue happens.

br
Chenjian

Hmm. How odd. Thanks so much for helping me test this. Much appreciated!

Well, let’s see. What could I have screwed up here…

To the best of my recollection (and my bash history), this board should have been built with:

  • Tegra124_Linux_R19.3.0_armhf.tbz2
  • Tegra_Linux_Sample-Root-Filesystem_R19.3.0_armhf.tbz2

Prepared according to the instructions (https://developer.nvidia.com/sites/default/files/akamai/mobile/docs/l4t_quick_start_guide.txt), and flashed to use uboot:

sudo ./flash.sh -S 14GiB -L bootloader/ardbeg/u-boot.bin jetson-tk1 mmcblk0p1

Then in situ, built the kernel with:

CONFIG_SPI_TEGRA114=m
CONFIG_SPI_SPIDEV=m

According to the instructions here: https://devtalk.nvidia.com/default/topic/762653/?comment=4390235, with the sole exception that subsequent threads (e.g., https://devtalk.nvidia.com/default/topic/766303/jetson-tk1/-customkernel-the-grinch-19-3-8-for-jetson-tk1-developed/) indicated that u-boot allows it to be installed locally without having to re-flash from the external host.

Let me know if you see anything obviously missing or wrong.

I should probably confirm all these steps by wiping the board and starting clean, but I don’t have the time (or the heart) right at the moment, since things are at least functional enough that I’m able to get work done.

Thanks again,

{c}

Hello, cc:
If that error matters, would you please try latest 21.4 release?

I cannot get same issue in my local R21 building.

br
ChenJian