SPI clock max speed

Hi,

I’m on T5000 module with R38.4 Jetpack 7.1

I use a custom board and SPI3 (which is spidev2.0).

It works as expected (tested in hardware loopback and I receive what I send).

I use the code from Linus Torvald to test SPI: linux/tools/spi/spidev_test.c at master · torvalds/linux · GitHub

Works like a charm at 10Mhz for example:

sudo ./spidev_test -v -s 10000000 -D /dev/spidev2.0

However I want to try to go with higher clock rate for example 40MHz:

sudo ./spidev_test -v -s 40000000 -D /dev/spidev2.0

However it seem to be caped at 23.82353MHz see here the clocks:

root@thor:~# cat /sys/kernel/debug/bpmp/debug/clk/spi3/parent
pllp_out0
root@thor:~# cat /sys/kernel/debug/bpmp/debug/clk/spi3/possible_parents
pllp_out0 osc
root@thor:~# cat /sys/kernel/debug/bpmp/debug/clk/spi3/rate
23823530

I verified with a scope that the clock is limited to 23.8MHz and it is indeed the case.

I tried to change clock to osc and it’s max clock is even less (about 3.1MHz):

root@thor:~# echo “osc” > /sys/kernel/debug/bpmp/debug/clk/spi3/parent
root@thor:~# cat /sys/kernel/debug/bpmp/debug/clk/spi3/rate
3176471

How to go at 40MHz or even 50MHz on the SPI?
What are the limitations?

Regards.

Here’s values from following test on Thor dev kit where we can confirm the configured BPMP SPI controller clock from debugfs; but I don’t have a scope to verify the actual frequency on the wire.

sudo su

echo 1 | tee /sys/kernel/debug/bpmp/debug/clk/spi3/mrq_rate_locked

grep . /sys/kernel/debug/bpmp/debug/clk/spi3/max_rate 

echo 45000000 | tee /sys/kernel/debug/bpmp/debug/clk/spi3/rate
grep . /sys/kernel/debug/bpmp/debug/clk/spi3/rate

echo 50000000 | tee /sys/kernel/debug/bpmp/debug/clk/spi3/rate
grep . /sys/kernel/debug/bpmp/debug/clk/spi3/rate

echo 56250000 | tee /sys/kernel/debug/bpmp/debug/clk/spi3/rate
grep . /sys/kernel/debug/bpmp/debug/clk/spi3/rate

echo 101250000 | tee /sys/kernel/debug/bpmp/debug/clk/spi3/rate
grep . /sys/kernel/debug/bpmp/debug/clk/spi3/rate

Result

1
101250000
45000000
45000000
50000000
45000000
56250000
50625000
101250000
101250000

Please check the max_rate for spi3 and configure it for the rate

# cat /sys/kernel/debug/bpmp/debug/clk/spi3/max_rate
# echo <max_rate> > /sys/kernel/debug/bpmp/debug/clk/spi3/rate

Hi,

Yes I can change the rate by command and it “seem” applyed because when I readback it seem good (rounded to the nearest value).
For example with 40MHz:

root@thor:~# echo 40000000 > /sys/kernel/debug/bpmp/debug/clk/spi3/rate
root@thor:~# grep . /sys/kernel/debug/bpmp/debug/clk/spi3/rate
36818182

However when I launch a transfer via the program, which try to change the rate to the desired one via IOCTL it seem something goes wrong in the driver, selecting the wrong speed.

You can see the revelent code here:

If I launch the program for 40Mhz (be carrefull that if you previouslly launched program with same frequency, it seem that it do not modify the rate again as it seem to change value only if it is not the current one):

thor@thor:~$ sudo ./spidev_test -v -s 40000000 -D /dev/spidev2.0
spi mode: 0x0
bits per word: 8
max speed: 40000000 Hz (40000 kHz)
TX | FF FF FF FF FF FF 40 00 00 00 00 95 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF F0 0D |…@…|
RX | FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF |…|

Then the rate is now 23.8MHz:

root@thor:~# cat /sys/kernel/debug/bpmp/debug/clk/spi3/rate
23823530

You can test yourself by compiling following those lines:

wget https://raw.githubusercontent.com/torvalds/linux/refs/heads/master/tools/spi/spidev_test.c
gcc spidev_test.c -o spidev_test

Then launch with:

sudo ./spidev_test -v -s 40000000 -D /dev/spidev2.0

Also please note again that it all works changing rate via the test program until we try a rate greater than the 23.8MHz limit

spidev_test -s 40000000 requests a maximum SPI transfer speed of 40 MHz through the Linux spidev ioctl interface. It does not directly read, write, or change /sys/kernel/debug/bpmp/debug/clk/spi3/rate; that BPMP debugfs node is separate from the userspace spidev API.

Usage: ./spidev_test [options]
 version 21
  -D --device   device to use (default /dev/spidev0.0)
  -s --speed    max speed (Hz)
  -g --length   transaction length in bytes. Multiple lengths can be comma separated.
  -z --debug    enable debug mode. Specifying more than once to increase the verboisty.
  -p --pattern  choose pattern type. Available patterns are:
                0: sequential bytes
                1: even bytes
                2: odd bytes
                3: reverse sequential bytes
                4: random bytes with crc
  -F prefix     Dump rx/tx buffer as raw to file. With no prefix specified, current pid will be used
  -f --file     Pattern file. User defined pattern. Not allowed with -p option.
                Pattern file must be in space seperated bytes in hex without
                "0x" prefix. Sample data: 01 AA 23 5F 3C 12
  -d --delay    delay (usec). For SPI master this will be the minimum delay 
                the driver waits before initiating each transfer. For slave
                this indicates the max delay that the slave waits before it times out.
  -E --expect   Expected transaction length in bytes.
                To test variable length data transfer feature.
                Without this option, spidev_test will assume it as normal transfer
                and will validate the actual transferred length with requested transaction length.
                Value should be less than or equal to the transaction length.
  -b --bpw      bits per word 
  -H --cpha     clock phase
  -O --cpol     clock polarity
  -L --lsb      least significant bit first
  -C --cs-high  chip select active high
  -c --cnt      continuous pause mode 1: start or 2:stop controller.
  -n --nruns    Number of times to repeat the test. -1 is infinite.
  -u --udelay   delay b/w initiating each transfer (usec). Multiple delays can be comma separated
  -r --receive  only receive data
  -t --transmit only transmit data
  -v --minsize  variable length packet start
  -V --maxsize  variable length packet end
  -W --waitb4   wait for a keystroke before the first transfer
  -w --stoperr  stop on all errors
  -P --packet   packet mode

Examples:
To transfer 100 messages of 30 bytes each with random bytes,
  spidev_test -D/dev/spidev0.0 -s18000000 -n100 -g30 -p4

To transfer 100 messages of sizes 8 and 3986 with delay of 2ms and 8ms respectively,
  spidev_test -D/dev/spidev0.0 -s18000000 -n100 -g8,3968 -u2000,8000

To transfer all bytes from user defined pattern file,
  spidev_test -D/dev/spidev0.0 -s18000000 -f/path/to/patternfile

To test variable length feature,
  spidev_test -D/dev/spidev0.0 -s18000000 -g 30 -E 20

Return codes:
 0   successfull transfer. This would mean that what was
     transferred was actually received. Note that a success
     would make sense only when the spi is run in a loopback
     type configuration, i.e., a miso-mosi loopback or a
     master-slave loopback
 1   Invalid argument
 5   I/O error
 6   Data mismatch error. The transfer happened but there is 
     mismatch in tx and rx pattern. Again this makes sense only
     for loopback as explained above
 22  Invalid argument

Yes I know that, I even posted the IOCTL code which requested the max SPI clock.

However when you use this IOCTL it also change the current rate in debugfs, I tested it and verified it too, that’s why after looking with my scope I was surprised that the frequency was such far away from 40MHz, but the rate in debugfs reflected the frequency I saw on my scope.

Try the program and look at the rate which change to get as close as possible to the requested frequency. And note that it seem clamped to a maximum of 23.8Mhz for no reason

How to change SPI speed then to get 40MHz via IOCTL or user space call?
What is the normal path to get 40MHz, because using debugfs is not

Regards

With spi40 herein, I believe we can lock and hold the SPI3 clock at 45 MHz, and successfully request 40 MHz via spidev ioctls. But the transfer still returns all FF, so the remaining problem does not appear to be the userspace speed request path. The problem may be in MB1 pinmux configuration for the SPI2 pads, which are currently configured as rsvd1 and tristated rather than assigned to SPI2.

Experimental thor-spi2-pinmux.patch to attempt and resolve that is below.

cd ./jp7/Linux_for_Tegra/source/kernel/kernel-noble/tools/spi

Create spi40.c

#include <errno.h>
#include <fcntl.h>
#include <linux/spi/spidev.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <unistd.h>

static void dump_buf(const char *label, const uint8_t *buf, size_t len)
{
    size_t i;

    printf("%s", label);
    for (i = 0; i < len; i++) {
        printf("%02X ", buf[i]);
    }
    printf(" |");
    for (i = 0; i < len; i++) {
        unsigned char c = buf[i];
        putchar((c >= 32 && c <= 126) ? c : '.');
    }
    printf("|\n");
}

int main(int argc, char **argv)
{
    const char *dev = "/dev/spidev2.0";
    int fd;
    uint8_t mode = SPI_MODE_0;
    uint8_t bits = 8;
    uint32_t req_speed = 40000000;
    uint32_t rd_speed = 0;
    uint8_t tx[8] = { '1', '2', '3', '4', '5', '6', '7', '8' };
    uint8_t rx[8] = { 0 };

    struct spi_ioc_transfer tr = {
        .tx_buf = (unsigned long)tx,
        .rx_buf = (unsigned long)rx,
        .len = sizeof(tx),
        .speed_hz = 40000000,
        .delay_usecs = 0,
        .bits_per_word = 8,
        .cs_change = 0,
        .tx_nbits = 0,
        .rx_nbits = 0,
        .word_delay_usecs = 0,
        .pad = 0,
    };

    if (argc > 1) {
        dev = argv[1];
    }

    fd = open(dev, O_RDWR);
    if (fd < 0) {
        fprintf(stderr, "open(%s) failed: %s\n", dev, strerror(errno));
        return 1;
    }

    if (ioctl(fd, SPI_IOC_WR_MODE, &mode) == -1) {
        fprintf(stderr, "SPI_IOC_WR_MODE failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    if (ioctl(fd, SPI_IOC_RD_MODE, &mode) == -1) {
        fprintf(stderr, "SPI_IOC_RD_MODE failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    if (ioctl(fd, SPI_IOC_WR_BITS_PER_WORD, &bits) == -1) {
        fprintf(stderr, "SPI_IOC_WR_BITS_PER_WORD failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    if (ioctl(fd, SPI_IOC_RD_BITS_PER_WORD, &bits) == -1) {
        fprintf(stderr, "SPI_IOC_RD_BITS_PER_WORD failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    if (ioctl(fd, SPI_IOC_WR_MAX_SPEED_HZ, &req_speed) == -1) {
        fprintf(stderr, "SPI_IOC_WR_MAX_SPEED_HZ failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    if (ioctl(fd, SPI_IOC_RD_MAX_SPEED_HZ, &rd_speed) == -1) {
        fprintf(stderr, "SPI_IOC_RD_MAX_SPEED_HZ failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    printf("device: %s\n", dev);
    printf("mode: %u\n", mode);
    printf("bits per word: %u\n", bits);
    printf("requested max speed: %u Hz\n", req_speed);
    printf("readback max speed:  %u Hz\n", rd_speed);
    printf("transfer speed_hz:   %u Hz\n", tr.speed_hz);

    dump_buf("TX: ", tx, sizeof(tx));

    if (ioctl(fd, SPI_IOC_MESSAGE(1), &tr) < 1) {
        fprintf(stderr, "SPI_IOC_MESSAGE failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    dump_buf("RX: ", rx, sizeof(rx));

    close(fd);
    return 0;
}

You can test with:

cd <jp7>/Linux_for_Tegra/source/kernel/kernel-noble/tools/spi$ 
sudo sh -c '
echo 1 > /sys/kernel/debug/bpmp/debug/clk/spi3/mrq_rate_locked
echo 45000000 > /sys/kernel/debug/bpmp/debug/clk/spi3/rate
cat /sys/kernel/debug/bpmp/debug/clk/spi3/rate
'

sudo ./spi40 /dev/spidev2.0
sudo cat /sys/kernel/debug/bpmp/debug/clk/spi3/rate


thor-spi2-pinmux.patch

cat > /tmp/thor-spi2-pinmux.patch <<'EOF'
diff --git a/bootloader/tegra264-mb1-bct-pinmux-p3834-xxxx-p4071-0008.dtsi b/bootloader/tegra264-mb1-bct-pinmux-p3834-xxxx-p4071-0008.dtsi
--- a/bootloader/tegra264-mb1-bct-pinmux-p3834-xxxx-p4071-0008.dtsi
+++ b/bootloader/tegra264-mb1-bct-pinmux-p3834-xxxx-p4071-0008.dtsi
@@ -1519,42 +1519,42 @@
 			spi2_sck_pcc7 {
 				nvidia,pins = "spi2_sck_pcc7";
-				nvidia,function = "rsvd1";
+				nvidia,function = "spi2_sck";
 				nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
-				nvidia,tristate = <TEGRA_PIN_ENABLE>;
+				nvidia,tristate = <TEGRA_PIN_DISABLE>;
 				nvidia,enable-input = <TEGRA_PIN_DISABLE>;
 				nvidia,drv-type = <TEGRA_PIN_1X_DRIVER>;
 				nvidia,e-io-od = <TEGRA_PIN_DISABLE>;
 				nvidia,e-lpbk = <TEGRA_PIN_DISABLE>;
 			};

 			spi2_miso_pdd0 {
 				nvidia,pins = "spi2_miso_pdd0";
-				nvidia,function = "rsvd1";
+				nvidia,function = "spi2_din";
 				nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
-				nvidia,tristate = <TEGRA_PIN_ENABLE>;
-				nvidia,enable-input = <TEGRA_PIN_DISABLE>;
+				nvidia,tristate = <TEGRA_PIN_DISABLE>;
+				nvidia,enable-input = <TEGRA_PIN_ENABLE>;
 				nvidia,drv-type = <TEGRA_PIN_1X_DRIVER>;
 				nvidia,e-io-od = <TEGRA_PIN_DISABLE>;
 				nvidia,e-lpbk = <TEGRA_PIN_DISABLE>;
 			};

 			spi2_mosi_pdd1 {
 				nvidia,pins = "spi2_mosi_pdd1";
-				nvidia,function = "rsvd1";
+				nvidia,function = "spi2_dout";
 				nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
-				nvidia,tristate = <TEGRA_PIN_ENABLE>;
+				nvidia,tristate = <TEGRA_PIN_DISABLE>;
 				nvidia,enable-input = <TEGRA_PIN_DISABLE>;
 				nvidia,drv-type = <TEGRA_PIN_1X_DRIVER>;
 				nvidia,e-io-od = <TEGRA_PIN_DISABLE>;
 				nvidia,e-lpbk = <TEGRA_PIN_DISABLE>;
 			};

 			spi2_cs0_n_pdd2 {
 				nvidia,pins = "spi2_cs0_n_pdd2";
-				nvidia,function = "rsvd1";
+				nvidia,function = "spi2_cs0";
 				nvidia,pull = <TEGRA_PIN_PULL_DOWN>;
-				nvidia,tristate = <TEGRA_PIN_ENABLE>;
+				nvidia,tristate = <TEGRA_PIN_DISABLE>;
 				nvidia,enable-input = <TEGRA_PIN_DISABLE>;
 				nvidia,drv-type = <TEGRA_PIN_1X_DRIVER>;
 				nvidia,e-io-od = <TEGRA_PIN_DISABLE>;
 				nvidia,e-lpbk = <TEGRA_PIN_DISABLE>;
 			};
EOF

Hi XamiX,

$ sudo ./spidev_test -D /dev/spidev0.0 -s 40000000 -v -p "HelloWorld123456789abcdef"
spi mode: 0x0
bits per word: 8
max speed: 40000000 Hz (40000 kHz)
TX | 48 65 6C 6C 6F 57 6F 72 6C 64 31 32 33 34 35 36 37 38 39 61 62 63 64 65 66 __ __ __ __ __ __ __  |HelloWorld123456789abcdef|
RX | 48 65 6C 6C 6F 57 6F 72 6C 64 31 32 33 34 35 36 37 38 39 61 62 63 64 65 66 __ __ __ __ __ __ __  |HelloWorld123456789abcdef|

$ sudo cat /sys/kernel/debug/bpmp/debug/clk/spi1/rate
23823530

It seems SPI working with 23.8MHz.

To get a real 40 MHz SPI on your custom board:

  • Keep the SPI traces short and direct (SoC ↔ device, no long detours).
  • Don’t add extra muxes or level shifters unless you really need them.
  • Configure the pinmux for SPI pins including the recommended POR pad settings(drive, slew, pulls).

Please also get a scope to check the signal when you perform SPI transaction with 10MHz, 20MHz, 30MHz, 40MHz respectively.

Hi,

@whitesscott, the fact that you use debugfs and set frequency via rate and also set mrq_rate_locked to 1 lock the rate!

So yes it is not updated anymore by ioctl call and use the one you settled via debugfs.

The problem is not this one, the problem is:

How to set SPI speed greater than 23823530Hz via IOCTL?
Using debugfs we can go with greater speed for example setting it to 27MHz:

root@thor:~# echo 27000000 > /sys/kernel/debug/bpmp/debug/clk/spi3/rate
root@thor:~# cat /sys/kernel/debug/bpmp/debug/clk/spi3/rate
27000000

And with my scope I also validate that it is working at 27MHz and can read and write via hardware loopack.

HOWEVER as also seen by @KevinFFF when using IOCTL (spidev_test) to set speed, it is clamped to 23823530, it is impossible to get a higher speed without using debugfs

It seem there is a problem in driver or somewhere which limit settling SPI speed in user land via IOCTL.

The question is simple:

How to set SPI speed greater than 23823530 without touching anything in debugfs and using only userland code?

This limitation to 23823530 is likely a bug as it is possible to go with higher speed using debugfs!

Regards

I think I have found the reason:

In the driver’s probe function, if spi-max-frequency isn’t specified in the device tree, it defaults to 25 MHz:

	if (of_property_read_u32(pdev->dev.of_node, "spi-max-frequency",
				 &host->max_speed_hz))
		host->max_speed_hz = 25000000; /* 25MHz */

And it’s my case, so it limit it to 25MHz which lead to the nearest value of 23823530Hz:

root@thor:/sys/kernel/tracing# echo 25000000 > /sys/kernel/debug/bpmp/debug/clk/spi3/rate
root@thor:/sys/kernel/tracing# cat /sys/kernel/debug/bpmp/debug/clk/spi3/rate
23823530

So I think I just need to set spi-max-frequency in my device tree.

EDIT:
It was already defined at 50MHz but for spidev:

spi@810c440000 {
			compatible = "nvidia,tegra234-spi\0nvidia,tegra210-spi";
			status = "okay";
			#address-cells = <0x01>;
			#size-cells = <0x00>;
			reg = <0x81 0xc440000 0x00 0x10000>;
			interrupts = <0x00 0xa5 0x04>;
			dma-coherent;
			dma-names = "rx\0tx";
			dmas = <0x133 0x0e 0x0e 0x80e 0x133 0x0e 0x18 0x80e>;
			iommus = <0x05 0x80e>;
			clocks = <0x02 0x2c 0x02 0x16>;
			clock-names = "spi";
			assigned-clocks = <0x02 0x2c>;
			assigned-clock-parents = <0x02 0x16>;
			resets = <0x02 0x26>;
			reset-names = "spi";
			phandle = <0x2af>;
                       spi-max-frequency = <0x2faf080>; // This was added to define max clock to 50MHz

			prod-settings {
				#prod-cells = <0x04>;

				prod {
					prod = <0x00 0x194 0x80000000 0x00>;
				};
			};

			spi@0 {
				compatible = "tegra-spidev";
				reg = <0x00>;
				spi-max-frequency = <0x2faf080>; // Here it is 50MHz already defined
			};
		};

So by default it is specified in spi@0 subblocks but must be defined in the parent to work with the correct driver.

After adding it, all seem to work correctly.

Regards