Slow SD card access speed (read+write) with Jetson Nano production module

Hi jetson_user,

Sorry, not clear enough. I meant below properties under sdmmc3 DT.

cap-mmc-highspeed;
cap-sd-highspeed;
sd-uhs-sdr104;
sd-uhs-sdr50;
sd-uhs-sdr25;
sd-uhs-sdr12;

Please take this one as what I said as “patch in #11”.

Also, if the benchmark tool is GUI based, please share the screenshot or the result.

btw, could you also try if running jetson_clocks would enhance the performance?

Hi WayneWWW,

Please find attached the outputs in terms of images (the names of the images are in accordance with the test conditions , like with and without the uhs patch for sdmmc3 and ios and benchmark outputs).

We already run jetson_clocks by default at startup, and the cpu and gpu frequencies are already fixed to max.

Please let me know your opinion.
OutputsWithAndWithoutUHSEnabled.zip (439 KB)

Hi jetson_user,

Could you also share the test result of using iozone?

./iozone -ecI -+n -L64 -S32 -s64m -r512k -i0 -i1 -l8 -u8 -m -t8 -F file1 file2 file3 file4 file5 file6 file7 file8

Hi WayneWWW,

This benchmark does not seem to have support for aarch64. Do you have specific link or version of iozone that I should install? I looked for one with aarch64 support and found an rpm file, which I am figuring out how to install. Am I proceeding in the wrong direction?

Besides, could you please let me know the purpose of this benchmark meanwhile? The thing is we are running out of time (deadline to fix this is coming Monday) and would need the most effective method to debug. Since the benchmark suite I am using (gnome-disks) is already indicative enough that the SD card access speeds are slow,and also our observation when we copy huge files to the SD card.

Hi WayneWWW,

Please find attached the outputs of the iozone command you gave. This is for both with the patch case, i.e, all uhs speeds enabled, and without. Please let me know if this is right.
iozone_output_without_all_uhs_speeds_enabled.txt (2 KB)
iozone_output_with_all_uhs_speeds_enabled.txt (2 KB)

Hi WayneWWW,

Do you have any updates on this one yet?

Hi jetson_user,

Please try with this patch. It is verified by other customer too.

diff --git a/kernel-dts/tegra210-porg-p3448-common.dtsi b/kernel-dts/tegra210-porg-p3448-common.dtsi
index a73561e..4c214ba 100644
--- a/kernel-dts/tegra210-porg-p3448-common.dtsi
+++ b/kernel-dts/tegra210-porg-p3448-common.dtsi
@@ -274,7 +274,6 @@
                mmc-ddr-1_8v;
                mmc-ocr-mask = <3>;
                uhs-mask = <0x0>;
-               max-clk-limit = <400000>;
                tap-delay = <3>;
        };

Although max-clk-limit is set to 204MHz from the soc dt, it is re-written to be restricted at 40KHz in the platform dtsi.

Hi WayneWWW,

Thanks for your information.

I remove max-clk-limit is in dtsi, and the clk output below.

$cat /sys/kernel/debug/mmc1/clock
9600000

Hi jetson_user,

Do you use dmesg -w command?
is it to show “Tuning done, restoring the best tap value :91” ?
Thank you!

Karis

Hi WayneWWW,

Thank you for the solution! We checked in the TX2 kernel and device tree source code (on which we did not have this slow SD card access issue) and found that the max-clk-limit was set to <208000000> for sdr104. Do you recommend this? If no, we shall just use the patch you provided, i.e., just delete the max-clk-limit.

Hi Karis,

I tested with the above solution and got clock speed as 204000000. Also, my dmesg -w output gives:

[    1.649105] mmc1: Tuning correction cannot be applied
[    1.649127] mmc1: hw tuning done ...
[    1.649177] mmc1: new ultra high speed SDR104 SDXC card at address e624
[    1.649475] mmcblk1: mmc1:e624 SN64G 59.5 GiB

Please find attached the complete dmesg output.
dmesg-w.txt (49.5 KB)

Hi jetson_user,

Thanks for your reply.

Sorry! I am not to explain clear.

I find a problem on our nano, if I use dmesg -w and unplug SDcard, it shows “Tuning done, restoring the best tap value :91” ,I would like to know, do you have the same problem.

Thank you!

Karis

Hi Karis,

Sorry didn’t get you earlier. Yes I have the same problem. I get the below logs with dmesg -w:

[  146.018945] mmc1: card e624 removed
[  146.019720] sdhci-tegra sdhci-tegra.2: Tuning done, restoring the best tap value : 23
[  146.075067] sdhci-tegra sdhci-tegra.2: Tuning done, restoring the best tap value : 23

The “Tuning done, restoring the best tap value : 23” logs are repeated.

Also, are you using a production version or dev version?

Hi jetson_user,

Thank you for your test.

The logs are repeated,We have the same problem.

I use our carrier board and nano module of emmc.

Hi jetson_user,

Please use my patch instead. I guess the speed problem is resolved,right?

Hi WayneWWW,

Yes the speed problem got resolved. Thanks!

I was just wondering whether there might be any regressions, especially after Karis’s observation above.

Hi WayneWWW,

We ran into some issues because of which I had to reopen this ticket, sorry. The thing is my dtb changes take into effect only after a hard reboot (power off + on). But when I do a soft reboot, the dtb changes related to SD card speed are not reflected and they show some older dtb values. For instance, when I do a soft reboot, and then check the ios, I get the following output:

cat /sys/kernel/debug/mmc1/ios
clock:		50000000 Hz
vdd:		21 (3.3 ~ 3.4 V)
bus mode:	2 (push-pull)
chip select:	0 (don't care)
power mode:	2 (on)
bus width:	2 (4 bits)
timing spec:	2 (sd high-speed)
signal voltage:	0 (3.30 V)
driver type:	0 (driver type B)

But I have changed the dtb already before flashing, so this should not be the case. Instead what I should get as an output is the following, which I also get with a hard reboot (power off+on).

cat /sys/kernel/debug/mmc1/ios    
clock:		204000000 Hz
vdd:		21 (3.3 ~ 3.4 V)
bus mode:	2 (push-pull)
chip select:	0 (don't care)
power mode:	2 (on)
bus width:	2 (4 bits)
timing spec:	6 (sd uhs SDR104)
signal voltage:	1 (1.80 V)
driver type:	0 (driver type B)

It seems an old dtb is loaded from somewhere which I need to prevent. How do I do this? For the soft reboot case, I also observe that the SD card access speeds are actually slower 4 times (like 23 MBps vs 85 MBps on a hard reboot), when tested with the gnome-disks benchmarking utility that I mentioned before.

Hi WayneWWW,

Any updates on this?

Hi.
This sounds a known issue. Please refer to below

Next release would have this patch.

Actually, when you hit any new issue, you should file a new topic instead of reopen the old issue.

Hi WayneWWW,

I moved the discussion to a new topic: Jetson Nano SD card enters back to high speed mode instead of uhs mode after soft reboot. The solution provided above does not solve the issue. Basically it is for a different issue - the person loses his dtb changes on a reboot, which is not the same in my case. In my case the SD card switches to a low(er) speed mode (details in the next forum) on soft reboot and it is a coincidence that the speed it switches to are the same as before my dtb changes, hence the confusion. I can always get back to the high speed mode (as intended by the dtb changes) by a hard reboot, and I have no clue why this behaviour is different in both the reboot cases.

Besides, the changes in patch provided in the link in the answer above is already present in the the L4T sources (32.2) I have.

Updated another patch for similar speed issue we just resolved in your new post . Please take a look.

Thanks.