when i copy big file to ssd, which is connected to the nano through usb, its speed less than 10m/s, while in pc the same ssd can up to 200m/s. the ssd format is ext4，and the nano Firmware is jetpack 4.4.1. I also try to use older firmware or the other ssd，things not change still. the speed is too slow, I use dd test tool get the same results. however when i use the DISK software’s benchmark, it have the right speed 200-300M/s.
Hi,
Could you check dmesg and see if the device is enumerated as highspeed(USB2) instead of superspeed(USB3)?
I think it is enumerated as superspeed device
[ 0.458726] iommu: Adding device 70090000.xusb to group 2
[ 0.531484] vdd-usb-vbus: supplied by vdd-5v0-sys
[ 0.531535] vdd-usb-vbus: 5000 mV
[ 0.532061] vdd-usb-vbus2: 5000 mV
[ 0.575563] usbcore: registered new interface driver usbfs
[ 0.575636] usbcore: registered new interface driver hub
[ 0.575726] usbcore: registered new device driver usb
[ 0.641359] vdd-usb-vbus2: supplied by vdd-3v3-sys
[ 0.643574] vdd-usb-hub-en: supplied by vdd-1v8-sys
[ 0.643628] vdd-usb-hub-en: 5000 mV
[ 1.353408] tegra-xusb-padctl 7009f000.xusb_padctl: TEGRA_FUSE_SKU_CALIB_0 = 0x9229c12
[ 1.353441] tegra-xusb-padctl 7009f000.xusb_padctl: TEGRA_FUSE_USB_CALIB_EXT_0 = 0x4
[ 1.355118] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.0, lane = usb2-0, function = xusb
[ 1.355298] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.1, lane = usb2-1, function = xusb
[ 1.355424] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.2, lane = usb2-2, function = xusb
[ 1.355634] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.3, lane = pcie-0, function = pcie-x1
[ 1.355750] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.4, lane = pcie-1, function = pcie-x4
[ 1.355879] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.5, lane = pcie-2, function = pcie-x4
[ 1.355985] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.6, lane = pcie-3, function = pcie-x4
[ 1.356093] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.7, lane = pcie-4, function = pcie-x4
[ 1.356196] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.8, lane = pcie-5, function = xusb
[ 1.356306] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.9, lane = pcie-6, function = xusb
[ 4.834671] usbcore: registered new interface driver r8152
[ 4.840224] usbcore: registered new interface driver asix
[ 4.845684] usbcore: registered new interface driver ax88179_178a
[ 4.851828] usbcore: registered new interface driver cdc_ether
[ 4.857703] usbcore: registered new interface driver net1080
[ 4.863408] usbcore: registered new interface driver cdc_subset
[ 4.869372] usbcore: registered new interface driver zaurus
[ 4.875028] usbcore: registered new interface driver cdc_ncm
[ 4.918837] tegra-xusb 70090000.xusb: USB2 port 0 has OTG_CAP
[ 4.925251] tegra-xusb-padctl 7009f000.xusb_padctl: enabled OTG on UTMI pad 0
[ 4.933606] tegra-xusb 70090000.xusb: extcon 0: ffffffc0f9104000 id
[ 4.942116] usbcore: registered new interface driver uas
[ 4.947518] tegra-xusb 70090000.xusb: Firmware timestamp: 2019-10-17 15:58:59 UTC, Version: 50.25 release
[ 4.947585] tegra-xusb 70090000.xusb: xHCI Host Controller
[ 4.947615] tegra-xusb 70090000.xusb: new USB bus registered, assigned bus number 1
[ 4.948380] tegra-xusb 70090000.xusb: hcc params 0x0184f525 hci version 0x100 quirks 0x00050010
[ 4.948447] tegra-xusb 70090000.xusb: irq 61, io mem 0x70090000
[ 4.948737] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[ 4.948742] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 4.948746] usb usb1: Product: xHCI Host Controller
[ 4.948749] usb usb1: Manufacturer: Linux 4.9.140-tegra xhci-hcd
[ 4.948752] usb usb1: SerialNumber: 70090000.xusb
[ 4.949963] tegra-xusb 70090000.xusb: xHCI Host Controller
[ 4.949974] tegra-xusb 70090000.xusb: new USB bus registered, assigned bus number 2
[ 4.950054] usb usb2: We don’t know the algorithms for LPM for this host, disabling LPM.
[ 4.950230] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[ 4.950235] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 4.950239] usb usb2: Product: xHCI Host Controller
[ 4.950242] usb usb2: Manufacturer: Linux 4.9.140-tegra xhci-hcd
[ 4.950245] usb usb2: SerialNumber: 70090000.xusb
[ 5.051274] tegra-xusb-padctl 7009f000.xusb_padctl: power on UTMI pads 1
[ 5.051336] tegra-xusb-padctl 7009f000.xusb_padctl: power on UTMI pads 2
[ 5.095528] usbcore: registered new interface driver usb-storage
[ 5.101678] usbcore: registered new interface driver usbserial
[ 5.132050] usbcore: registered new interface driver xpad
[ 5.281368] usb 1-2: new high-speed USB device number 2 using tegra-xusb
[ 5.323442] usb 1-2: New USB device found, idVendor=0bda, idProduct=5411
[ 5.330171] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 5.368104] usb 1-2: Product: 4-Port USB 2.1 Hub
[ 5.372753] usb 1-2: Manufacturer: Generic
[ 5.499235] usb 1-3: new full-speed USB device number 3 using tegra-xusb
[ 5.525100] usb 1-3: New USB device found, idVendor=8087, idProduct=0a2b
[ 5.525105] usb 1-3: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 5.671706] usb 1-2.3: new full-speed USB device number 4 using tegra-xusb
[ 5.729358] usb 1-2.3: New USB device found, idVendor=24ae, idProduct=2003
[ 5.729365] usb 1-2.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 5.729369] usb 1-2.3: Product: RAPOO 5G Wireless Device
[ 5.729373] usb 1-2.3: Manufacturer: RAPOO
[ 5.855760] input: RAPOO RAPOO 5G Wireless Device as /devices/70090000.xusb/usb1/1-2/1-2.3/1-2.3:1.0/0003:24AE:2003.0001/input/input0
[ 5.927844] hid-generic 0003:24AE:2003.0001: input,hidraw0: USB HID v1.10 Keyboard [RAPOO RAPOO 5G Wireless Device] on usb-70090000.xusb-2.3/input0
[ 5.944323] input: RAPOO RAPOO 5G Wireless Device as /devices/70090000.xusb/usb1/1-2/1-2.3/1-2.3:1.1/0003:24AE:2003.0002/input/input1
[ 6.015683] hid-generic 0003:24AE:2003.0002: input,hidraw1: USB HID v1.10 Mouse [RAPOO RAPOO 5G Wireless Device] on usb-70090000.xusb-2.3/input1
[ 6.028689] usbcore: registered new interface driver usbhid
[ 6.034273] usbhid: USB HID core driver
[ 6.062940] usbcore: registered new interface driver snd-usb-audio
[ 6.579037] vdd-usb-vbus: disabling
[ 6.582530] vdd-usb-vbus2: disabling
[ 9.115821] tegra-xusb 70090000.xusb: Upgrade port 0 to USB3.0
[ 9.121661] tegra-xusb 70090000.xusb: Upgrade port 1 to USB3.0
[ 9.444204] usb 2-1: new SuperSpeed USB device number 2 using tegra-xusb
[ 9.483988] usb 2-1: New USB device found, idVendor=0bda, idProduct=0411
[ 9.490699] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 9.497831] usb 2-1: Product: 4-Port USB 3.1 Hub
[ 9.502447] usb 2-1: Manufacturer: Generic
[ 9.811616] usb 2-1.1: new SuperSpeed USB device number 3 using tegra-xusb
[ 9.836729] usb 2-1.1: New USB device found, idVendor=05a9, idProduct=f581
[ 9.843607] usb 2-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=4
[ 9.850912] usb 2-1.1: Product: USB Camera-OV580
[ 9.855528] usb 2-1.1: Manufacturer: Omnivision Technologies, Inc.
[ 9.861704] usb 2-1.1: SerialNumber: OV0001
[ 9.868372] hid-generic 0003:05A9:F581.0003: hidraw2: USB HID v1.10 Device [Omnivision Technologies, Inc. USB Camera-OV580] on usb-70090000.xusb-1.1/input2
[ 9.967610] usb 2-1.4: new SuperSpeed USB device number 4 using tegra-xusb
[ 9.992561] usb 2-1.4: New USB device found, idVendor=152d, idProduct=0576
[ 9.999439] usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 10.006744] usb 2-1.4: Product: USB to ATA/ATAPI Bridge
[ 10.011967] usb 2-1.4: Manufacturer: JMicron
[ 10.016237] usb 2-1.4: SerialNumber: 0123456789ABCDEF
[ 18.592534] usbcore: registered new interface driver btusb
[ 18.934452] input: USB Camera-OV580 as /devices/70090000.xusb/usb2/2-1/2-1.1/2-1.1:1.0/input/input4
[ 18.934626] usbcore: registered new interface driver uvcvideo
[ 20.079525] usb0: HOST MAC 0e:82:4c:e6:3e:92
[ 20.079623] usb0: MAC 0e:82:4c:e6:3e:93
[ 20.097417] l4tbr0: port 2(usb0) entered blocking state
[ 20.097422] l4tbr0: port 2(usb0) entered disabled state
[ 20.097754] device usb0 entered promiscuous mode
[ 20.105582] IPv6: ADDRCONF(NETDEV_UP): usb0: link is not ready
chao@chao-nano:~$ lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/4p, 5000M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 1: Dev 3, If 0, Class=Video, Driver=uvcvideo, 5000M
|__ Port 1: Dev 3, If 1, Class=Video, Driver=uvcvideo, 5000M
|__ Port 1: Dev 3, If 2, Class=Human Interface Device, Driver=usbhid, 5000M
|__ Port 4: Dev 4, If 0, Class=Mass Storage, Driver=uas, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/5p, 480M
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 3: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 3: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 3: Dev 3, If 0, Class=Wireless, Driver=btusb, 12M
|__ Port 3: Dev 3, If 1, Class=Wireless, Driver=btusb, 12M
Hi,
Please execute sudo jetson_clocks and try again. It maximizes the clocks of hardware blocks(CPU, GPU, …) and should bring improvement.
Hi,
I tried that before, it seems to have no improvement.
I also noticed something strange, when I use fdisk -l in Jetson Nano it shows:
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
while I check the same SSD in my PC, the value is different:
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Could it be related to this？
Hi,
We have verified USB3 throughput with USB3 pendrive and it can reach 5Gbps. A bit strange the performance is work with SSD storage. Are you able to share how you profile the throughput?
Hi,
First time I found that problem was when I was going to record ros bag through Jetson Nano, and as result it only recorded 10fps image, however my camera is 50fps.
I tried to copy a rosbag file from SD card to SSD, and I read the speed value from file UI, it shows only 10M/s.
I also use ‘dd’ cmd to test:
chao@chao-nano:~/Documents$ time dd if=/dev/zero of=test.dbf bs=4k count=500000
500000+0 records in
500000+0 records out
2048000000 bytes (2.0 GB, 1.9 GiB) copied, 100.214 s, 20.4 MB/s
real 1m40.226s
user 0m0.504s
sys 0m7.508s
And many times after I use the dd cmd to write data to SSD, or just copy file to SSD directly, the system go wrong, it will show:
“bash: cannot create temp file for here-document: Read-only file system”,
and I also cannot connect to Jetson nano throush ssh that time， shows:
“ssh_exchange_identification: read: Connection reset by peer”.
The size of the file that I write into useing dd cmd is 1.1GB, less than 2.0GB. It seems during the write process something wrong.
However, I use the benchmark tool in Disks software, the result is different:
Sample Size 10.0MiB(10,486,760bytes)
Average Read Rate 303.6 MB/s(100 samples)
Average Write Rate 364.6 MB/s(100 samples)
Average Access Time 0.20 msec(1000 samples)
And I use dd cmd to test read speed:
chao@chao-nano:~/Documents$ time dd if=test.dbf of=/dev/null bs=4k
274432+0 records in
274432+0 records out
1124073472 bytes (1.1 GB, 1.0 GiB) copied, 4.51455 s, 249 MB/s
real 0m4.526s
user 0m0.160s
sys 0m2.436s
During the test I also measure the voltage and current, the voltage is around 5.2V and the current is around 0.5A
Hi,
So you use Jetson-Nano-with-SD-slot module? Not with emmc?
Hi,
yes, does that affect the i/o speed?
Hi,
Since the rootfs is on SD card, sdio may impact the result.
Could you check throughput via iozone3? Please refer to
Jetson Xavier NX suppor NVMe SSD max speed - #8 by carolyuu
Hi,
I used the iozone3 to test:
chao@chao-nano:~$ sudo iozone -ecI -+n -L64 -S32 -s64m -r1024k -i0 -i1 -l8 -u8 -m -t8 -F /mnt/file1 /mnt/file2 /mnt/file3 /mnt/file4 /mnt/file5 /mnt/file6 /mnt/file7 /mnt/file8
Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Mon Nov 23 18:49:51 2020
Include fsync in write timing
Include close in write timing
O_DIRECT feature enabled
No retest option selected
File size set to 65536 kB
Record Size 1024 kB
Multi_buffer. Work area 16777216 bytes
Command line used: iozone -ecI -+n -L64 -S32 -s64m -r1024k -i0 -i1 -l8 -u8 -m -t8 -F /mnt/file1 /mnt/file2 /mnt/file3 /mnt/file4 /mnt/file5 /mnt/file6 /mnt/file7 /mnt/file8
Output is in kBytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 32 kBytes.
Processor cache line size set to 64 bytes.
File stride size set to 17 * record size.
Min process = 8
Max process = 8
Throughput test with 8 processes
Each process writes a 65536 kByte file in 1024 kByte records
Children see throughput for 8 initial writers = 185997.27 kB/sec
Parent sees throughput for 8 initial writers = 19048.90 kB/sec
Min throughput per process = 5052.89 kB/sec
Max throughput per process = 46172.72 kB/sec
Avg throughput per process = 23249.66 kB/sec
Min xfer = 8192.00 kB
Children see throughput for 8 readers = 225206.55 kB/sec
Parent sees throughput for 8 readers = 222445.90 kB/sec
Min throughput per process = 21470.19 kB/sec
Max throughput per process = 41848.25 kB/sec
Avg throughput per process = 28150.82 kB/sec
Min xfer = 33792.00 kB
iozone test complete.
I am not very clear about the output. What does the “Parent sees throughput” mean? Compared to readers, the writers is slower.
Hi,
The test result looks OK.
Children see throughput for 8 initial writers = 185997.27 kB/sec
Children see throughput for 8 readers = 225206.55 kB/sec
For running 8 write processes, the throughput is 185997.278 ~= 1.4Gbps.
For running 8 read processes, the throughput is 225206.558 ~= 1.8Gbps.
Your case is single process. If you can run the read/write in multiple process, the performance can be better.
Hi,
Parent sees throughput for 8 initial writers = 19048.90 kB/sec
What’s the difference between Children and Parent, why ‘Parent sees throughput for 8 initial writers’ is so slow compared to others?