NVMe SSD write performance degradation between TX1 and TX2

I have a J120 (rev11) base board with a 1TB EVO 970 PRO installed. I’ve measured the write performance using

dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync

Write performance using a TX1 (Jetpack 4.3.1) is around 430 MB/s.

Running the same command on a TX2 - same board, same SSD I only get 230 MB/s.

I’ve used the https://auvidea.eu/download/firmware/TX2/v2.2/J90_v2.2_4.2.2.tar.bz2 from AUVIDEA to get the USB on the board working.

lspci -vv reports “Width x4” on both setups. I’ve also tried running jetsonclocks.

I am running a bit out of ideas. One thing I did notice is that the plugin-manager seems to have changed on 4.3.1, and that it doesn’t seem to apply any of the fragments - nothing is reported on dmesg, as I believe it used to. I also couldn’t find any documentation on what changed there. E.g. I believe ‘pcie0_lane2_mux’ is disabled.

Hi jschwendner,

Could you put this test on nvidia devkit and share the result again?

pcie0_lane2_mux should not matter here. If this issue is related to this mux, then the pcie device would just be disabled.

What would be the best way to connect the NVMe to the devkit? The M.2 slot on the board does not seem to fit a full size M.2 I believe.

I have connected the ssd to the dev-board now, using a pcie-nvme adapter. The result is the same. I’ve also noticed that when using O_DIRECT as parameter in the dd, the write speeds are around 700 MB/s. The values that I would actually expect from the SSD.
I’ve tested on both tx1 and tx2 using the same 4.9 kernel, and could still see the difference as described above.