eSATA ports accessible to only xavier B on devkit

Hi,

I found all eSATA ports on devkit are only accessible to xavier B in Linux. is it possible to assign some eSATA ports to Xavier A? I need to mount one ssd directly in Linux on Xavier A.

Thanks
Jerry

Dear jerryc_sf,

Could you please help to check it with the following command?
I can access eSATA device no TegraA.

nvidia@tegra-ubuntu:~$ cat /proc/partitions
major minor  #blocks  name

 254        0   31490048 vblkdev0
 254        1   31471360 vblkdev0p1
 254       16   75119616 vblkdev1
 254       32    4194304 vblkdev2
 179        0    7634944 mmcblk0
 179       24       4096 mmcblk0rpmb
 179       16       8192 mmcblk0boot1
 179        8       8192 mmcblk0boot0
  31        0      65536 mtdblock0
   8        0   58615704 sda
   8        1   58612736 sda1
   8       16   58615704 sdb
   8       17   58612736 sdb1
nvidia@tegra-ubuntu:~$ df
Filesystem      1K-blocks     Used Available Use% Mounted on
/dev/vblkdev0p1  30845624 16076156  13179524  55% /
none             24805120        0  24805120   0% /dev
tmpfs            25339552       12  25339540   1% /dev/shm
tmpfs            25339552   181520  25158032   1% /run
tmpfs                5120        0      5120   0% /run/lock
tmpfs            25339552        0  25339552   0% /sys/fs/cgroup
/dev/vblkdev2     4186108       16   4186092   1% /storage
tmpfs             2533956       72   2533884   1% /run/user/1001
/dev/vblkdev1    73808088    53012  69982712   1% /media/nvidia/b89296a6-850b-4e06-82cb-36f927241cb5
/dev/sda1        57560316   725064  53888232   2% /media/nvidia/6659fd54-0ad8-4ee2-a1f6-aaa39bef9766
nvidia@tegra-ubuntu:~$

Hi SteveNV,

I knew how these commands work. /dev/sda and /dev/sdb etc. only show up in Linux on XavierB for ssd sata drives connected to eSATA ports. I saw /dev/sda on xavierA only if a usb drive is connected to a XavierA usb port.

Thanks
Jerry

please kindly let us know what could be the issue not seeing it on Xavier A (or Tegra a).

Also, On Xavier B(Tegra B), dmesg outputs the following for the samsung ssd driver connected at eSATA port 0. please let us know if it is using at least SATA 3Gb/s speed? I feel it is kind of slow (not like a ssd but a hard drive)

[   27.389127] scsi host0: mvsas
[   27.738898] sas: phy-0:0 added to port-0:0, phy_mask:0x1 (               0)
[   27.738961] sas: DOING DISCOVERY on port 0, pid:214
[   27.738982] sas: DONE DISCOVERY on port 0, pid:214, result:0
[   27.739046] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[   27.739091] sas: ata1: end_device-0:0: dev error handler
[   27.897739] ata1.00: supports DRM functions and may not be fully accessible
[   27.898796] ata1.00: ATA-11: Samsung SSD 860 EVO 1TB, RVT02B6Q, max UDMA/133
[   27.898807] ata1.00: 1953525168 sectors, multi 1: LBA48 NCQ (depth 31/32)
[   27.900110] ata1.00: supports DRM functions and may not be fully accessible
[   27.901905] ata1.00: configured for UDMA/133
[   27.901978] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1
[   27.948373] scsi 0:0:0:0: Direct-Access     ATA      Samsung SSD 860  2B6Q PQ: 0 ANSI: 5
[   28.044370] ata1.00: Enabling discard_zeroes_data
[   28.044471] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)
[   28.044822] sd 0:0:0:0: [sda] Write Protect is off
[   28.044827] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   28.044979] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   28.046353] carmel_ras carmel_ras: Deferring probe, arm64_ras hasnt been probed yet
[   28.079992] carmel_ras carmel_ras: Deferring probe, arm64_ras hasnt been probed yet
[   28.096356] ata1.00: Enabling discard_zeroes_data
[   28.101311]  sda: sda1
[   28.175245] ata1.00: Enabling discard_zeroes_data
[   28.175555] sd 0:0:0:0: [sda] Attached SCSI disk
[   30.633087] sas: Enter sas_scsi_recover_host busy: 1 failed: 1
[   30.633097] sas: ata1: end_device-0:0: cmd error handler
[   30.633148] sas: ata1: end_device-0:0: dev error handler
[   30.633221] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 1 tries: 1
[   30.645100] sas: Enter sas_scsi_recover_host busy: 1 failed: 1
[   30.645109] sas: ata1: end_device-0:0: cmd error handler
[   30.645151] sas: ata1: end_device-0:0: dev error handler
[   30.645247] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 1 tries: 1
[   31.474494] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[   31.474671] systemd[1]: Started CUPS Scheduler.
[   31.479278] systemd[1]: Found device Samsung_SSD_860_EVO_1TB primary.
[   31.642157] systemd[1]: Started CUPS Scheduler.

Dear Jerryc_sf,

Could you please let me know the DriveAGX PDK version?

Could you please try to run below commands to check the enumeration?

================
cat /proc/partition
lsblk
ls /dev/sd*

Mount the eSATA drive on Tegra to /media/
NOTE1 - create /media/sdd1

mount /dev/sdd1 /media/sdd1

Regarding eSATA speed, could you please help to check below commands?
dd if=/dev/zero of=/media/sdd1/80gb bs=260M count=320 &

DriveAGX PDK S/N: E3550-B03-S0886
Firmware/OS: Linux DriveAGX Software 8.0

Samsung SSD connected at eSATA port 0

On Xavier A:

nvidia@tegra-ubuntu:~$ cat /proc/partitions 
major minor  #blocks  name

 254        0   31490048 vblkdev0
 254        1   31471360 vblkdev0p1
 254       16   75119616 vblkdev1
 254       32    4194304 vblkdev2
 179        0    7634944 mmcblk0
 179       24       4096 mmcblk0rpmb
 179       16       8192 mmcblk0boot1
 179        8       8192 mmcblk0boot0
  31        0      65536 mtdblock0

nvidia@tegra-ubuntu:~$ lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0rpmb  179:24   0    4M  0 disk 
mtdblock0     31:0    0   64M  0 disk 
vblkdev2     254:32   0    4G  0 disk /storage
mmcblk0boot0 179:8    0    8M  1 disk 
vblkdev0     254:0    0   30G  0 disk 
└─vblkdev0p1 254:1    0   30G  0 part /
mmcblk0boot1 179:16   0    8M  1 disk 
mmcblk0      179:0    0  7.3G  0 disk 
vblkdev1     254:16   0 71.7G  0 disk 

nvidia@tegra-ubuntu:~$ ls /dev/sd*
ls: cannot access '/dev/sd*': No such file or directory

On Xavier B:

nvidia@tegra-ubuntu:~$ cat /proc/partitions 
major minor  #blocks  name

 254        0   31490048 vblkdev0
 254        1   31471360 vblkdev0p1
 254       16   75119616 vblkdev1
 254       32    4194304 vblkdev2
 179        0    7634944 mmcblk0
 179       24       4096 mmcblk0rpmb
 179       16       8192 mmcblk0boot1
 179        8       8192 mmcblk0boot0
  31        0      65536 mtdblock0
   8        0  976762584 sda
   8        1  976700872 sda1

nvidia@tegra-ubuntu:~$ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
mmcblk0rpmb  179:24   0     4M  0 disk 
mtdblock0     31:0    0    64M  0 disk 
vblkdev2     254:32   0     4G  0 disk /storage
mmcblk0boot0 179:8    0     8M  1 disk 
vblkdev0     254:0    0    30G  0 disk 
└─vblkdev0p1 254:1    0    30G  0 part /
sda            8:0    0 931.5G  0 disk 
└─sda1         8:1    0 931.5G  0 part /mnt/SSD1
mmcblk0boot1 179:16   0     8M  1 disk 
mmcblk0      179:0    0   7.3G  0 disk 
vblkdev1     254:16   0  71.7G  0 disk

nvidia@tegra-ubuntu:~$ ls /dev/sd*
/dev/sda  /dev/sda1

will do dd command performance test later

could you please kindly update the status on this issue?

Dear jerryc_sf,

This is the default PEX switch config we ship with all the boards – you can use the instructions in the standard docs (Flashing Basics -> Flashing the PCIe Switch and Configuration File) to do this.
https://www.nvidia.com/en-us/nvdocs/drive/5_1/linux/DRIVE_Linux_AGX_PDK_Development_Guide/Flashing_Basics/flash_pcie_uart/

However DriveSW assumes the ssd on Xavier B so if this is changed some samples/apps may break.

Could you let me know why you want to change the ssd making? Thanks.

Dear SteveNV

I want to see below URL.
https://www.nvidia.com/en-us/nvdocs/drive/5_1/linux/DRIVE_Linux_AGX_PDK_Development_Guide/Flashing_Basics/flash_pcie_uart/
But I couldn’t see the page, maybe broken link.

Would you tell us right URL? Or contents of the page via pdf or something like that?

Thank you in advance for your cooperation.

Dear okada-ki9vsnr,

Please refer to the link. Thanks.

https://docs.nvidia.com/drive/active/5.1.0.2L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide%2FFlashing%20Basics%2Fflash_pcie_uart.html%23

Were any of you able to get the eSATA ports working on Xavier A?

I am using DRIVE OS v5.1.6.1
PCIe switch flashing instructions for my version of DRIVE OS are here: https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/Flashing%20Basics/flashing_pcie.html

According to the document, switch configuration files should be present in this location:

Switch Configuration File
The switch configuration files are preset at:
•For PCIe switch-1:
drive-t186ref-foundation/firmwares/bin/t19x/microsemi/pcie/bins/e3550/config_files/pm8534/
•For PCIe switch-2:
drive-t186ref-foundation/firmwares/bin/t19x/microsemi/pcie/bins/e3550/config_files/pm8533/
Details on each configuration file, for example partition and ownership details, are provided in the README file available at:
drive-t186ref-foundation/firmwares/bin/t19x/microsemi/pcie/bins/e3550/config_files/pm8534/README

But I do not have the microsemi directory or anything below it under drive-t186ref-foundation/firmwares/bin/t19x

Please follow https://developer.nvidia.com/drive/downloads -> Installation Notes (Click to toggle open/close) -> “3. UPDATE the PCIe SWITCH FIRMWARE” to get its run file. Thanks!

Thanks for the link @VickNV, I downloaded the pcie switch config files but ran into some issues while updating the firmware.

nvidia@tegra-ubuntu:/lib/firmware/pcie-switch/tools/prebuilt$ lsmod | grep switchtec
ntb_hw_switchtec 24576 0
ntb 16384 1 ntb_hw_switchtec
switchtec 32768 1 ntb_hw_switchtec
nvidia@tegra-ubuntu:/lib/firmware/pcie-switch/tools/prebuilt$ lspci
0000:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad0 (rev a1)
0000:01:00.0 PCI bridge: PMC-Sierra Inc. Device 8534
0000:01:00.1 Bridge: PMC-Sierra Inc. Device 8534
0000:02:00.0 PCI bridge: PMC-Sierra Inc. Device 8534
0000:02:01.0 PCI bridge: PMC-Sierra Inc. Device 8534
0000:04:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)
0001:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad2 (rev a1)
0004:00:00.0 PCI bridge: NVIDIA Corporation Device 1ad1 (rev a1)
0004:01:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)
nvidia@tegra-ubuntu:/lib/firmware/pcie-switch/tools/prebuilt$
nvidia@tegra-ubuntu:/lib/firmware/pcie-switch/tools/prebuilt$ sudo ./switchtec fw-update /home/nvidia/drive-t186ref-foundation/firmwares/bin/t19x/microsemi/pcie/bins/e3550/config_files/pm8534/usecase4.0.pmc /dev/switchtec0
/home/nvidia/drive-t186ref-foundation/firmwares/bin/t19x/microsemi/pcie/bins/e3550/config_files/pm8534/usecase4.0.pmc: Inappropriate ioctl for device

Would really appreciate if you can provide some help/insight.

Please try with “./switchtec fw-update <img_file>”. Below is the details usage of the command.

$ ./switchtec help fw-update
Usage: ./switchtec help./switchtec fw-update <device> <img_file> [OPTIONS]

Flash the firmware with a new image


Positional Arguments:
   <device>                             --- switchtec device to operate on.
                                            can be a device path
                                            (/dev/switchtec0), an index (0,
                                            1, 2) or a pci address (3:00.1)
   <img_file>                           --- image file to use as the new
                                            firmware


Environment Variables:
    SWITCHTEC_DEV                       --- if set, the value will be used
                                            for the <device> argument


Options:
  [  --yes, -y ]                        --- assume yes when prompted
  [  --dont-activate, -A ]              --- don't activate the new image, use
                                            fw-toggle to do so when it is
                                            safe
  [  --force, -f ]                      --- force interrupting an existing
                                            fw-update command in case
                                            firmware is stuck in the busy
                                            state
  [  --set-boot-rw, -W ]                --- set the bootloader partition as
                                            RW (only valid for BOOT images)

thanks that worked