Hi there,
I’m setting up the Mellanox CX-5 (MCX515A-CCAT) NIC for my server with Ubuntu 20.04 LTS. When running a traffic generator, It can only achieve 60% of the line rate speed (in the order of 90Mpps with 64B packets). By reading the DPDK manual, I knew that changing the PCIe MaxPayload and MaxReadReq might be helpful.
As recommended, I first checked out the current value with setpci -s 0b:00.0 68.w, the value was 0000 (the value is actually pretty wired). When I used setpci -s 0b:00.0 68.w=3000 to change the MaxReadReq from 128 bytes to 512 bytes, it completed without error (echo $? give 0). However, when I rechecked the value with setpci -s 0b:00.0 68.w, the value was still 0000 without any changes.
I totally have no idea what was wrong in my configurations. Could anyone help me with this? Thanks in advance.
Wendi
# lspci -s 0b:00.0 -vvvv
0b:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
DeviceName: pciPassthru0
Subsystem: Mellanox Technologies ConnectX®-5 EN network interface card, 100GbE single-port QSFP28, PCIe3.0 x16, tall bracket; MCX515A-CCAT
Physical Slot: 192
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 19
Region 0: Memory at e4000000 (64-bit, prefetchable) [size=32M]
Capabilities: [60] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x32, ASPM L0s, Exit Latency L0s <64ns
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s (ok), Width x32 (ok)
TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-, NROPrPrP-, LTR-
10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS-, TPHComp-, ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
AtomicOpsCtl: ReqEn-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [48] Vital Product Data
Product Name: CX515A - ConnectX-5 QSFP28
Read-only fields:
[PN] Part number: MCX515A-CCAT
[EC] Engineering changes: B5
[V2] Vendor specific: MCX515A-CCAT
[SN] Serial number: MT2131J08328
[V3] Vendor specific: be59bbea0ef0eb118000b8cef6fcdd20
[VA] Vendor specific: MLX:MODL=CX515A:MN=MLNX:CSKU=V2:UUID=V3:PCI=V0
[V0] Vendor specific: PCIeGen3 x16
[VU] Vendor specific: MT2131J08328MLNXS0D0F0
[RV] Reserved: checksum good, 2 byte(s) reserved
End
Capabilities: [9c] MSI-X: Enable+ Count=64 Masked-
Vector table: BAR=0 offset=00002000
PBA: BAR=0 offset=00003000
Capabilities: [c0] Vendor Specific Information: Len=18 <?>
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot-,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ AdvNonFatalErr+
AERCap: First Error Pointer: 08, ECRCGenCap+ ECRCGenEn+ ECRCChkCap+ ECRCChkEn+
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [1c0 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn-, PerformEqu-
LaneErrStat: 0
Kernel driver in use: mlx5_core
Kernel modules: mlx5_core
# lspci -s 0b:00.0 -xxx
0b:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
00: b3 15 17 10 07 04 10 00 00 00 00 02 08 40 00 00
10: 0c 00 00 e4 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 b3 15 06 00
30: 00 00 00 00 60 00 00 00 00 00 00 00 0a 01 00 00
40: 01 00 c3 81 08 00 00 00 03 9c e4 80 ea 00 00 78
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 01
60: 10 48 02 00 00 00 00 00 00 00 00 00 02 06 00 00
70: 00 00 02 02 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 11 c0 3f 80
a0: 00 20 00 00 00 30 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 09 40 18 00 07 00 00 30 0d 00 00 00 00 00 00 00
d0: fc ff 13 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Hello loganwhitevon,
Thank you for posting your query on our community. Please ensure to check the system logs for any errors related to the PCIe configuration.
Additionally, you could also try to modify the PCIe max read request value from the system’s BIOS settings. Make sure to save the changes and reboot the system to apply the modifications.
If the issue still persists, I would like to request you to submit a support ticket for further troubleshooting. The support ticket can be opened by emailing " Networking-support@nvidia.com ". Please note that an active support contract would be required for the same. For contracts information, please feel free to reach out to our contracts team at " Networking-Contracts@nvidia.com "
Thanks & Regards,
~ Nvidia Support
Hi @sribhargavid,
Thank you so much for replying. The dmesg messages are shown as follows, and it seems everything works fine. Apart from the MaxReadReq register, I have tried to reset other registers of this NIC, it does not work either.
Incidentally, my NIC runs on a VM ESXi VM with PCIe passthrough to the VM. ESXi version is 6.5, OS version is Ubuntu 20.04, MLXN_OFED driver version is MLNX_OFED_LINUX-5.8-2.0.3.0-ubuntu20.04-x86_64, and NIC firmware is burned with fw-ConnectX5-rel-16_35_2000-MCX515A-CCA_Ax_Bx-UEFI-14.28.16-FlexBoot-3.6.805.bin.
Can you please help me with the reason? Thank you.
# dmesg | grep mlx
[ 5.843935] mlx5_core 0000:0b:00.0: firmware version: 16.35.2000
[ 5.844035] mlx5_core 0000:0b:00.0: 128.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x32 link)
[ 5.872910] mlx5_core 0000:0b:00.0: handle_hca_cap:549:(pid 383): log_max_qp value in current profile is 18, changing it to HCA capability limit (17)
[ 6.325698] mlx5_core 0000:0b:00.0: Rate limit: 127 rates are supported, range: 0Mbps to 97656Mbps
[ 6.334825] mlx5_core 0000:0b:00.0: E-Switch: Total vports 2, per vport: max uc(128) max mc(2048)
[ 6.365152] mlx5_core 0000:0b:00.0: Port module event: module 0, Cable plugged
[ 6.370321] mlx5_core 0000:0b:00.0: mlx5_pcie_event:294:(pid 8): PCIe slot advertised sufficient power (27W).
[ 6.395391] mlx5_core 0000:0b:00.0: MLX5E: StrdRq(1) RqSz(8) StrdSz(2048) RxCqeCmprss(0)
[ 6.679887] mlx5_core 0000:0b:00.0 ens192: renamed from eth0
[ 6.686941] mlx5_ib: Mellanox Connect-IB Infiniband driver v5.0-0
[ 18.654031] mlx5_core 0000:0b:00.0: E-Switch: cleanup
[ 23.489702] mlx_compat: loading out-of-tree module taints kernel.
[ 23.489922] mlx_compat: module verification failed: signature and/or required key missing - tainting kernel
[ 23.649834] mlx5_core 0000:0b:00.0: firmware version: 16.35.2000
[ 23.649937] mlx5_core 0000:0b:00.0: 128.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x32 link)
[ 24.121789] mlx5_core 0000:0b:00.0: Rate limit: 127 rates are supported, range: 0Mbps to 97656Mbps
[ 24.122683] mlx5_core 0000:0b:00.0: E-Switch: Total vports 2, per vport: max uc(128) max mc(2048)
[ 24.138853] mlx5_core 0000:0b:00.0: Port module event: module 0, Cable plugged
[ 24.139398] mlx5_core 0000:0b:00.0: mlx5_pcie_event:304:(pid 604): PCIe slot advertised sufficient power (27W).
[ 24.170242] mlx5_core 0000:0b:00.0: mlx5_fw_tracer_start:821:(pid 1497): FWTracer: Ownership granted and active
[ 24.182830] mlx5_core 0000:0b:00.0: MLX5E: StrdRq(1) RqSz(8) StrdSz(2048) RxCqeCmprss(0)
[ 24.418768] mlx5_core 0000:0b:00.0: Supported tc offload range - chains: 4294967294, prios: 4294967295
[ 24.487864] mlx5_core 0000:0b:00.0 ens192np0: renamed from eth0
# dmesg | grep pci
[ 1.351232] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 1.721952] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 1.722183] pci_bus 0000:00: root bus resource [mem 0x000cc000-0x000cffff window]
[ 1.722414] pci_bus 0000:00: root bus resource [mem 0x000d0000-0x000d3fff window]
[ 1.722644] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff window]
[ 1.722874] pci_bus 0000:00: root bus resource [mem 0x000d8000-0x000dbfff window]
[ 1.723106] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[ 1.723337] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 1.723542] pci_bus 0000:00: root bus resource [io 0x0d00-0xfeff window]
[ 1.723748] pci_bus 0000:00: root bus resource [bus 00-7f]
[ 1.724044] pci 0000:00:00.0: [8086:7190] type 00 class 0x060000
[ 1.727035] pci 0000:00:01.0: [8086:7191] type 01 class 0x060400
[ 1.732170] pci 0000:00:07.0: [8086:7110] type 00 class 0x060100
[ 1.734911] pci 0000:00:07.1: [8086:7111] type 00 class 0x01018a
[ 1.740841] pci 0000:00:07.1: reg 0x20: [io 0x1060-0x106f]
[ 1.743165] pci 0000:00:07.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 1.743392] pci 0000:00:07.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 1.743592] pci 0000:00:07.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 1.743804] pci 0000:00:07.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 1.745601] pci 0000:00:07.3: [8086:7113] type 00 class 0x068000
[ 1.753083] pci 0000:00:07.3: quirk: [io 0x1000-0x103f] claimed by PIIX4 ACPI
[ 1.753340] pci 0000:00:07.3: quirk: [io 0x1040-0x104f] claimed by PIIX4 SMB
[ 1.755988] pci 0000:00:07.7: [15ad:0740] type 00 class 0x088000
[ 1.759391] pci 0000:00:07.7: reg 0x10: [io 0x1080-0x10bf]
[ 1.763676] pci 0000:00:07.7: reg 0x14: [mem 0xfebfe000-0xfebfffff 64bit]
[ 1.784914] pci 0000:00:0f.0: [15ad:0405] type 00 class 0x030000
[ 1.792997] pci 0000:00:0f.0: reg 0x10: [io 0x1070-0x107f]
[ 1.800997] pci 0000:00:0f.0: reg 0x14: [mem 0xe8000000-0xefffffff pref]
[ 1.808993] pci 0000:00:0f.0: reg 0x18: [mem 0xfe000000-0xfe7fffff]
[ 1.840997] pci 0000:00:0f.0: reg 0x30: [mem 0x00000000-0x00007fff pref]
[ 1.843670] pci 0000:00:10.0: [1000:0030] type 00 class 0x010000
[ 1.848996] pci 0000:00:10.0: reg 0x10: [io 0x1400-0x14ff]
[ 1.856998] pci 0000:00:10.0: reg 0x14: [mem 0xfeba0000-0xfebbffff 64bit]
[ 1.864997] pci 0000:00:10.0: reg 0x1c: [mem 0xfebc0000-0xfebdffff 64bit]
[ 1.877000] pci 0000:00:10.0: reg 0x30: [mem 0x00000000-0x00003fff pref]
[ 1.879566] pci 0000:00:11.0: [15ad:0790] type 01 class 0x060401
[ 1.884306] pci 0000:00:15.0: [15ad:07a0] type 01 class 0x060400
[ 1.887147] pci 0000:00:15.0: PME# supported from D0 D3hot D3cold
[ 1.888133] pci 0000:00:15.1: [15ad:07a0] type 01 class 0x060400
[ 1.890968] pci 0000:00:15.1: PME# supported from D0 D3hot D3cold
[ 1.891954] pci 0000:00:15.2: [15ad:07a0] type 01 class 0x060400
[ 1.894799] pci 0000:00:15.2: PME# supported from D0 D3hot D3cold
[ 1.895779] pci 0000:00:15.3: [15ad:07a0] type 01 class 0x060400
[ 1.898618] pci 0000:00:15.3: PME# supported from D0 D3hot D3cold
[ 1.899615] pci 0000:00:15.4: [15ad:07a0] type 01 class 0x060400
[ 1.902458] pci 0000:00:15.4: PME# supported from D0 D3hot D3cold
[ 1.903446] pci 0000:00:15.5: [15ad:07a0] type 01 class 0x060400
[ 1.906285] pci 0000:00:15.5: PME# supported from D0 D3hot D3cold
[ 1.907266] pci 0000:00:15.6: [15ad:07a0] type 01 class 0x060400
[ 1.910095] pci 0000:00:15.6: PME# supported from D0 D3hot D3cold
[ 1.911078] pci 0000:00:15.7: [15ad:07a0] type 01 class 0x060400
[ 1.913921] pci 0000:00:15.7: PME# supported from D0 D3hot D3cold
[ 1.915376] pci 0000:00:16.0: [15ad:07a0] type 01 class 0x060400
[ 1.918199] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
[ 1.919183] pci 0000:00:16.1: [15ad:07a0] type 01 class 0x060400
[ 1.922016] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold
[ 1.923033] pci 0000:00:16.2: [15ad:07a0] type 01 class 0x060400
[ 1.925865] pci 0000:00:16.2: PME# supported from D0 D3hot D3cold
[ 1.926858] pci 0000:00:16.3: [15ad:07a0] type 01 class 0x060400
[ 1.929699] pci 0000:00:16.3: PME# supported from D0 D3hot D3cold
[ 1.930687] pci 0000:00:16.4: [15ad:07a0] type 01 class 0x060400
[ 1.933530] pci 0000:00:16.4: PME# supported from D0 D3hot D3cold
[ 1.934518] pci 0000:00:16.5: [15ad:07a0] type 01 class 0x060400
[ 1.937352] pci 0000:00:16.5: PME# supported from D0 D3hot D3cold
[ 1.938345] pci 0000:00:16.6: [15ad:07a0] type 01 class 0x060400
[ 1.941165] pci 0000:00:16.6: PME# supported from D0 D3hot D3cold
[ 1.942169] pci 0000:00:16.7: [15ad:07a0] type 01 class 0x060400
[ 1.944999] pci 0000:00:16.7: PME# supported from D0 D3hot D3cold
[ 1.946456] pci 0000:00:17.0: [15ad:07a0] type 01 class 0x060400
[ 1.949277] pci 0000:00:17.0: PME# supported from D0 D3hot D3cold
[ 1.950287] pci 0000:00:17.1: [15ad:07a0] type 01 class 0x060400
[ 1.953145] pci 0000:00:17.1: PME# supported from D0 D3hot D3cold
[ 1.954160] pci 0000:00:17.2: [15ad:07a0] type 01 class 0x060400
[ 1.956995] pci 0000:00:17.2: PME# supported from D0 D3hot D3cold
[ 1.958005] pci 0000:00:17.3: [15ad:07a0] type 01 class 0x060400
[ 1.960828] pci 0000:00:17.3: PME# supported from D0 D3hot D3cold
[ 1.961901] pci 0000:00:17.4: [15ad:07a0] type 01 class 0x060400
[ 1.964770] pci 0000:00:17.4: PME# supported from D0 D3hot D3cold
[ 1.965768] pci 0000:00:17.5: [15ad:07a0] type 01 class 0x060400
[ 1.968594] pci 0000:00:17.5: PME# supported from D0 D3hot D3cold
[ 1.969596] pci 0000:00:17.6: [15ad:07a0] type 01 class 0x060400
[ 1.972454] pci 0000:00:17.6: PME# supported from D0 D3hot D3cold
[ 1.973474] pci 0000:00:17.7: [15ad:07a0] type 01 class 0x060400
[ 1.976319] pci 0000:00:17.7: PME# supported from D0 D3hot D3cold
[ 1.977788] pci 0000:00:18.0: [15ad:07a0] type 01 class 0x060400
[ 1.980616] pci 0000:00:18.0: PME# supported from D0 D3hot D3cold
[ 1.981607] pci 0000:00:18.1: [15ad:07a0] type 01 class 0x060400
[ 1.984475] pci 0000:00:18.1: PME# supported from D0 D3hot D3cold
[ 1.985464] pci 0000:00:18.2: [15ad:07a0] type 01 class 0x060400
[ 1.988318] pci 0000:00:18.2: PME# supported from D0 D3hot D3cold
[ 1.989316] pci 0000:00:18.3: [15ad:07a0] type 01 class 0x060400
[ 1.992168] pci 0000:00:18.3: PME# supported from D0 D3hot D3cold
[ 1.993198] pci 0000:00:18.4: [15ad:07a0] type 01 class 0x060400
[ 1.996059] pci 0000:00:18.4: PME# supported from D0 D3hot D3cold
[ 1.997036] pci 0000:00:18.5: [15ad:07a0] type 01 class 0x060400
[ 1.999883] pci 0000:00:18.5: PME# supported from D0 D3hot D3cold
[ 2.000867] pci 0000:00:18.6: [15ad:07a0] type 01 class 0x060400
[ 2.003674] pci 0000:00:18.6: PME# supported from D0 D3hot D3cold
[ 2.004660] pci 0000:00:18.7: [15ad:07a0] type 01 class 0x060400
[ 2.007507] pci 0000:00:18.7: PME# supported from D0 D3hot D3cold
[ 2.012215] pci_bus 0000:01: extended config space not accessible
[ 2.027614] pci 0000:00:01.0: PCI bridge to [bus 01]
[ 2.028190] pci_bus 0000:02: extended config space not accessible
[ 2.039165] pci 0000:02:01.0: [15ad:07e0] type 00 class 0x010601
[ 2.060738] pci 0000:02:01.0: reg 0x24: [mem 0xfd5ff000-0xfd5fffff]
[ 2.063512] pci 0000:02:01.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[ 2.064522] pci 0000:02:01.0: PME# supported from D3hot
[ 2.080712] pci 0000:00:11.0: PCI bridge to [bus 02] (subtractive decode)
[ 2.080962] pci 0000:00:11.0: bridge window [io 0x2000-0x3fff]
[ 2.081031] pci 0000:00:11.0: bridge window [mem 0xfd500000-0xfdffffff]
[ 2.081305] pci 0000:00:11.0: bridge window [mem 0xe7b00000-0xe7ffffff 64bit pref]
[ 2.081541] pci 0000:00:11.0: bridge window [mem 0x000a0000-0x000bffff window] (subtractive decode)
[ 2.081808] pci 0000:00:11.0: bridge window [mem 0x000cc000-0x000cffff window] (subtractive decode)
[ 2.082074] pci 0000:00:11.0: bridge window [mem 0x000d0000-0x000d3fff window] (subtractive decode)
[ 2.082340] pci 0000:00:11.0: bridge window [mem 0x000d4000-0x000d7fff window] (subtractive decode)
[ 2.082606] pci 0000:00:11.0: bridge window [mem 0x000d8000-0x000dbfff window] (subtractive decode)
[ 2.082873] pci 0000:00:11.0: bridge window [mem 0xc0000000-0xfebfffff window] (subtractive decode)
[ 2.083139] pci 0000:00:11.0: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 2.083389] pci 0000:00:11.0: bridge window [io 0x0d00-0xfeff window] (subtractive decode)
[ 2.084516] pci 0000:03:00.0: [15ad:07b0] type 00 class 0x020000
[ 2.088998] pci 0000:03:00.0: reg 0x10: [mem 0xfd4fc000-0xfd4fcfff]
[ 2.096998] pci 0000:03:00.0: reg 0x14: [mem 0xfd4fd000-0xfd4fdfff]
[ 2.104997] pci 0000:03:00.0: reg 0x18: [mem 0xfd4fe000-0xfd4fffff]
[ 2.112997] pci 0000:03:00.0: reg 0x1c: [io 0x4000-0x400f]
[ 2.136997] pci 0000:03:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[ 2.138215] pci 0000:03:00.0: supports D1 D2
[ 2.138365] pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 2.139826] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
[ 2.155616] pci 0000:00:15.0: PCI bridge to [bus 03]
[ 2.155833] pci 0000:00:15.0: bridge window [io 0x4000-0x4fff]
[ 2.156066] pci 0000:00:15.0: bridge window [mem 0xfd400000-0xfd4fffff]
[ 2.171651] pci 0000:00:15.1: PCI bridge to [bus 04]
[ 2.171877] pci 0000:00:15.1: bridge window [io 0x7000-0x7fff]
[ 2.172101] pci 0000:00:15.1: bridge window [mem 0xfd100000-0xfd1fffff]
[ 2.172374] pci 0000:00:15.1: bridge window [mem 0xe3d00000-0xe3dfffff 64bit pref]
[ 2.188010] pci 0000:00:15.2: PCI bridge to [bus 05]
[ 2.188221] pci 0000:00:15.2: bridge window [io 0xb000-0xbfff]
[ 2.188445] pci 0000:00:15.2: bridge window [mem 0xfcd00000-0xfcdfffff]
[ 2.188719] pci 0000:00:15.2: bridge window [mem 0xe3900000-0xe39fffff 64bit pref]
[ 2.204129] pci 0000:00:15.3: PCI bridge to [bus 06]
[ 2.204396] pci 0000:00:15.3: bridge window [mem 0xfc900000-0xfc9fffff]
[ 2.204670] pci 0000:00:15.3: bridge window [mem 0xe3500000-0xe35fffff 64bit pref]
[ 2.220060] pci 0000:00:15.4: PCI bridge to [bus 07]
[ 2.220304] pci 0000:00:15.4: bridge window [mem 0xfc500000-0xfc5fffff]
[ 2.220578] pci 0000:00:15.4: bridge window [mem 0xe3100000-0xe31fffff 64bit pref]
[ 2.235979] pci 0000:00:15.5: PCI bridge to [bus 08]
[ 2.236223] pci 0000:00:15.5: bridge window [mem 0xfc100000-0xfc1fffff]
[ 2.236497] pci 0000:00:15.5: bridge window [mem 0xe2d00000-0xe2dfffff 64bit pref]
[ 2.251977] pci 0000:00:15.6: PCI bridge to [bus 09]
[ 2.252220] pci 0000:00:15.6: bridge window [mem 0xfbd00000-0xfbdfffff]
[ 2.252494] pci 0000:00:15.6: bridge window [mem 0xe2900000-0xe29fffff 64bit pref]
[ 2.267993] pci 0000:00:15.7: PCI bridge to [bus 0a]
[ 2.268236] pci 0000:00:15.7: bridge window [mem 0xfb900000-0xfb9fffff]
[ 2.268511] pci 0000:00:15.7: bridge window [mem 0xe2500000-0xe25fffff 64bit pref]
[ 2.269591] pci 0000:0b:00.0: [15b3:1017] type 00 class 0x020000
[ 2.276997] pci 0000:0b:00.0: reg 0x10: [mem 0xe4000000-0xe5ffffff 64bit pref]
[ 2.306517] pci 0000:0b:00.0: PME# supported from D3cold
[ 2.308381] pci 0000:0b:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
[ 2.323603] pci 0000:00:16.0: PCI bridge to [bus 0b]
[ 2.323943] pci 0000:00:16.0: bridge window [mem 0xe4000000-0xe5ffffff 64bit pref]
[ 2.339587] pci 0000:00:16.1: PCI bridge to [bus 0c]
[ 2.339800] pci 0000:00:16.1: bridge window [io 0x8000-0x8fff]
[ 2.340024] pci 0000:00:16.1: bridge window [mem 0xfd000000-0xfd0fffff]
[ 2.340298] pci 0000:00:16.1: bridge window [mem 0xe3c00000-0xe3cfffff 64bit pref]
[ 2.355888] pci 0000:00:16.2: PCI bridge to [bus 0d]
[ 2.356100] pci 0000:00:16.2: bridge window [io 0xc000-0xcfff]
[ 2.356324] pci 0000:00:16.2: bridge window [mem 0xfcc00000-0xfccfffff]
[ 2.356597] pci 0000:00:16.2: bridge window [mem 0xe3800000-0xe38fffff 64bit pref]
[ 2.372194] pci 0000:00:16.3: PCI bridge to [bus 0e]
[ 2.372441] pci 0000:00:16.3: bridge window [mem 0xfc800000-0xfc8fffff]
[ 2.372716] pci 0000:00:16.3: bridge window [mem 0xe3400000-0xe34fffff 64bit pref]
[ 2.388265] pci 0000:00:16.4: PCI bridge to [bus 0f]
[ 2.388509] pci 0000:00:16.4: bridge window [mem 0xfc400000-0xfc4fffff]
[ 2.388785] pci 0000:00:16.4: bridge window [mem 0xe3000000-0xe30fffff 64bit pref]
[ 2.404330] pci 0000:00:16.5: PCI bridge to [bus 10]
[ 2.404576] pci 0000:00:16.5: bridge window [mem 0xfc000000-0xfc0fffff]
[ 2.404849] pci 0000:00:16.5: bridge window [mem 0xe2c00000-0xe2cfffff 64bit pref]
[ 2.420289] pci 0000:00:16.6: PCI bridge to [bus 11]
[ 2.420541] pci 0000:00:16.6: bridge window [mem 0xfbc00000-0xfbcfffff]
[ 2.420815] pci 0000:00:16.6: bridge window [mem 0xe2800000-0xe28fffff 64bit pref]
[ 2.436338] pci 0000:00:16.7: PCI bridge to [bus 12]
[ 2.436582] pci 0000:00:16.7: bridge window [mem 0xfb800000-0xfb8fffff]
[ 2.436855] pci 0000:00:16.7: bridge window [mem 0xe2400000-0xe24fffff 64bit pref]
[ 2.452353] pci 0000:00:17.0: PCI bridge to [bus 13]
[ 2.452565] pci 0000:00:17.0: bridge window [io 0x5000-0x5fff]
[ 2.452792] pci 0000:00:17.0: bridge window [mem 0xfd300000-0xfd3fffff]
[ 2.453064] pci 0000:00:17.0: bridge window [mem 0xe3f00000-0xe3ffffff 64bit pref]
[ 2.468611] pci 0000:00:17.1: PCI bridge to [bus 14]
[ 2.468821] pci 0000:00:17.1: bridge window [io 0x9000-0x9fff]
[ 2.469030] pci 0000:00:17.1: bridge window [mem 0xfcf00000-0xfcffffff]
[ 2.469319] pci 0000:00:17.1: bridge window [mem 0xe3b00000-0xe3bfffff 64bit pref]
[ 2.484878] pci 0000:00:17.2: PCI bridge to [bus 15]
[ 2.485038] pci 0000:00:17.2: bridge window [io 0xd000-0xdfff]
[ 2.485263] pci 0000:00:17.2: bridge window [mem 0xfcb00000-0xfcbfffff]
[ 2.485535] pci 0000:00:17.2: bridge window [mem 0xe3700000-0xe37fffff 64bit pref]
[ 2.499724] pci 0000:00:17.3: PCI bridge to [bus 16]
[ 2.500208] pci 0000:00:17.3: bridge window [mem 0xfc700000-0xfc7fffff]
[ 2.500562] pci 0000:00:17.3: bridge window [mem 0xe3300000-0xe33fffff 64bit pref]
[ 2.516099] pci 0000:00:17.4: PCI bridge to [bus 17]
[ 2.516347] pci 0000:00:17.4: bridge window [mem 0xfc300000-0xfc3fffff]
[ 2.516619] pci 0000:00:17.4: bridge window [mem 0xe2f00000-0xe2ffffff 64bit pref]
[ 2.532161] pci 0000:00:17.5: PCI bridge to [bus 18]
[ 2.532409] pci 0000:00:17.5: bridge window [mem 0xfbf00000-0xfbffffff]
[ 2.532703] pci 0000:00:17.5: bridge window [mem 0xe2b00000-0xe2bfffff 64bit pref]
[ 2.548161] pci 0000:00:17.6: PCI bridge to [bus 19]
[ 2.548410] pci 0000:00:17.6: bridge window [mem 0xfbb00000-0xfbbfffff]
[ 2.548681] pci 0000:00:17.6: bridge window [mem 0xe2700000-0xe27fffff 64bit pref]
[ 2.564191] pci 0000:00:17.7: PCI bridge to [bus 1a]
[ 2.564437] pci 0000:00:17.7: bridge window [mem 0xfb700000-0xfb7fffff]
[ 2.564709] pci 0000:00:17.7: bridge window [mem 0xe2300000-0xe23fffff 64bit pref]
[ 2.580166] pci 0000:00:18.0: PCI bridge to [bus 1b]
[ 2.580379] pci 0000:00:18.0: bridge window [io 0x6000-0x6fff]
[ 2.580600] pci 0000:00:18.0: bridge window [mem 0xfd200000-0xfd2fffff]
[ 2.580871] pci 0000:00:18.0: bridge window [mem 0xe3e00000-0xe3efffff 64bit pref]
[ 2.596253] pci 0000:00:18.1: PCI bridge to [bus 1c]
[ 2.596465] pci 0000:00:18.1: bridge window [io 0xa000-0xafff]
[ 2.596686] pci 0000:00:18.1: bridge window [mem 0xfce00000-0xfcefffff]
[ 2.596962] pci 0000:00:18.1: bridge window [mem 0xe3a00000-0xe3afffff 64bit pref]
[ 2.612179] pci 0000:00:18.2: PCI bridge to [bus 1d]
[ 2.612391] pci 0000:00:18.2: bridge window [io 0xe000-0xefff]
[ 2.612613] pci 0000:00:18.2: bridge window [mem 0xfca00000-0xfcafffff]
[ 2.612883] pci 0000:00:18.2: bridge window [mem 0xe3600000-0xe36fffff 64bit pref]
[ 2.628253] pci 0000:00:18.3: PCI bridge to [bus 1e]
[ 2.628498] pci 0000:00:18.3: bridge window [mem 0xfc600000-0xfc6fffff]
[ 2.628769] pci 0000:00:18.3: bridge window [mem 0xe3200000-0xe32fffff 64bit pref]
[ 2.644250] pci 0000:00:18.4: PCI bridge to [bus 1f]
[ 2.644486] pci 0000:00:18.4: bridge window [mem 0xfc200000-0xfc2fffff]
[ 2.644737] pci 0000:00:18.4: bridge window [mem 0xe2e00000-0xe2efffff 64bit pref]
[ 2.659048] pci 0000:00:18.5: PCI bridge to [bus 20]
[ 2.659280] pci 0000:00:18.5: bridge window [mem 0xfbe00000-0xfbefffff]
[ 2.659531] pci 0000:00:18.5: bridge window [mem 0xe2a00000-0xe2afffff 64bit pref]
[ 2.673842] pci 0000:00:18.6: PCI bridge to [bus 21]
[ 2.674070] pci 0000:00:18.6: bridge window [mem 0xfba00000-0xfbafffff]
[ 2.674326] pci 0000:00:18.6: bridge window [mem 0xe2600000-0xe26fffff 64bit pref]
[ 2.684635] pci 0000:00:18.7: PCI bridge to [bus 22]
[ 2.684863] pci 0000:00:18.7: bridge window [mem 0xfb600000-0xfb6fffff]
[ 2.684978] pci 0000:00:18.7: bridge window [mem 0xe2200000-0xe22fffff 64bit pref]
[ 2.797648] pci 0000:00:0f.0: vgaarb: setting as boot VGA device
[ 2.797648] pci 0000:00:0f.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[ 2.797648] pci 0000:00:0f.0: vgaarb: bridge control possible
[ 2.912636] PCI: pci_cache_line_size set to 64 bytes
[ 3.114514] pci 0000:00:15.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 03] add_size 200000 add_align 100000
[ 3.114836] pci 0000:00:15.3: bridge window [io 0x1000-0x0fff] to [bus 06] add_size 1000
[ 3.115079] pci 0000:00:15.4: bridge window [io 0x1000-0x0fff] to [bus 07] add_size 1000
[ 3.115327] pci 0000:00:15.5: bridge window [io 0x1000-0x0fff] to [bus 08] add_size 1000
[ 3.115570] pci 0000:00:15.6: bridge window [io 0x1000-0x0fff] to [bus 09] add_size 1000
[ 3.115812] pci 0000:00:15.7: bridge window [io 0x1000-0x0fff] to [bus 0a] add_size 1000
[ 3.116055] pci 0000:00:16.0: bridge window [io 0x1000-0x0fff] to [bus 0b] add_size 1000
[ 3.116298] pci 0000:00:16.0: bridge window [mem 0x00100000-0x000fffff] to [bus 0b] add_size 200000 add_align 100000
[ 3.116594] pci 0000:00:16.3: bridge window [io 0x1000-0x0fff] to [bus 0e] add_size 1000
[ 3.116836] pci 0000:00:16.4: bridge window [io 0x1000-0x0fff] to [bus 0f] add_size 1000
[ 3.117093] pci 0000:00:16.5: bridge window [io 0x1000-0x0fff] to [bus 10] add_size 1000
[ 3.117336] pci 0000:00:16.6: bridge window [io 0x1000-0x0fff] to [bus 11] add_size 1000
[ 3.117578] pci 0000:00:16.7: bridge window [io 0x1000-0x0fff] to [bus 12] add_size 1000
[ 3.117827] pci 0000:00:17.3: bridge window [io 0x1000-0x0fff] to [bus 16] add_size 1000
[ 3.118070] pci 0000:00:17.4: bridge window [io 0x1000-0x0fff] to [bus 17] add_size 1000
[ 3.118319] pci 0000:00:17.5: bridge window [io 0x1000-0x0fff] to [bus 18] add_size 1000
[ 3.118562] pci 0000:00:17.6: bridge window [io 0x1000-0x0fff] to [bus 19] add_size 1000
[ 3.118804] pci 0000:00:17.7: bridge window [io 0x1000-0x0fff] to [bus 1a] add_size 1000
[ 3.119049] pci 0000:00:18.3: bridge window [io 0x1000-0x0fff] to [bus 1e] add_size 1000
[ 3.119291] pci 0000:00:18.4: bridge window [io 0x1000-0x0fff] to [bus 1f] add_size 1000
[ 3.119533] pci 0000:00:18.5: bridge window [io 0x1000-0x0fff] to [bus 20] add_size 1000
[ 3.119774] pci 0000:00:18.6: bridge window [io 0x1000-0x0fff] to [bus 21] add_size 1000
[ 3.120016] pci 0000:00:18.7: bridge window [io 0x1000-0x0fff] to [bus 22] add_size 1000
[ 3.120344] pci 0000:00:15.0: BAR 15: assigned [mem 0xc0000000-0xc01fffff 64bit pref]
[ 3.120583] pci 0000:00:16.0: BAR 14: assigned [mem 0xc0200000-0xc03fffff]
[ 3.120792] pci 0000:00:0f.0: BAR 6: assigned [mem 0xc0400000-0xc0407fff pref]
[ 3.121033] pci 0000:00:10.0: BAR 6: assigned [mem 0xc0408000-0xc040bfff pref]
[ 3.121269] pci 0000:00:15.3: BAR 13: no space for [io size 0x1000]
[ 3.121460] pci 0000:00:15.3: BAR 13: failed to assign [io size 0x1000]
[ 3.121665] pci 0000:00:15.4: BAR 13: no space for [io size 0x1000]
[ 3.121856] pci 0000:00:15.4: BAR 13: failed to assign [io size 0x1000]
[ 3.122061] pci 0000:00:15.5: BAR 13: no space for [io size 0x1000]
[ 3.122251] pci 0000:00:15.5: BAR 13: failed to assign [io size 0x1000]
[ 3.122455] pci 0000:00:15.6: BAR 13: no space for [io size 0x1000]
[ 3.122646] pci 0000:00:15.6: BAR 13: failed to assign [io size 0x1000]
[ 3.122850] pci 0000:00:15.7: BAR 13: no space for [io size 0x1000]
[ 3.123040] pci 0000:00:15.7: BAR 13: failed to assign [io size 0x1000]
[ 3.123244] pci 0000:00:16.0: BAR 13: no space for [io size 0x1000]
[ 3.123434] pci 0000:00:16.0: BAR 13: failed to assign [io size 0x1000]
[ 3.123639] pci 0000:00:16.3: BAR 13: no space for [io size 0x1000]
[ 3.123829] pci 0000:00:16.3: BAR 13: failed to assign [io size 0x1000]
[ 3.124033] pci 0000:00:16.4: BAR 13: no space for [io size 0x1000]
[ 3.124996] pci 0000:00:16.4: BAR 13: failed to assign [io size 0x1000]
[ 3.125178] pci 0000:00:16.5: BAR 13: no space for [io size 0x1000]
[ 3.125341] pci 0000:00:16.5: BAR 13: failed to assign [io size 0x1000]
[ 3.125516] pci 0000:00:16.6: BAR 13: no space for [io size 0x1000]
[ 3.125679] pci 0000:00:16.6: BAR 13: failed to assign [io size 0x1000]
[ 3.125859] pci 0000:00:16.7: BAR 13: no space for [io size 0x1000]
[ 3.126023] pci 0000:00:16.7: BAR 13: failed to assign [io size 0x1000]
[ 3.126198] pci 0000:00:17.3: BAR 13: no space for [io size 0x1000]
[ 3.126361] pci 0000:00:17.3: BAR 13: failed to assign [io size 0x1000]
[ 3.126538] pci 0000:00:17.4: BAR 13: no space for [io size 0x1000]
[ 3.126702] pci 0000:00:17.4: BAR 13: failed to assign [io size 0x1000]
[ 3.126877] pci 0000:00:17.5: BAR 13: no space for [io size 0x1000]
[ 3.127040] pci 0000:00:17.5: BAR 13: failed to assign [io size 0x1000]
[ 3.127215] pci 0000:00:17.6: BAR 13: no space for [io size 0x1000]
[ 3.127378] pci 0000:00:17.6: BAR 13: failed to assign [io size 0x1000]
[ 3.127553] pci 0000:00:17.7: BAR 13: no space for [io size 0x1000]
[ 3.127716] pci 0000:00:17.7: BAR 13: failed to assign [io size 0x1000]
[ 3.127891] pci 0000:00:18.3: BAR 13: no space for [io size 0x1000]
[ 3.128054] pci 0000:00:18.3: BAR 13: failed to assign [io size 0x1000]
[ 3.128235] pci 0000:00:18.4: BAR 13: no space for [io size 0x1000]
[ 3.128398] pci 0000:00:18.4: BAR 13: failed to assign [io size 0x1000]
[ 3.128573] pci 0000:00:18.5: BAR 13: no space for [io size 0x1000]
[ 3.128736] pci 0000:00:18.5: BAR 13: failed to assign [io size 0x1000]
[ 3.128911] pci 0000:00:18.6: BAR 13: no space for [io size 0x1000]
[ 3.129086] pci 0000:00:18.6: BAR 13: failed to assign [io size 0x1000]
[ 3.129262] pci 0000:00:18.7: BAR 13: no space for [io size 0x1000]
[ 3.129425] pci 0000:00:18.7: BAR 13: failed to assign [io size 0x1000]
[ 3.129612] pci 0000:00:18.7: BAR 13: no space for [io size 0x1000]
[ 3.129775] pci 0000:00:18.7: BAR 13: failed to assign [io size 0x1000]
[ 3.129950] pci 0000:00:18.6: BAR 13: no space for [io size 0x1000]
[ 3.130113] pci 0000:00:18.6: BAR 13: failed to assign [io size 0x1000]
[ 3.130289] pci 0000:00:18.5: BAR 13: no space for [io size 0x1000]
[ 3.130452] pci 0000:00:18.5: BAR 13: failed to assign [io size 0x1000]
[ 3.130627] pci 0000:00:18.4: BAR 13: no space for [io size 0x1000]
[ 3.130790] pci 0000:00:18.4: BAR 13: failed to assign [io size 0x1000]
[ 3.130967] pci 0000:00:18.3: BAR 13: no space for [io size 0x1000]
[ 3.131130] pci 0000:00:18.3: BAR 13: failed to assign [io size 0x1000]
[ 3.131312] pci 0000:00:17.7: BAR 13: no space for [io size 0x1000]
[ 3.131476] pci 0000:00:17.7: BAR 13: failed to assign [io size 0x1000]
[ 3.131651] pci 0000:00:17.6: BAR 13: no space for [io size 0x1000]
[ 3.131815] pci 0000:00:17.6: BAR 13: failed to assign [io size 0x1000]
[ 3.131990] pci 0000:00:17.5: BAR 13: no space for [io size 0x1000]
[ 3.132155] pci 0000:00:17.5: BAR 13: failed to assign [io size 0x1000]
[ 3.132330] pci 0000:00:17.4: BAR 13: no space for [io size 0x1000]
[ 3.132493] pci 0000:00:17.4: BAR 13: failed to assign [io size 0x1000]
[ 3.132669] pci 0000:00:17.3: BAR 13: no space for [io size 0x1000]
[ 3.132832] pci 0000:00:17.3: BAR 13: failed to assign [io size 0x1000]
[ 3.133028] pci 0000:00:16.7: BAR 13: no space for [io size 0x1000]
[ 3.133193] pci 0000:00:16.7: BAR 13: failed to assign [io size 0x1000]
[ 3.133368] pci 0000:00:16.6: BAR 13: no space for [io size 0x1000]
[ 3.133531] pci 0000:00:16.6: BAR 13: failed to assign [io size 0x1000]
[ 3.133706] pci 0000:00:16.5: BAR 13: no space for [io size 0x1000]
[ 3.133870] pci 0000:00:16.5: BAR 13: failed to assign [io size 0x1000]
[ 3.134045] pci 0000:00:16.4: BAR 13: no space for [io size 0x1000]
[ 3.134208] pci 0000:00:16.4: BAR 13: failed to assign [io size 0x1000]
[ 3.134383] pci 0000:00:16.3: BAR 13: no space for [io size 0x1000]
[ 3.134546] pci 0000:00:16.3: BAR 13: failed to assign [io size 0x1000]
[ 3.134721] pci 0000:00:16.0: BAR 13: no space for [io size 0x1000]
[ 3.134884] pci 0000:00:16.0: BAR 13: failed to assign [io size 0x1000]
[ 3.135059] pci 0000:00:15.7: BAR 13: no space for [io size 0x1000]
[ 3.135671] pci 0000:00:15.7: BAR 13: failed to assign [io size 0x1000]
[ 3.135847] pci 0000:00:15.6: BAR 13: no space for [io size 0x1000]
[ 3.136011] pci 0000:00:15.6: BAR 13: failed to assign [io size 0x1000]
[ 3.136186] pci 0000:00:15.5: BAR 13: no space for [io size 0x1000]
[ 3.136349] pci 0000:00:15.5: BAR 13: failed to assign [io size 0x1000]
[ 3.136524] pci 0000:00:15.4: BAR 13: no space for [io size 0x1000]
[ 3.136687] pci 0000:00:15.4: BAR 13: failed to assign [io size 0x1000]
[ 3.136862] pci 0000:00:15.3: BAR 13: no space for [io size 0x1000]
[ 3.137039] pci 0000:00:15.3: BAR 13: failed to assign [io size 0x1000]
[ 3.137212] pci 0000:00:01.0: PCI bridge to [bus 01]
[ 3.137503] pci 0000:02:01.0: BAR 6: assigned [mem 0xfd500000-0xfd50ffff pref]
[ 3.137691] pci 0000:00:11.0: PCI bridge to [bus 02]
[ 3.137845] pci 0000:00:11.0: bridge window [io 0x2000-0x3fff]
[ 3.138049] pci 0000:00:11.0: bridge window [mem 0xfd500000-0xfdffffff]
[ 3.138253] pci 0000:00:11.0: bridge window [mem 0xe7b00000-0xe7ffffff 64bit pref]
[ 3.138513] pci 0000:03:00.0: BAR 6: assigned [mem 0xfd400000-0xfd40ffff pref]
[ 3.138701] pci 0000:00:15.0: PCI bridge to [bus 03]
[ 3.138853] pci 0000:00:15.0: bridge window [io 0x4000-0x4fff]
[ 3.139056] pci 0000:00:15.0: bridge window [mem 0xfd400000-0xfd4fffff]
[ 3.139259] pci 0000:00:15.0: bridge window [mem 0xc0000000-0xc01fffff 64bit pref]
[ 3.139562] pci 0000:00:15.1: PCI bridge to [bus 04]
[ 3.139717] pci 0000:00:15.1: bridge window [io 0x7000-0x7fff]
[ 3.139921] pci 0000:00:15.1: bridge window [mem 0xfd100000-0xfd1fffff]
[ 3.140123] pci 0000:00:15.1: bridge window [mem 0xe3d00000-0xe3dfffff 64bit pref]
[ 3.140436] pci 0000:00:15.2: PCI bridge to [bus 05]
[ 3.140592] pci 0000:00:15.2: bridge window [io 0xb000-0xbfff]
[ 3.140796] pci 0000:00:15.2: bridge window [mem 0xfcd00000-0xfcdfffff]
[ 3.141014] pci 0000:00:15.2: bridge window [mem 0xe3900000-0xe39fffff 64bit pref]
[ 3.141289] pci 0000:00:15.3: PCI bridge to [bus 06]
[ 3.141473] pci 0000:00:15.3: bridge window [mem 0xfc900000-0xfc9fffff]
[ 3.141676] pci 0000:00:15.3: bridge window [mem 0xe3500000-0xe35fffff 64bit pref]
[ 3.141942] pci 0000:00:15.4: PCI bridge to [bus 07]
[ 3.142126] pci 0000:00:15.4: bridge window [mem 0xfc500000-0xfc5fffff]
[ 3.142330] pci 0000:00:15.4: bridge window [mem 0xe3100000-0xe31fffff 64bit pref]
[ 3.142597] pci 0000:00:15.5: PCI bridge to [bus 08]
[ 3.142780] pci 0000:00:15.5: bridge window [mem 0xfc100000-0xfc1fffff]
[ 3.142983] pci 0000:00:15.5: bridge window [mem 0xe2d00000-0xe2dfffff 64bit pref]
[ 3.143250] pci 0000:00:15.6: PCI bridge to [bus 09]
[ 3.143434] pci 0000:00:15.6: bridge window [mem 0xfbd00000-0xfbdfffff]
[ 3.143637] pci 0000:00:15.6: bridge window [mem 0xe2900000-0xe29fffff 64bit pref]
[ 3.143903] pci 0000:00:15.7: PCI bridge to [bus 0a]
[ 3.144086] pci 0000:00:15.7: bridge window [mem 0xfb900000-0xfb9fffff]
[ 3.144290] pci 0000:00:15.7: bridge window [mem 0xe2500000-0xe25fffff 64bit pref]
[ 3.144558] pci 0000:00:16.0: PCI bridge to [bus 0b]
[ 3.144740] pci 0000:00:16.0: bridge window [mem 0xc0200000-0xc03fffff]
[ 3.144942] pci 0000:00:16.0: bridge window [mem 0xe4000000-0xe5ffffff 64bit pref]
[ 3.145230] pci 0000:00:16.1: PCI bridge to [bus 0c]
[ 3.145384] pci 0000:00:16.1: bridge window [io 0x8000-0x8fff]
[ 3.145587] pci 0000:00:16.1: bridge window [mem 0xfd000000-0xfd0fffff]
[ 3.145789] pci 0000:00:16.1: bridge window [mem 0xe3c00000-0xe3cfffff 64bit pref]
[ 3.146054] pci 0000:00:16.2: PCI bridge to [bus 0d]
[ 3.146209] pci 0000:00:16.2: bridge window [io 0xc000-0xcfff]
[ 3.146411] pci 0000:00:16.2: bridge window [mem 0xfcc00000-0xfccfffff]
[ 3.146614] pci 0000:00:16.2: bridge window [mem 0xe3800000-0xe38fffff 64bit pref]
[ 3.146880] pci 0000:00:16.3: PCI bridge to [bus 0e]
[ 3.147063] pci 0000:00:16.3: bridge window [mem 0xfc800000-0xfc8fffff]
[ 3.147266] pci 0000:00:16.3: bridge window [mem 0xe3400000-0xe34fffff 64bit pref]
[ 3.147531] pci 0000:00:16.4: PCI bridge to [bus 0f]
[ 3.147713] pci 0000:00:16.4: bridge window [mem 0xfc400000-0xfc4fffff]
[ 3.147915] pci 0000:00:16.4: bridge window [mem 0xe3000000-0xe30fffff 64bit pref]
[ 3.148182] pci 0000:00:16.5: PCI bridge to [bus 10]
[ 3.148411] pci 0000:00:16.5: bridge window [mem 0xfc000000-0xfc0fffff]
[ 3.148615] pci 0000:00:16.5: bridge window [mem 0xe2c00000-0xe2cfffff 64bit pref]
[ 3.148888] pci 0000:00:16.6: PCI bridge to [bus 11]
[ 3.149088] pci 0000:00:16.6: bridge window [mem 0xfbc00000-0xfbcfffff]
[ 3.149292] pci 0000:00:16.6: bridge window [mem 0xe2800000-0xe28fffff 64bit pref]
[ 3.149561] pci 0000:00:16.7: PCI bridge to [bus 12]
[ 3.149743] pci 0000:00:16.7: bridge window [mem 0xfb800000-0xfb8fffff]
[ 3.149945] pci 0000:00:16.7: bridge window [mem 0xe2400000-0xe24fffff 64bit pref]
[ 3.150211] pci 0000:00:17.0: PCI bridge to [bus 13]
[ 3.150364] pci 0000:00:17.0: bridge window [io 0x5000-0x5fff]
[ 3.150567] pci 0000:00:17.0: bridge window [mem 0xfd300000-0xfd3fffff]
[ 3.150769] pci 0000:00:17.0: bridge window [mem 0xe3f00000-0xe3ffffff 64bit pref]
[ 3.151035] pci 0000:00:17.1: PCI bridge to [bus 14]
[ 3.151189] pci 0000:00:17.1: bridge window [io 0x9000-0x9fff]
[ 3.151392] pci 0000:00:17.1: bridge window [mem 0xfcf00000-0xfcffffff]
[ 3.151593] pci 0000:00:17.1: bridge window [mem 0xe3b00000-0xe3bfffff 64bit pref]
[ 3.151884] pci 0000:00:17.2: PCI bridge to [bus 15]
[ 3.152039] pci 0000:00:17.2: bridge window [io 0xd000-0xdfff]
[ 3.152244] pci 0000:00:17.2: bridge window [mem 0xfcb00000-0xfcbfffff]
[ 3.152445] pci 0000:00:17.2: bridge window [mem 0xe3700000-0xe37fffff 64bit pref]
[ 3.152713] pci 0000:00:17.3: PCI bridge to [bus 16]
[ 3.152895] pci 0000:00:17.3: bridge window [mem 0xfc700000-0xfc7fffff]
[ 3.153113] pci 0000:00:17.3: bridge window [mem 0xe3300000-0xe33fffff 64bit pref]
[ 3.153382] pci 0000:00:17.4: PCI bridge to [bus 17]
[ 3.153565] pci 0000:00:17.4: bridge window [mem 0xfc300000-0xfc3fffff]
[ 3.153767] pci 0000:00:17.4: bridge window [mem 0xe2f00000-0xe2ffffff 64bit pref]
[ 3.154033] pci 0000:00:17.5: PCI bridge to [bus 18]
[ 3.154216] pci 0000:00:17.5: bridge window [mem 0xfbf00000-0xfbffffff]
[ 3.154417] pci 0000:00:17.5: bridge window [mem 0xe2b00000-0xe2bfffff 64bit pref]
[ 3.154683] pci 0000:00:17.6: PCI bridge to [bus 19]
[ 3.154865] pci 0000:00:17.6: bridge window [mem 0xfbb00000-0xfbbfffff]
[ 3.155067] pci 0000:00:17.6: bridge window [mem 0xe2700000-0xe27fffff 64bit pref]
[ 3.155408] pci 0000:00:17.7: PCI bridge to [bus 1a]
[ 3.155651] pci 0000:00:17.7: bridge window [mem 0xfb700000-0xfb7fffff]
[ 3.155867] pci 0000:00:17.7: bridge window [mem 0xe2300000-0xe23fffff 64bit pref]
[ 3.156140] pci 0000:00:18.0: PCI bridge to [bus 1b]
[ 3.156295] pci 0000:00:18.0: bridge window [io 0x6000-0x6fff]
[ 3.156499] pci 0000:00:18.0: bridge window [mem 0xfd200000-0xfd2fffff]
[ 3.156701] pci 0000:00:18.0: bridge window [mem 0xe3e00000-0xe3efffff 64bit pref]
[ 3.156969] pci 0000:00:18.1: PCI bridge to [bus 1c]
[ 3.157141] pci 0000:00:18.1: bridge window [io 0xa000-0xafff]
[ 3.157348] pci 0000:00:18.1: bridge window [mem 0xfce00000-0xfcefffff]
[ 3.157550] pci 0000:00:18.1: bridge window [mem 0xe3a00000-0xe3afffff 64bit pref]
[ 3.157817] pci 0000:00:18.2: PCI bridge to [bus 1d]
[ 3.157970] pci 0000:00:18.2: bridge window [io 0xe000-0xefff]
[ 3.158175] pci 0000:00:18.2: bridge window [mem 0xfca00000-0xfcafffff]
[ 3.158377] pci 0000:00:18.2: bridge window [mem 0xe3600000-0xe36fffff 64bit pref]
[ 3.158643] pci 0000:00:18.3: PCI bridge to [bus 1e]
[ 3.158826] pci 0000:00:18.3: bridge window [mem 0xfc600000-0xfc6fffff]
[ 3.159028] pci 0000:00:18.3: bridge window [mem 0xe3200000-0xe32fffff 64bit pref]
[ 3.159295] pci 0000:00:18.4: PCI bridge to [bus 1f]
[ 3.159478] pci 0000:00:18.4: bridge window [mem 0xfc200000-0xfc2fffff]
[ 3.159680] pci 0000:00:18.4: bridge window [mem 0xe2e00000-0xe2efffff 64bit pref]
[ 3.159946] pci 0000:00:18.5: PCI bridge to [bus 20]
[ 3.160129] pci 0000:00:18.5: bridge window [mem 0xfbe00000-0xfbefffff]
[ 3.160332] pci 0000:00:18.5: bridge window [mem 0xe2a00000-0xe2afffff 64bit pref]
[ 3.160598] pci 0000:00:18.6: PCI bridge to [bus 21]
[ 3.160780] pci 0000:00:18.6: bridge window [mem 0xfba00000-0xfbafffff]
[ 3.160998] pci 0000:00:18.6: bridge window [mem 0xe2600000-0xe26fffff 64bit pref]
[ 3.161270] pci 0000:00:18.7: PCI bridge to [bus 22]
[ 3.161454] pci 0000:00:18.7: bridge window [mem 0xfb600000-0xfb6fffff]
[ 3.161656] pci 0000:00:18.7: bridge window [mem 0xe2200000-0xe22fffff 64bit pref]
[ 3.161924] pci_bus 0000:00: resource 4 [mem 0x000a0000-0x000bffff window]
[ 3.162098] pci_bus 0000:00: resource 5 [mem 0x000cc000-0x000cffff window]
[ 3.162272] pci_bus 0000:00: resource 6 [mem 0x000d0000-0x000d3fff window]
[ 3.162445] pci_bus 0000:00: resource 7 [mem 0x000d4000-0x000d7fff window]
[ 3.162619] pci_bus 0000:00: resource 8 [mem 0x000d8000-0x000dbfff window]
[ 3.162793] pci_bus 0000:00: resource 9 [mem 0xc0000000-0xfebfffff window]
[ 3.162966] pci_bus 0000:00: resource 10 [io 0x0000-0x0cf7 window]
[ 3.163128] pci_bus 0000:00: resource 11 [io 0x0d00-0xfeff window]
[ 3.163291] pci_bus 0000:02: resource 0 [io 0x2000-0x3fff]
[ 3.163439] pci_bus 0000:02: resource 1 [mem 0xfd500000-0xfdffffff]
[ 3.163601] pci_bus 0000:02: resource 2 [mem 0xe7b00000-0xe7ffffff 64bit pref]
[ 3.163789] pci_bus 0000:02: resource 4 [mem 0x000a0000-0x000bffff window]
[ 3.163962] pci_bus 0000:02: resource 5 [mem 0x000cc000-0x000cffff window]
[ 3.164136] pci_bus 0000:02: resource 6 [mem 0x000d0000-0x000d3fff window]
[ 3.164316] pci_bus 0000:02: resource 7 [mem 0x000d4000-0x000d7fff window]
[ 3.164489] pci_bus 0000:02: resource 8 [mem 0x000d8000-0x000dbfff window]
[ 3.164663] pci_bus 0000:02: resource 9 [mem 0xc0000000-0xfebfffff window]
[ 3.164836] pci_bus 0000:02: resource 10 [io 0x0000-0x0cf7 window]
[ 3.165011] pci_bus 0000:02: resource 11 [io 0x0d00-0xfeff window]
[ 3.165179] pci_bus 0000:03: resource 0 [io 0x4000-0x4fff]
[ 3.165328] pci_bus 0000:03: resource 1 [mem 0xfd400000-0xfd4fffff]
[ 3.165923] pci_bus 0000:03: resource 2 [mem 0xc0000000-0xc01fffff 64bit pref]
[ 3.166111] pci_bus 0000:04: resource 0 [io 0x7000-0x7fff]
[ 3.166259] pci_bus 0000:04: resource 1 [mem 0xfd100000-0xfd1fffff]
[ 3.166421] pci_bus 0000:04: resource 2 [mem 0xe3d00000-0xe3dfffff 64bit pref]
[ 3.166608] pci_bus 0000:05: resource 0 [io 0xb000-0xbfff]
[ 3.166756] pci_bus 0000:05: resource 1 [mem 0xfcd00000-0xfcdfffff]
[ 3.166918] pci_bus 0000:05: resource 2 [mem 0xe3900000-0xe39fffff 64bit pref]
[ 3.167105] pci_bus 0000:06: resource 1 [mem 0xfc900000-0xfc9fffff]
[ 3.167266] pci_bus 0000:06: resource 2 [mem 0xe3500000-0xe35fffff 64bit pref]
[ 3.167454] pci_bus 0000:07: resource 1 [mem 0xfc500000-0xfc5fffff]
[ 3.167615] pci_bus 0000:07: resource 2 [mem 0xe3100000-0xe31fffff 64bit pref]
[ 3.167802] pci_bus 0000:08: resource 1 [mem 0xfc100000-0xfc1fffff]
[ 3.167964] pci_bus 0000:08: resource 2 [mem 0xe2d00000-0xe2dfffff 64bit pref]
[ 3.168150] pci_bus 0000:09: resource 1 [mem 0xfbd00000-0xfbdfffff]
[ 3.168319] pci_bus 0000:09: resource 2 [mem 0xe2900000-0xe29fffff 64bit pref]
[ 3.168507] pci_bus 0000:0a: resource 1 [mem 0xfb900000-0xfb9fffff]
[ 3.168668] pci_bus 0000:0a: resource 2 [mem 0xe2500000-0xe25fffff 64bit pref]
[ 3.168855] pci_bus 0000:0b: resource 1 [mem 0xc0200000-0xc03fffff]
[ 3.169040] pci_bus 0000:0b: resource 2 [mem 0xe4000000-0xe5ffffff 64bit pref]
[ 3.169229] pci_bus 0000:0c: resource 0 [io 0x8000-0x8fff]
[ 3.169377] pci_bus 0000:0c: resource 1 [mem 0xfd000000-0xfd0fffff]
[ 3.169540] pci_bus 0000:0c: resource 2 [mem 0xe3c00000-0xe3cfffff 64bit pref]
[ 3.169728] pci_bus 0000:0d: resource 0 [io 0xc000-0xcfff]
[ 3.169876] pci_bus 0000:0d: resource 1 [mem 0xfcc00000-0xfccfffff]
[ 3.170039] pci_bus 0000:0d: resource 2 [mem 0xe3800000-0xe38fffff 64bit pref]
[ 3.170227] pci_bus 0000:0e: resource 1 [mem 0xfc800000-0xfc8fffff]
[ 3.170389] pci_bus 0000:0e: resource 2 [mem 0xe3400000-0xe34fffff 64bit pref]
[ 3.170577] pci_bus 0000:0f: resource 1 [mem 0xfc400000-0xfc4fffff]
[ 3.170740] pci_bus 0000:0f: resource 2 [mem 0xe3000000-0xe30fffff 64bit pref]
[ 3.170928] pci_bus 0000:10: resource 1 [mem 0xfc000000-0xfc0fffff]
[ 3.171090] pci_bus 0000:10: resource 2 [mem 0xe2c00000-0xe2cfffff 64bit pref]
[ 3.171285] pci_bus 0000:11: resource 1 [mem 0xfbc00000-0xfbcfffff]
[ 3.171447] pci_bus 0000:11: resource 2 [mem 0xe2800000-0xe28fffff 64bit pref]
[ 3.171635] pci_bus 0000:12: resource 1 [mem 0xfb800000-0xfb8fffff]
[ 3.171798] pci_bus 0000:12: resource 2 [mem 0xe2400000-0xe24fffff 64bit pref]
[ 3.171986] pci_bus 0000:13: resource 0 [io 0x5000-0x5fff]
[ 3.172135] pci_bus 0000:13: resource 1 [mem 0xfd300000-0xfd3fffff]
[ 3.172297] pci_bus 0000:13: resource 2 [mem 0xe3f00000-0xe3ffffff 64bit pref]
[ 3.172484] pci_bus 0000:14: resource 0 [io 0x9000-0x9fff]
[ 3.172632] pci_bus 0000:14: resource 1 [mem 0xfcf00000-0xfcffffff]
[ 3.172793] pci_bus 0000:14: resource 2 [mem 0xe3b00000-0xe3bfffff 64bit pref]
[ 3.172983] pci_bus 0000:15: resource 0 [io 0xd000-0xdfff]
[ 3.173140] pci_bus 0000:15: resource 1 [mem 0xfcb00000-0xfcbfffff]
[ 3.173302] pci_bus 0000:15: resource 2 [mem 0xe3700000-0xe37fffff 64bit pref]
[ 3.173489] pci_bus 0000:16: resource 1 [mem 0xfc700000-0xfc7fffff]
[ 3.173650] pci_bus 0000:16: resource 2 [mem 0xe3300000-0xe33fffff 64bit pref]
[ 3.173837] pci_bus 0000:17: resource 1 [mem 0xfc300000-0xfc3fffff]
[ 3.173999] pci_bus 0000:17: resource 2 [mem 0xe2f00000-0xe2ffffff 64bit pref]
[ 3.174192] pci_bus 0000:18: resource 1 [mem 0xfbf00000-0xfbffffff]
[ 3.174353] pci_bus 0000:18: resource 2 [mem 0xe2b00000-0xe2bfffff 64bit pref]
[ 3.174540] pci_bus 0000:19: resource 1 [mem 0xfbb00000-0xfbbfffff]
[ 3.174702] pci_bus 0000:19: resource 2 [mem 0xe2700000-0xe27fffff 64bit pref]
[ 3.174889] pci_bus 0000:1a: resource 1 [mem 0xfb700000-0xfb7fffff]
[ 3.175051] pci_bus 0000:1a: resource 2 [mem 0xe2300000-0xe23fffff 64bit pref]
[ 3.175245] pci_bus 0000:1b: resource 0 [io 0x6000-0x6fff]
[ 3.175394] pci_bus 0000:1b: resource 1 [mem 0xfd200000-0xfd2fffff]
[ 3.175555] pci_bus 0000:1b: resource 2 [mem 0xe3e00000-0xe3efffff 64bit pref]
[ 3.175742] pci_bus 0000:1c: resource 0 [io 0xa000-0xafff]
[ 3.175890] pci_bus 0000:1c: resource 1 [mem 0xfce00000-0xfcefffff]
[ 3.176051] pci_bus 0000:1c: resource 2 [mem 0xe3a00000-0xe3afffff 64bit pref]
[ 3.176238] pci_bus 0000:1d: resource 0 [io 0xe000-0xefff]
[ 3.176386] pci_bus 0000:1d: resource 1 [mem 0xfca00000-0xfcafffff]
[ 3.176547] pci_bus 0000:1d: resource 2 [mem 0xe3600000-0xe36fffff 64bit pref]
[ 3.176734] pci_bus 0000:1e: resource 1 [mem 0xfc600000-0xfc6fffff]
[ 3.176896] pci_bus 0000:1e: resource 2 [mem 0xe3200000-0xe32fffff 64bit pref]
[ 3.177094] pci_bus 0000:1f: resource 1 [mem 0xfc200000-0xfc2fffff]
[ 3.177257] pci_bus 0000:1f: resource 2 [mem 0xe2e00000-0xe2efffff 64bit pref]
[ 3.177444] pci_bus 0000:20: resource 1 [mem 0xfbe00000-0xfbefffff]
[ 3.177605] pci_bus 0000:20: resource 2 [mem 0xe2a00000-0xe2afffff 64bit pref]
[ 3.177792] pci_bus 0000:21: resource 1 [mem 0xfba00000-0xfbafffff]
[ 3.177954] pci_bus 0000:21: resource 2 [mem 0xe2600000-0xe26fffff 64bit pref]
[ 3.178141] pci_bus 0000:22: resource 1 [mem 0xfb600000-0xfb6fffff]
[ 3.178308] pci_bus 0000:22: resource 2 [mem 0xe2200000-0xe22fffff 64bit pref]
[ 3.202983] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 3.203380] pci 0000:00:0f.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 4.443686] pcieport 0000:00:15.0: PME: Signaling with IRQ 24
[ 4.444098] pcieport 0000:00:15.0: pciehp: Slot #160 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.446445] pcieport 0000:00:15.1: PME: Signaling with IRQ 25
[ 4.446858] pcieport 0000:00:15.1: pciehp: Slot #161 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.449203] pcieport 0000:00:15.2: PME: Signaling with IRQ 26
[ 4.449611] pcieport 0000:00:15.2: pciehp: Slot #162 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.451919] pcieport 0000:00:15.3: PME: Signaling with IRQ 27
[ 4.452322] pcieport 0000:00:15.3: pciehp: Slot #163 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.454644] pcieport 0000:00:15.4: PME: Signaling with IRQ 28
[ 4.455053] pcieport 0000:00:15.4: pciehp: Slot #164 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.457400] pcieport 0000:00:15.5: PME: Signaling with IRQ 29
[ 4.457811] pcieport 0000:00:15.5: pciehp: Slot #165 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.460121] pcieport 0000:00:15.6: PME: Signaling with IRQ 30
[ 4.460550] pcieport 0000:00:15.6: pciehp: Slot #166 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.462884] pcieport 0000:00:15.7: PME: Signaling with IRQ 31
[ 4.463288] pcieport 0000:00:15.7: pciehp: Slot #167 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.465661] pcieport 0000:00:16.0: PME: Signaling with IRQ 32
[ 4.466071] pcieport 0000:00:16.0: pciehp: Slot #192 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.468342] pcieport 0000:00:16.1: PME: Signaling with IRQ 33
[ 4.468745] pcieport 0000:00:16.1: pciehp: Slot #193 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.471186] pcieport 0000:00:16.2: PME: Signaling with IRQ 34
[ 4.471594] pcieport 0000:00:16.2: pciehp: Slot #194 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.473900] pcieport 0000:00:16.3: PME: Signaling with IRQ 35
[ 4.474311] pcieport 0000:00:16.3: pciehp: Slot #195 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.476624] pcieport 0000:00:16.4: PME: Signaling with IRQ 36
[ 4.477656] pcieport 0000:00:16.4: pciehp: Slot #196 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.479948] pcieport 0000:00:16.5: PME: Signaling with IRQ 37
[ 4.480368] pcieport 0000:00:16.5: pciehp: Slot #197 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.482698] pcieport 0000:00:16.6: PME: Signaling with IRQ 38
[ 4.483108] pcieport 0000:00:16.6: pciehp: Slot #198 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.485430] pcieport 0000:00:16.7: PME: Signaling with IRQ 39
[ 4.485846] pcieport 0000:00:16.7: pciehp: Slot #199 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.488220] pcieport 0000:00:17.0: PME: Signaling with IRQ 40
[ 4.488623] pcieport 0000:00:17.0: pciehp: Slot #224 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.490989] pcieport 0000:00:17.1: PME: Signaling with IRQ 41
[ 4.491394] pcieport 0000:00:17.1: pciehp: Slot #225 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.493874] pcieport 0000:00:17.2: PME: Signaling with IRQ 42
[ 4.494289] pcieport 0000:00:17.2: pciehp: Slot #226 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.496605] pcieport 0000:00:17.3: PME: Signaling with IRQ 43
[ 4.497027] pcieport 0000:00:17.3: pciehp: Slot #227 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.499349] pcieport 0000:00:17.4: PME: Signaling with IRQ 44
[ 4.499753] pcieport 0000:00:17.4: pciehp: Slot #228 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.502111] pcieport 0000:00:17.5: PME: Signaling with IRQ 45
[ 4.502522] pcieport 0000:00:17.5: pciehp: Slot #229 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.504825] pcieport 0000:00:17.6: PME: Signaling with IRQ 46
[ 4.505260] pcieport 0000:00:17.6: pciehp: Slot #230 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.507660] pcieport 0000:00:17.7: PME: Signaling with IRQ 47
[ 4.508062] pcieport 0000:00:17.7: pciehp: Slot #231 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.510392] pcieport 0000:00:18.0: PME: Signaling with IRQ 48
[ 4.510828] pcieport 0000:00:18.0: pciehp: Slot #256 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.513144] pcieport 0000:00:18.1: PME: Signaling with IRQ 49
[ 4.513558] pcieport 0000:00:18.1: pciehp: Slot #257 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.515943] pcieport 0000:00:18.2: PME: Signaling with IRQ 50
[ 4.516348] pcieport 0000:00:18.2: pciehp: Slot #258 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.518690] pcieport 0000:00:18.3: PME: Signaling with IRQ 51
[ 4.519094] pcieport 0000:00:18.3: pciehp: Slot #259 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.521474] pcieport 0000:00:18.4: PME: Signaling with IRQ 52
[ 4.521884] pcieport 0000:00:18.4: pciehp: Slot #260 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.524195] pcieport 0000:00:18.5: PME: Signaling with IRQ 53
[ 4.524598] pcieport 0000:00:18.5: pciehp: Slot #261 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.526976] pcieport 0000:00:18.6: PME: Signaling with IRQ 54
[ 4.527380] pcieport 0000:00:18.6: pciehp: Slot #262 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.529765] pcieport 0000:00:18.7: PME: Signaling with IRQ 55
[ 4.530179] pcieport 0000:00:18.7: pciehp: Slot #263 AttnBtn+ PwrCtrl+ MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl+ LLActRep+
[ 4.715235] ehci-pci: EHCI PCI platform driver
[ 4.715776] ohci-pci: OHCI PCI platform driver
[ 6.370321] mlx5_core 0000:0b:00.0: mlx5_pcie_event:294:(pid 8): PCIe slot advertised sufficient power (27W).
[ 24.139398] mlx5_core 0000:0b:00.0: mlx5_pcie_event:304:(pid 604): PCIe slot advertised sufficient power (27W).
[ 36.586927] NNT Driver:: function name:create_file_name_mft, line:141 | MFT device name created: id: 4119, slot id: 0, device name: /dev/mst/mt4119_pciconf0 domain: 0x0 bus: 0xb
Hello loganwhitevon,
Thank you for the information. Please refer to this link on “How to configure Nvidia network device in Passthrough mode on VMware ESXi6.x”
The provided lspci o/p is gathered from the VM I presume. Have you ran lspci on the physical machine and compared the o/p to check for discrepancies. I would suggest you to check with VMware on what PCIe registers could be modified in Passthrough mode.
Thanks & Regards,
~ Nvidia Support
Thank you @sribhargavid. I have exactly followed the instructions you mentioned. However, I can still not able to change the value. Maybe the virtualization layer did some filters that filters out the parameter setting instruction. I have also posted a question on the ESXi forum, but unfortunately, got not response yet. Thank you for your information though, much appreciated.
Logan
Hi all,
I’m having the same issue and still figuring out what the reason might be.
I could not find any documentation of the specific PCIe registers for Nvidia GPUs yet. Is there something publicly available?