PowerEdge: Systems may not log downgraded PCIe links

A server may experience performance issues due to a downgraded PCIe link, but not log it in the LC or SEL. In these cases, refer to the operating system to check for an unexpected downgraded PCIe link.

The BIOS includes checks for unexpected PCIe link downgrade on both upstream and downstream, but there are situations where this could occur after the BIOS check and thus, not be logged.

Use the operating system tools to review negotiated PCIe link speed.

Example excerpt from lspci -vvv output for Linux based operating system:

0001:64:00.0 PCI bridge: Broadcom / LSI PEX890xx PCIe Gen 5 Switch (rev b0) (prog-if 00 [Normal decode])
    Subsystem: Dell PEX890xx PCIe Gen 5 Switch
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    NUMA node: 1
    Bus: primary=64, secondary=65, subordinate=6a, sec-latency=0
    I/O behind bridge: 0000f000-00000fff [disabled] [32-bit]
    Memory behind bridge: db000000-db2fffff [size=3M] [32-bit]
    Prefetchable memory behind bridge: 2e0000000000-2eafffffffff [size=704G] [32-bit]
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
    BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [68] Express (v2) Upstream Port, MSI 00
        DevCap:    MaxPayload 2048 bytes, PhantFunc 0
            ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ SlotPowerLimit 0W
        DevCtl:    CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
            RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 256 bytes, MaxReadReq 128 bytes
        DevSta:    CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap:    Port #200, Speed 32GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <4us, L1 <32us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; Disabled- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 16GT/s (downgraded), Width x16
            TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Not Supported, TimeoutDis- NROPrPrP- LTR-
             10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix+, MaxEETLPPrefixes 4
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS-
             AtomicOpsCap: Routing+
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: EgressBlck-
        LnkCap2: Supported Link Speeds: 2.5-32GT/s, Crosslink+ Retimer+ 2Retimers+ DRS-
        LnkCtl2: Target Link Speed: 32GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
             EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [b0] Subsystem: Dell PEX890xx PCIe Gen 5 Switch
    Capabilities: [e0] Vendor Specific Information: Len=20 <?>
    Capabilities: [100 v1] Device Serial Number 00-80-5e-72-6f-30-7d-18
    Capabilities: [fb4 v1] Vendor Specific Information: ID=0000 Rev=0 Len=008 <?>
    Capabilities: [10c v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn- PerformEqu-
        LaneErrStat: 0
    Capabilities: [150 v1] Virtual Channel
        Caps:    LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:    ArbSelect=Fixed
        Status:    InProgress-
        VC0:    Caps:    PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:    Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:    Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
            Status:    NegoPending- InProgress-
    Capabilities: [1e0 v1] Vendor Specific Information: ID=0001 Rev=2 Len=010 <?>
    Capabilities: [250 v1] Physical Layer 16.0 GT/s <?>
    Capabilities: [280 v1] Extended Capability ID 0x2a
    Capabilities: [180 v1] Lane Margining at the Receiver <?>
    Capabilities: [1d0 v1] Data Link Feature <?>
    Capabilities: [1f0 v1] Advanced Error Reporting
        UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt:    DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt+ UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
        CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:    RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ AdvNonFatalErr+
        AERCap:    First Error Pointer: 1f, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [220 v1] Vendor Specific Information: ID=2000 Rev=1 Len=020 <?>
    Kernel driver in use: pcieport