PCIe device management app
| Files | Last commit | Last update |
|---|---|---|
| 2 months ago | ||
| 1 month ago | ||
| 1 year ago | ||
| 1 year ago | ||
| 1 day ago | ||
| 1 day ago | ||
| 10 days ago | ||
| 3 months ago | ||
| 1 day ago | ||
| 4 months ago | ||
| 1 day ago | ||
| 1 day ago | ||
| 1 year ago | ||
| 9 days ago | ||
| 1 month ago | ||
| 3 months ago | ||
| 3 months ago | ||
| 1 day ago | ||
| 1 month ago | ||
| 3 months ago | ||
| 4 months ago | ||
| 1 year ago | ||
| 3 months ago | ||
| 3 months ago | ||
| 1 month ago |
pcie_device
Function Overview
- The pcie_device component provides end-to-end management of PCIe devices by the BMC, covering the entire process from service topology parsing and link establishment to device management. It consists of two main parts:
- PCIe device management
- PCIe service topology management (PCIe device loading and high-speed cable detection)
Directory Structure
├── biz_topo -- Service topology directory
│ ├── biz_topo.lua -- PCIe topology establishment
│ ├── biz_topo_service.lua -- Service topology entry
│ ├── class
│ │ ├── base.lua -- Common base class
│ │ ├── bdf_config.lua -- BDF configuration management class
│ │ ├── business_connector.lua -- Service connector class
│ │ ├── ipmi_channel_config.lua -- IPMI channel configuration management class
│ │ ├── load_info.lua -- Bus BDF information class
│ │ ├── pcie_addr_info.lua -- PCIe address information class
│ │ ├── serdes.lua -- SerDes class for connecting the CPU to the UBCDD
│ │ └── unit_configuration.lua -- Cable detection whitelist configuration class
│ ├── def.lua -- Static enumeration
│ ├── device_loader.lua -- PCIe device loading
│ ├── signal_mgr.lua -- PCIe signal management
│ ├── topo_reader.lua -- Topology monitoring module
│ └── topo_monitor.lua -- Cable detection
├── common.lua -- Public functions of the PCIE_DEVICE module
├── common_interface.lua -- Common resource collaboration interface of the PCIE_DEVICE module
├── device
│ ├── class
| | ├── eth_device.lua -- ETH_DEVICE class, which synchronizes the PCIe topology information of the device
| | ├── ocp_card.lua -- OCP_CARD class, which provides northbound and IPMI functions
│ │ ├── pcie_card.lua -- PCIE_CARD class, which provides northbound and IPMI functions
| | ├── pcie_card_flash_checker.lua -- PCIE_CARD_FLASH_CHECKER class, which monitors the flash status of the PCIe card
│ │ ├── pcie_device.lua -- PCIE_DEVICE class, which implements the methods for getting and setting device information
| | ├── pcie_function.lua -- PCIE_FUNCTION class, which ensures correct management and fault monitoring of the PCIe topology
| | └── pcie_slot.lua -- PCIE_SLOT class, which sets the power status of PCIe slots
│ ├── device_service.lua -- Implements BDF information synchronization and update
| └── jbog_device_mgmt.lua -- JBOG device management
├── device_mgmt
| ├── comm_defs.lua -- Common definitions of DEVICE_MGMT
| ├── comm_fun.lua -- Common functions of DEVICE_MGMT
| ├── device_mgmt.lua -- PCIe device management
| ├── pcie_card_mgmt.lua -- PCIe card management for bidirectional synchronization of PCIe card lifecycle and attributes between the device tree and resource tree
| └── pcie_device_mgmt.lua -- PCIe device management for bidirectional synchronization of the lifecycle and attributes of PCIe devices between the device tree and resource tree
├── imu -- Implementation of IMU interaction
│ └── init.lua
├── infrastructure
│ ├── collection_helper.lua -- Collection library
| ├── event.lua -- Alarm management
│ └── log_helper.lua -- Log collection library
├── pcie_device_app.lua -- Module initialization entry
├── service_manager.lua -- Service management class, which manages the lifecycle of all sub-services in the PCIE_DEVICE service.
└── ub_cable
├── class
| ├── base.lua -- Common base class
| ├── inter_node_cable_configuration.lua -- Parses and manages the cross-node PCIe cable connection topology.
| └── node_port_addr.lua -- Manages cable connections to PCIe device ports, including node port address modeling, cable configuration matching, peer information learning, and connection status alarm determination
├── imu_cmd.lua -- Queries the link setup status of a specific UB port
├── infrastructure
| ├── def.lua -- Common definitions of UB cables
| └── event.lua -- Alarm management
└── ub_cable_detection.lua -- PCIe cable detection and alarm management module
Key Features
Overall Architecture Design
- The architecture of this component consists of four layers: interface, service, domain, and infrastructure.
-
Interface layer: As the unified external exit of the component, it provides interfaces for information exchange between modules.
-
Service layer: implements the coordination and combination interaction logic between multiple internal domain objects and external services. As the exit of each service module, the service layer provides the invoking capability for external components.
device_service: device management service, which is based on the domain objects PCIeDevice and PCIeCard.- Manages PCIe devices (such as PCIe cards and PCIe-based NVMe drives).
biztopo_service: service topology service, which is based on the domain objects BizTopo, TopoMonitor, and DeviceLoader.- Integrates CSR node information, constructs a service topology tree, works with the PSR to dynamically obtain PCIe device links, and starts real-time detection of high-speed topology link connections.
- Provides PCIe device slot information to the BIOS, receives and processes PCIe card information reported by the BIOS, and loads and manages PCIe cards based on the auto-discovery service.
-
Domain layer: defines entities related to PCIe devices, standard PCIe cards, and service topology trees, and implements core service logic, including device loading, high-speed topology detection, and dynamic information obtaining and update by invoking interfaces provided by the infrastructure layer for static attributes.
biz_topo: parses and establishes the service topology of Tianchi basic components, and establishes links between CPUs and PCIe devices based on the entire system configuration.topo_monitor: provides the capability of detecting faults on high-speed links in the service topology.device_loader: manages non-standard PCIe devices based on service topology links and in-band reported information, and provides a device persistence mechanism.PCIeDevice: defines classes abstracted from all PCIe devices, carries PCIe bus-related attribute definitions and fault diagnosis error identifiers, and provides a unified entry for querying and setting common features of PCIe devices.PCIeCard: manages basic information about PCIe cards. This class is an abstract class that carries the common attributes and methods of different types of PCIe cards. The dynamic attributes that can be obtained and the update methods of different types of PCIe cards depend on different protocols. Therefore, each specific type of PCIe card is extended through inheritance, for example, SDI cards.signal_manager: provides a signal management mechanism based on the signal library and PCIe service requirements.
-
Infrastructure layer: encapsulates basic protocols and common functions, which are static capabilities and can be invoked by the upper layer.
common_tools: encapsulates common logic based on the service characteristics of the pcie_device component and provides the logic for the upper layer as functions.
Service Topology Establishment
- Service topology management is a core capability of the pcie_device component. Other features related to PCIe devices depend on the establishment of the service topology. The overall process is as follows:
- SerDes/BusinessConnector (CSR) and UnitConfiguration (PSR) object distribution and loading
- The internal service connector of the BCU is associated with the corresponding CPU resources. The internal business connector of the IEU connects downstream PCIe device connectors with upstream associations.
- Based on the high-speed cable connection relationship provided by the system configuration object UnitConfiguration, the service connector on the IEU side is associated with the service connector on the BCU side. In this way, the PCIe device connector can be associated with the corresponding CPU resources.

PCIe Device Loading
- We have designed a solution for managing standard PCIe devices based on the BDF reported by the BIOS.
- After the service topology is established, the BIOS retrieves the PCIe device slot and CPU resource configuration information from the BMC.
- The BIOS reports the device Bus/Device/Function (BDF) information of the PCIe devices to the BMC.
- The BMC uses the device BDF information to send a request to the PMU and retrieves the PCIe device quadruple information.
- The PCIe device CSR can be loaded based on the PCIe device quaternion, implementing management of standard PCIe devices.

High-Speed Topology/Cable Detection
-
After collecting the system link configuration information and real-time component connection information read from the SMC, the BMC can detect the real-time connection status of the current high-speed link. If the connection is incorrect, the BMC reports an alarm in real time to guide local O&M personnel to adjust the connection mode in a timely manner.
-
The high-speed topology/cable detection process is as follows:
- Initialize the port status and device configuration information, and identify the valid detection area.
- Run the topology discovery command to read the current topology link connection information from the SMC.
- Compare the real-time topology link connection information with the device link configuration information to analyze whether the current device access status is valid.
- Update the link detection status. If an invalid connection exists, an alarm is reported.


- Cable detection code process

External Interfaces
Resource Collaboration Interface
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - SetPcieErrorStatus
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
SetPcieErrorStatus |
a{ss}yyyyysy |
y |
Set different error event attribute statuses based on the error type. | Set different error event attribute statuses based on the error type. | The BDF information exists and is unique. Value: 0 (not occurred); 1 (occurred). |
N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - SetUCEByBIOS
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
SetUCEByBIOS |
a{ss}yyyyy |
y |
Set the uncorrectable error flag. | Set the uncorrectable error flag. | The BDF information exists and is unique. Value: 0 (no fault); 1 (faulty). |
N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - SetBandwidthReduction
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
SetBandwidthReduction |
a{ss}yyyyy |
y |
Set the bandwidth reduction flag. | Set the bandwidth reduction flag. | The BDF information exists and is unique. Value: 0 (no fault); 1 (faulty). |
N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - SetLinkSpeedReduced
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
SetLinkSpeedReduced |
a{ss}yyyyy |
y |
Set the link rate decrease flag. | Set the link rate decrease flag. | The BDF information exists and is unique. Value: 0 (no fault); 1 (faulty). |
N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - SetDiagnosticFault
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
SetDiagnosticFault |
a{ss}yyyyyy |
y |
Set the major fault alarm flag. | Set the major fault alarm flag. | The BDF information exists and is unique. Value: 0 (no fault); 1 (faulty). |
N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - SetPredictiveFault
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
SetPredictiveFault |
a{ss}yyyyyy |
y |
Set the pre-fault flag. | Set the pre-fault flag. | The BDF information exists and is unique. Value: 0 (no fault); 1 (faulty). |
N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - GetDeviceName
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
GetDeviceName |
a{ss}yyyyy |
ys |
Obtain the silkscreen information of a PCIe device. | Obtain the silkscreen information of a PCIe device. | The BDF information exists and is unique. | N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - GetPCIeCardNum
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
GetPCIeCardNum |
a{ss} |
u |
Obtain the number of PCIe cards. | Obtain the number of PCIe cards. | N/A | Number of PCIe cards | N/A | N/A | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} - SetFaultByBios
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeDevices/${Id} |
bmc.kepler.Systems.PCIeDevices.PCIeDevice |
SetFaultByBios |
a{ss}yyyy |
y |
Set the value of the FaultByBios attribute. | Set the value of the FaultByBios attribute. | The BDF information exists and is unique. FaultStatus value: 0 (no fault); 1 (faulty). |
N/A | N/A | The corresponding PCIe device needs to be loaded and initialized. | N/A | The Id is unique under the SystemId. |
N/A |
/bmc/kepler/Systems/${SystemId}/PCIeSlots/${Id} - PowerControl
| Path | Interface | Method | Input Parameter | Output Parameter | Description | Service Function | Constraint | Unit | Exclusion Relationship | Dependency | Example Format | Uniqueness Requirement | Compatibility Requirement |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
/bmc/kepler/Systems/${SystemId}/PCIeSlots/${Id} |
bmc.kepler.Systems.PCIeSlot |
PowerControl |
a{ss}s |
Control the power supply status of a slot. | Control the power supply status of a slot. | Off: The slot is powered off. On: The slot is powered on. |
N/A | N/A | N/A | N/A | The Id is unique under the SystemId. |
N/A |
IPMI Commands
- For details about the parameters, see the configuration file
ipmi.jsonof this component.
| Method | Description |
|---|---|
| SetCpuBusSize | The BIOS uses this command to report CPU bus information. |
| UpdateIntegratedPcieConfig | The BIOS uses this command to report the BDF information of the integrated PCIe controller in the socket. |
| GetMezzSilkName | The BIOS uses this command to obtain the silkscreen information of the mezzanine card. |
| GetPcieSilkName | The BIOS uses this command to obtain the silkscreen information of the PCIe card. |
| GetNicSilkName | The BIOS uses this command to obtain the silkscreen information of the NIC. |
| GetRaidSilkName | The BIOS uses this command to obtain the silkscreen information of the RAID controller card. |
| GetDeviceLocationName | The BIOS uses this command to query the names and locations of PCIe cards and NICs. |
| GetPCIeCardModel | The BIOS uses this command to query the card model based on the BDF. |
| GetPCIeDevPresentStatus | The BIOS uses this command to obtain the PCIe device presence status. |
| WritePCIeVPDInfoToBmc | The BIOS uses this command to report PCIe's vital product data (VPD) information. |
| SetFDMInfoPCIeAddrInfo | The BIOS uses this command to report and process in-band PCIe topology information. |
| SetFDMInfoMultiSystemPCIeAddrInfo | The BIOS uses this command to report and process the in-band PCIe topology information of multiple hosts. |
Configuration Description
Key Class Definition
- To know which SerDes of which CPU a PCIe device is connected to, the following physical components are introduced:
- PCIeSlot: two PCIe slots on the IEU, corresponding to the two PCIeConnector
- Riser: I/O expander inserted into the BCU, used to insert PCIe devices. There are two types of BusinessConnector objects: upstream and downstream. The UpstreamResources of the downstream object in the riser is configured with the name of the upstream object.
- UBCDD: a connector with BusinessConnector objects (upstream and downstream).
- SerDes-x: also called HiLink, represents a set of interfaces provided by the CPU. The interfaces are connected to the UBCDD.

UnitConfiguration
- UnitConfiguration carries the connection relationship between the BCU and IEU.
- Slot: globally unique and provided by hardware
- SrcPortName: Slot 7 corresponds to B4a(B4b), and slot 8 corresponds to B4c(B4d).
- TargetPortID: upstream port ID of the riser card
- As shown in the figure, SerDes_1_8 with x8 corresponds to B4c (B4d), and SerDes_1_7 and SerDes_1_10 with x4 correspond to B4a (B4b).
(ActualResourceOrder and Ports have a corresponding sequence.)

SerDes
- SerDes carries the CPU resource configuration of the BCU component.
- Device: indicates that the device associated with the SerDes is 14, 14, 15, or 15. It is also the Device of PCIeAddrInfo.
- Socket: indicates the CPU to which the SerDes is connected.

PCIeDevice
- PCIe device management objects:
- BDF: RootBDF, physical BDF, which is fixed based on the slot.
- DevBDF: device BDF, which is allocated by the BIOS.
- FunctionClass: card type
PCIeAddrInfo
- Carries PCIe device slot information.
- Load PCIeAddrInfo and then PCIeDevice. The slot number and BDF in PCIeDevice are synchronized from PCIeAddrInfo.
- The biztopo module sets the PCIeAddrInfo attributes.