--- name: ipmi-bmc-management description: Use when working with IPMI/BMC out-of-band server mgmt. version: 1.0.0 author: Hermes Agent license: MIT platforms: [linux] metadata: hermes: tags: [ipmi, bmc, redfish, supermicro, out-of-band, hardware, firmware] related_skills: [proxmox-lxc-deployment, truenas] --- # IPMI / BMC Out-of-Band Management Manage servers out-of-band via their BMC (Baseboard Management Controller). Covers read-only inventory, sensor/thermal readout, firmware version checks, and the modern Redfish REST API. Credentials live in fact_store, never in this skill. ## Install ```bash sudo apt-get install -y ipmitool # CLI (also: freeipmi-tools) ``` ## ipmitool — read-only inventory ```bash ipmitool -I lanplus -H -U -P -C 3 ``` Useful read-only commands: - `mc info` — BMC identity (manufacturer, product ID, firmware rev, IPMI version) - `fru print` — board serial, mfg date, product info - `sensor list` / `sdr list` — thermals, voltages, fans, GPU temps - `chassis status` — power state, fault flags, power-restore policy - `power status` — on/off - `lan print` — BMC network config (IP, MAC, gateway, cipher suites) - `user list` — configured BMC accounts - `sel info` / `sel list` — system event log (check for full/overflow) ## CRITICAL: Supermicro cipher-suite quirk Supermicro BMCs (H12SSL-i and similar) FAIL the default RMCP+ cipher suite (17 / SHA256) with: ``` Error in open session response message : invalid role Error: Unable to establish IPMI v2 / RMCP+ session ``` **Fix: force cipher suite 3** with `-C 3`. This is the single most common failure when scripting Supermicro IPMI. `-C 17` and `-L ADMINISTRATOR` do NOT help — only `-C 3` works. IPMI 1.5 (`-I lan`) also works but is insecure. ## Redfish — the modern API (preferred for firmware inventory) Supermicro BMCs expose a full Redfish REST API. Use it for firmware version checks — it's cleaner than raw OEM IPMI commands (which often return "Request data length invalid"). ```bash # Root (confirms Redfish + vendor) curl -sk -u 'USER:PASS' https:///redfish/v1/ # Firmware inventory — BMC, BIOS, CPLD versions in one shot curl -sk -u 'USER:PASS' https:///redfish/v1/UpdateService/FirmwareInventory # Per-component version curl -sk -u 'USER:PASS' https:///redfish/v1/UpdateService/FirmwareInventory/BMC curl -sk -u 'USER:PASS' https:///redfish/v1/UpdateService/FirmwareInventory/BIOS # System info (BiosVersion, Model, ProcessorSummary) curl -sk -u 'USER:PASS' https:///redfish/v1/Systems/1 ``` FirmwareInventory members are named `BMC`, `BIOS`, `Motherboard_CPLD_1`, `GPU` etc. Each returns `Version`, `Updateable`, `ReleaseDate`. GPUs show `Version: None` (not firmware-managed via BMC). ## Firmware currency check 1. Read current versions via Redfish FirmwareInventory (BMC + BIOS + CPLD). 2. Cross-check against Supermicro download center: `https://www.supermicro.com/en/support/resources/downloadcenter/firmware/MBD-/BIOS` (search result snippets list "BIOS Revision: X.Y" and "BMC Firmware Revision: X.Y.Z"). 3. Report the gap. Note: Supermicro BMC downgrades are NOT supported after security updates — upgrades are one-way. ## Firmware UPDATE (flash) — method decision **The H12/H12SS bundle ships SAA (SuperServer Automation Assistant) UEFI, NOT SUM.** SUM is the older in-band updater; the H12 firmware zip contains `SAA.efi` + `flash.nsh`. Two hard rules from Supermicro's own package readme: 1. **"Using AFU tool will end up with the BIOS corruption. It should never be used!"** — the AMI AFU flasher is forbidden on H12. 2. **BIOS flash on H12SS-and-newer MUST go through the BMC** — DOS/EFI standalone BIOS flashing is no longer supported. The BMC web UI uploads the .bin to the BMC flashdisk and the BMC does the flash. | Method | License? | Notes | |---|---|---| | **BMC web UI → Maintenance → Firmware Management** | No | The supported path for BOTH BMC and BIOS on H12. Upload the .bin; BMC does the flash. ~2.5 min BMC, a few min BIOS. | | SAA UEFI shell (`flash.nsh `) | No | In-band alternative; requires booting host into EFI shell. | | Redfish SimpleUpdate | YES (DCMS) | Returns `SMC.1.0.OemLicenseNotPassed` — "Not licensed... DCMS needed" | | Redfish OEM SmcUpdateService.Install | No | Available (Targets + InstallOptions) but web UI is the documented path | | ipmitool hpm upgrade | No | Only for .hpm images; Supermicro ships .bin | **Probe the license block before assuming Redfish works:** `curl -sk -u 'U:P' https:///redfish/v1/UpdateService/SimpleUpdateActionInfo` — if it returns `OemLicenseNotPassed`, SimpleUpdate is out. **Upgrade order:** BMC first (closes BMC CVEs, one-way), then BIOS (closes CPU microcode/AGESA CVEs). BIOS flash requires host OFF. Apply BMC and BIOS in one session — mismatched BIOS/BMC version pairs can hang at POST code FF. **Recovery safety net:** Supermicro BMCs keep quad-image redundancy — FirmwareInventory lists `BMC/Backup_BMC/Golden_BMC/Staging_BMC` (same for BIOS). A failed flash can boot from Backup/Golden via the web UI "Recover" option or Redfish OEM Install with `InstallOptions=["Recover"]`. **Post-flash checks:** re-test the cipher suite (newer BMC firmware may switch from cipher 3 to 17/SHA256), re-verify login, and re-read FirmwareInventory to confirm the new version. A DHCP BMC may also change IP after reset — re-find by MAC (fixed) via ARP scan if the IP moves. ## Security advisories to check - Supermicro Security Center: https://www.supermicro.com/en/support/security_center - Supermicro BMC advisories (e.g. July 2026 CVE-2026-3821, CVSS 8.8, SMASH arbitrary code execution — fix is a BMC firmware update). - AMD bulletins (e.g. AMD-SB-7054 / CVE-2025-54502, CVSS 7.1, affects EPYC 7002 "Rome" — fix is RomePI 1.0.0.P AGESA via BIOS update). Cross-check the board's CPU generation against the bulletin's affected list. ## SEL (system event log) health `sel info` shows `Percent Used` and `Overflow`. If 100% full with `Overflow: true`, new events are being DROPPED. Clearing is a WRITE op — confirm with the operator before `sel clear`. Supermicro OEM events often decode as `Unknown #0xff` (raw vendor events, not standard IPMI). ## Security notes - Legacy IPMI (RMCP+ cipher 3) is weak; prefer Redfish over HTTPS where possible. - Supermicro publishes BMC security advisories (e.g. July 2026, CVE-2026-3821, up to 8.8 CVSS). Old BMC firmware predates these — flag for update. - `lan print` shows `Bad Password Threshold` (default 3) and lockout interval — repeated bad attempts lock the account. Don't brute-force. ## Pitfalls - **Cipher 17 fails on Supermicro** — always `-C 3` (see above). - **Raw OEM commands** (`raw 0x30 0x90 ...`) for BIOS version return "Request data length invalid" on many Supermicro boards — use Redfish instead. - **BMC up ≠ host up.** The BMC can report "System Power: on" while the host OS is unreachable (no route to host / ARP INCOMPLETE). Verify host reachability separately (SSH port probe) before assuming the box is down. - **Credentials in fact_store, not here.** Look up the BMC IP/user/pass in fact_store before connecting; never hardcode passwords in a skill. - **Supermicro download-center PDFs are bot-blocked (403).** Release-notes PDFs and the firmware download page return 403 to curl/SearXNG/browser. BUT the firmware itself is freely mirrored and NOT blocked — download the actual .bin files from `https://ftp.abacus.cz/support/FW/MB/SUPERMICRO/BIOS//` (or `https://sm.t3x.net/`), then verify SHA256. Thomas-Krenn also mirrors tested-stable bundles at `https://www.thomas-krenn.com/en/download?product=` (their `/redx/tools/mb_download.php/...` links are curl-able). Only the Supermicro.com release-notes PDFs stay behind the wall — use Thomas-Krenn wiki changelogs or search-result snippets for those. ## References - `references/h12ssl-i-firmware-update.md` — H12SSL-i concrete firmware data, security drivers, license-block evidence, known issues, changelog sources.