Hardware doesn't last forever. Servers, their disks and memory, switches, routers, ports, cables and cooling units all fail eventually, at random โ and only while in use. Your job is to limit the damage and repair fast. ๐งฏ
| What fails | Consequence |
|---|---|
| ๐ฅ๏ธ A whole server | It hosts nothing anymore. Its units restart at once on other servers if there is room; otherwise they stay unavailable. Its power LED blinks red. |
| ๐พ A disk or RAM module | The server keeps running but loses that part of its capacity (a server with 4 disks loses a quarter of its storage). Units that no longer fit move elsewhere as soon as there is room. Its status LED turns amber. |
| ๐ A switch, router, port or cable | Traffic is rerouted if another path exists. Otherwise, the units depending on it are disconnected. |
| โ๏ธ A cooling unit | It stops cooling until repaired. See Heat and Cooling. |
A new failure raises an alert, and the Network page counts the ongoing ones. Use search (L) with the Failed filter and Locate all to find them in the building:

Open the device's management page (look at its management port, E). The Failures card lists everything broken on it โ including its ports and cables โ with a Repair button.

| Failure | Cost | Waiting time |
|---|---|---|
| ๐งต Cable | Cheap | Short |
| ๐ Port | Moderate | Short |
| ๐ฅ๏ธ Device | A share of its purchase price | Longer |
| ๐พ Disk / ๐งฎ RAM | One spare part (no technician!) | Short, once the part is in place |
A failed cable can also simply be unplugged and replaced with a new one, without waiting for anyone.
Disks and RAM modules are replaced with spare parts, sold in the Spare parts section of the shop and delivered in small boxes.
No part in stock? The Order a part button of the Failures card orders one โ but then you wait for the delivery while the server limps along.
Keep a few spare parts on the shelves of the storage room. A degraded server stays degraded for as long as the part takes to arrive.