1 2 The Linux IPMI Driver 3 --------------------- 4 Corey Minyard 5 <minyard@mvista.com> 6 <minyard@acm.org> 7 8The Intelligent Platform Management Interface, or IPMI, is a 9standard for controlling intelligent devices that monitor a system. 10It provides for dynamic discovery of sensors in the system and the 11ability to monitor the sensors and be informed when the sensor's 12values change or go outside certain boundaries. It also has a 13standardized database for field-replacable units (FRUs) and a watchdog 14timer. 15 16To use this, you need an interface to an IPMI controller in your 17system (called a Baseboard Management Controller, or BMC) and 18management software that can use the IPMI system. 19 20This document describes how to use the IPMI driver for Linux. If you 21are not familiar with IPMI itself, see the web site at 22http://www.intel.com/design/servers/ipmi/index.htm. IPMI is a big 23subject and I can't cover it all here! 24 25Basic Design 26------------ 27 28The Linux IPMI driver is designed to be very modular and flexible, you 29only need to take the pieces you need and you can use it in many 30different ways. Because of that, it's broken into many chunks of 31code. These chunks are: 32 33ipmi_msghandler - This is the central piece of software for the IPMI 34system. It handles all messages, message timing, and responses. The 35IPMI users tie into this, and the IPMI physical interfaces (called 36System Management Interfaces, or SMIs) also tie in here. This 37provides the kernelland interface for IPMI, but does not provide an 38interface for use by application processes. 39 40ipmi_devintf - This provides a userland IOCTL interface for the IPMI 41driver, each open file for this device ties in to the message handler 42as an IPMI user. 43 44ipmi_kcs_drv - A driver for the KCS SMI. Most system have a KCS 45interface for IPMI. 46 47 48Much documentation for the interface is in the include files. The 49IPMI include files are: 50 51ipmi.h - Contains the user interface and IOCTL interface for IPMI. 52 53ipmi_smi.h - Contains the interface for SMI drivers to use. 54 55ipmi_msgdefs.h - General definitions for base IPMI messaging. 56 57 58Addressing 59---------- 60 61The IPMI addressing works much like IP addresses, you have an overlay 62to handle the different address types. The overlay is: 63 64 struct ipmi_addr 65 { 66 int addr_type; 67 short channel; 68 char data[IPMI_MAX_ADDR_SIZE]; 69 }; 70 71The addr_type determines what the address really is. The driver 72currently understands two different types of addresses. 73 74"System Interface" addresses are defined as: 75 76 struct ipmi_system_interface_addr 77 { 78 int addr_type; 79 short channel; 80 }; 81 82and the type is IPMI_SYSTEM_INTERFACE_ADDR_TYPE. This is used for talking 83straight to the BMC on the current card. The channel must be 84IPMI_BMC_CHANNEL. 85 86Messages that are destined to go out on the IPMB bus use the 87IPMI_IPMB_ADDR_TYPE address type. The format is 88 89 struct ipmi_ipmb_addr 90 { 91 int addr_type; 92 short channel; 93 unsigned char slave_addr; 94 unsigned char lun; 95 }; 96 97The "channel" here is generally zero, but some devices support more 98than one channel, it corresponds to the channel as defined in the IPMI 99spec. 100 101 102Messages 103-------- 104 105Messages are defined as: 106 107struct ipmi_msg 108{ 109 unsigned char netfn; 110 unsigned char lun; 111 unsigned char cmd; 112 unsigned char *data; 113 int data_len; 114}; 115 116The driver takes care of adding/stripping the header information. The 117data portion is just the data to be send (do NOT put addressing info 118here) or the response. Note that the completion code of a response is 119the first item in "data", it is not stripped out because that is how 120all the messages are defined in the spec (and thus makes counting the 121offsets a little easier :-). 122 123When using the IOCTL interface from userland, you must provide a block 124of data for "data", fill it, and set data_len to the length of the 125block of data, even when receiving messages. Otherwise the driver 126will have no place to put the message. 127 128Messages coming up from the message handler in kernelland will come in 129as: 130 131 struct ipmi_recv_msg 132 { 133 struct list_head link; 134 135 /* The type of message as defined in the "Receive Types" 136 defines above. */ 137 int recv_type; 138 139 ipmi_user_t *user; 140 struct ipmi_addr addr; 141 long msgid; 142 struct ipmi_msg msg; 143 144 /* Call this when done with the message. It will presumably free 145 the message and do any other necessary cleanup. */ 146 void (*done)(struct ipmi_recv_msg *msg); 147 148 /* Place-holder for the data, don't make any assumptions about 149 the size or existence of this, since it may change. */ 150 unsigned char msg_data[IPMI_MAX_MSG_LENGTH]; 151 }; 152 153You should look at the receive type and handle the message 154appropriately. 155 156 157The Upper Layer Interface (Message Handler) 158------------------------------------------- 159 160The upper layer of the interface provides the users with a consistent 161view of the IPMI interfaces. It allows multiple SMI interfaces to be 162addressed (because some boards actually have multiple BMCs on them) 163and the user should not have to care what type of SMI is below them. 164 165 166Creating the User 167 168To user the message handler, you must first create a user using 169ipmi_create_user. The interface number specifies which SMI you want 170to connect to, and you must supply callback functions to be called 171when data comes in. The callback function can run at interrupt level, 172so be careful using the callbacks. This also allows to you pass in a 173piece of data, the handler_data, that will be passed back to you on 174all calls. 175 176Once you are done, call ipmi_destroy_user() to get rid of the user. 177 178From userland, opening the device automatically creates a user, and 179closing the device automatically destroys the user. 180 181 182Messaging 183 184To send a message from kernel-land, the ipmi_request() call does 185pretty much all message handling. Most of the parameter are 186self-explanatory. However, it takes a "msgid" parameter. This is NOT 187the sequence number of messages. It is simply a long value that is 188passed back when the response for the message is returned. You may 189use it for anything you like. 190 191Responses come back in the function pointed to by the ipmi_recv_hndl 192field of the "handler" that you passed in to ipmi_create_user(). 193Remember again, these may be running at interrupt level. Remember to 194look at the receive type, too. 195 196From userland, you fill out an ipmi_req_t structure and use the 197IPMICTL_SEND_COMMAND ioctl. For incoming stuff, you can use select() 198or poll() to wait for messages to come in. However, you cannot use 199read() to get them, you must call the IPMICTL_RECEIVE_MSG with the 200ipmi_recv_t structure to actually get the message. Remember that you 201must supply a pointer to a block of data in the msg.data field, and 202you must fill in the msg.data_len field with the size of the data. 203This gives the receiver a place to actually put the message. 204 205If the message cannot fit into the data you provide, you will get an 206EMSGSIZE error and the driver will leave the data in the receive 207queue. If you want to get it and have it truncate the message, us 208the IPMICTL_RECEIVE_MSG_TRUNC ioctl. 209 210When you send a command (which is defined by the lowest-order bit of 211the netfn per the IPMI spec) on the IPMB bus, the driver will 212automatically assign the sequence number to the command and save the 213command. If the response is not receive in the IPMI-specified 5 214seconds, it will generate a response automatically saying the command 215timed out. If an unsolicited response comes in (if it was after 5 216seconds, for instance), that response will be ignored. 217 218In kernelland, after you receive a message and are done with it, you 219MUST call ipmi_free_recv_msg() on it, or you will leak messages. Note 220that you should NEVER mess with the "done" field of a message, that is 221required to properly clean up the message. 222 223Note that when sending, there is an ipmi_request_supply_msgs() call 224that lets you supply the smi and receive message. This is useful for 225pieces of code that need to work even if the system is out of buffers 226(the watchdog timer uses this, for instance). You supply your own 227buffer and own free routines. This is not recommended for normal use, 228though, since it is tricky to manage your own buffers. 229 230 231Events and Incoming Commands 232 233The driver takes care of polling for IPMI events and receiving 234commands (commands are messages that are not responses, they are 235commands that other things on the IPMB bus have sent you). To receive 236these, you must register for them, they will not automatically be sent 237to you. 238 239To receive events, you must call ipmi_set_gets_events() and set the 240"val" to non-zero. Any events that have been received by the driver 241since startup will immediately be delivered to the first user that 242registers for events. After that, if multiple users are registered 243for events, they will all receive all events that come in. 244 245For receiving commands, you have to individually register commands you 246want to receive. Call ipmi_register_for_cmd() and supply the netfn 247and command name for each command you want to receive. Only one user 248may be registered for each netfn/cmd, but different users may register 249for different commands. 250 251From userland, equivalent IOCTLs are provided to do these functions. 252 253 254The Lower Layer (SMI) Interface 255------------------------------- 256 257As mentioned before, multiple SMI interfaces may be registered to the 258message handler, each of these is assigned an interface number when 259they register with the message handler. They are generally assigned 260in the order they register, although if an SMI unregisters and then 261another one registers, all bets are off. 262 263The ipmi_smi.h defines the interface for SMIs, see that for more 264details. 265 266 267The KCS Driver 268-------------- 269 270The KCS driver allows up to 4 KCS interfaces to be configured in the 271system. By default, the driver will register one KCS interface at the 272spec-specified I/O port 0xca2 without interrupts. You can change this 273at module load time (for a module) with: 274 275 insmod ipmi_kcs_drv.o kcs_ports=<port1>,<port2>... kcs_addrs=<addr1>,<addr2> 276 kcs_irqs=<irq1>,<irq2>... kcs_trydefaults=[0|1] 277 278The KCS driver supports two types of interfaces, ports (for I/O port 279based KCS interfaces) and memory addresses (for KCS interfaces in 280memory). The driver will support both of them simultaneously, setting 281the port to zero (or just not specifying it) will allow the memory 282address to be used. The port will override the memory address if it 283is specified and non-zero. kcs_trydefaults sets whether the standard 284IPMI interface at 0xca2 and any interfaces specified by ACPE are 285tried. By default, the driver tries it, set this value to zero to 286turn this off. 287 288When compiled into the kernel, the addresses can be specified on the 289kernel command line as: 290 291 ipmi_kcs=<bmc1>:<irq1>,<bmc2>:<irq2>....,[nodefault] 292 293The <bmcx> values is either "p<port>" or "m<addr>" for port or memory 294addresses. So for instance, a KCS interface at port 0xca2 using 295interrupt 9 and a memory interface at address 0xf9827341 with no 296interrupt would be specified "ipmi_kcs=p0xca2:9,m0xf9827341". 297If you specify zero for in irq or don't specify it, the driver will 298run polled unless the software can detect the interrupt to use in the 299ACPI tables. 300 301By default, the driver will attempt to detect a KCS device at the 302spec-specified 0xca2 address and any address specified by ACPI. If 303you want to turn this off, use the "nodefault" option. 304 305If you have high-res timers compiled into the kernel, the driver will 306use them to provide much better performance. Note that if you do not 307have high-res timers enabled in the kernel and you don't have 308interrupts enabled, the driver will run VERY slowly. Don't blame me, 309the KCS interface sucks. 310 311 312Other Pieces 313------------ 314 315Watchdog 316 317A watchdog timer is provided that implements the Linux-standard 318watchdog timer interface. It has three module parameters that can be 319used to control it: 320 321 insmod ipmi_watchdog timeout=<t> pretimeout=<t> action=<action type> 322 preaction=<preaction type> preop=<preop type> 323 324The timeout is the number of seconds to the action, and the pretimeout 325is the amount of seconds before the reset that the pre-timeout panic will 326occur (if pretimeout is zero, then pretimeout will not be enabled). 327 328The action may be "reset", "power_cycle", or "power_off", and 329specifies what to do when the timer times out, and defaults to 330"reset". 331 332The preaction may be "pre_smi" for an indication through the SMI 333interface, "pre_int" for an indication through the SMI with an 334interrupts, and "pre_nmi" for a NMI on a preaction. This is how 335the driver is informed of the pretimeout. 336 337The preop may be set to "preop_none" for no operation on a pretimeout, 338"preop_panic" to set the preoperation to panic, or "preop_give_data" 339to provide data to read from the watchdog device when the pretimeout 340occurs. A "pre_nmi" setting CANNOT be used with "preop_give_data" 341because you can't do data operations from an NMI. 342 343When preop is set to "preop_give_data", one byte comes ready to read 344on the device when the pretimeout occurs. Select and fasync work on 345the device, as well. 346 347When compiled into the kernel, the kernel command line is available 348for configuring the watchdog: 349 350 ipmi_wdog=<timeout>[,<pretimeout>[,<option>[,<options>....]]] 351 352The options are the actions and preaction above (if an option 353controlling the same thing is specified twice, the last is taken). An 354options "start_now" is also there, if included, the watchdog will 355start running immediately when all the drivers are ready, it doesn't 356have to have a user hooked up to start it. 357 358The watchdog will panic and start a 120 second reset timeout if it 359gets a pre-action. During a panic or a reboot, the watchdog will 360start a 120 timer if it is running to make sure the reboot occurs. 361 362Note that if you use the NMI preaction for the watchdog, you MUST 363NOT use nmi watchdog mode 1. If you use the NMI watchdog, you 364must use mode 2. 365