BNX2X(4)

NAME
bnx2x - Broadcom NetXtreme II BCM57710/BCM57711/BCM57711E/BCM57712/BCM57712_MF/ BCM57800/BCM57800_MF/BCM57810/BCM57810_MF/BCM57840/BCM57840_MF 10Gb PCIE Ethernet Network Controllers and Broadcom NetXtreme II BCM57840 10Gb/20Gb PCIE Ethernet Network Controllers.

SYNOPSIS
insmod bnx2x.ko [int_mode=1]
insmod bnx2x.ko [int_mode=2]
insmod bnx2x.ko [disable_tpa=1]
insmod bnx2x.ko [dropless_fc=1]
insmod bnx2x.ko [autogreeen=2]

modprobe bnx2x [int_mode=1]
modprobe bnx2x [int_mode=2]
modprobe bnx2x [disable_tpa=1]
modprobe bnx2x [dropless_fc=1
modprobe bnx2x [autogreeen=2]

DESCRIPTION
bnx2x is the network device driver for the Broadcom NetXtreme II BCM5771x/BCM578xx series PCIE 10/20-Gigabit Ethernet Network Interface Card (NIC). The driver has been tested on 2.6.x kernels starting from 2.6.9.

Refer to the README.TXT from the driver package on how to compile and install the driver.

Refer to various Linux documentations on how to configure network protocol and address.

DRIVER DEPENDENCIES
The driver uses library functions in the crc32 and zlib_inflate libraries. On most kernels, these libraries are already built into the kernel. In some cases, it may be necessary to load these library modules before the driver or unresolved symbol errors will appear. Using modprobe will resolve the dependencies automatically. In rare cases where the crc32 and zlib_inflate libraries are not enabled in the kernel, it will be necessary to compile the kernel again with the libraries enabled. The driver uses also library functions in the crc32c library. On new kernels, this library is already built into the kernel. In some old kernels, it may be necessary to load this library module before the driver or unresolved symbol errors will appear. Using modprobe will resolve the dependencies automatically. On systems where GRO feature is available, driver uses functions from 8021q library. In some kernels this library is already built into the kernel, in others it may be necessary to load this library module before the driver or unresolved symbol errors will appear. Using modprobe will resolve the dependencies automatically.

DRIVER SETTINGS
The bnx2x driver settings can be queried and changed using ethtool. The latest ethtool can be downloaded from http://sourceforge.net/projects/gkernel if it is not already installed. See the ethtool man page for more information. ethtool settings do not persist across reboot or module reload. The ethtool commands can be put in a startup script such as /etc/rc.local to preserve the settings across a reboot. On Red Hat distributions, "ethtool -s" parameters can be specified in the ifcfg-ethx scripts using the ETHTOOL_OPTS keyword. The specified ethtool parameters will be set during ifup. Example: /etc/sysconfig/network-scripts/ifcfg-eth0: ETHTOOL_OPTS="wol g speed 100 duplex half autoneg off" Note that when setting the number of RSS rings via ethtool '-L' option, only

PARAMETERS
Several optional parameters can be supplied as a command line argument to the insmod or modprobe command. These parameters can also be set in modprobe.conf. See the man page for more information.

The optional parameter int_mode is used to force using an interrupt mode other than MSI-X. By default, the driver will try to enable MSI-X if it is supported by the kernel. In case MSI-X is not attainable, the driver will try to enable MSI if it is supported by the kernel. In case MSI is not attainable, the driver will use legacy INTx mode. In some old kernels, it's impossible to use MSI if device has used MSI-X before and impossible to use MSI-X if device has used MSI before, in these cases system reboot in between is required.

Set the int_mode parameter to 1 as shown below to force using the legacy INTx mode on all NetXtreme II NICs in the system.

insmod bnx2x.ko int_mode=1

or

modprobe bnx2x int_mode=1

Set the int_mode parameter to 2 as shown below to force using MSI mode on all NetXtreme II NICs in the system.

insmod bnx2x.ko int_mode=2

or

modprobe bnx2x int_mode=2

The optional parameter disable_tpa can be used to disable the Transparent Packet Aggregation (TPA) feature. By default, the driver will aggregate TCP packets, but if a user would like to disable this advanced feature - it can be done.

Set the disable_tpa parameter to 1 as shown below to disable the TPA feature on all NetXtreme II NICs in the system.

insmod bnx2x.ko disable_tpa=1

or

modprobe bnx2x disable_tpa=1

Use ethtool (if available) to disable TPA (LRO) for a specific NetXtreme II NIC.

The optional parameter dropless_fc can be used to enable a complementary flow control mechanism on 57711, 57711E, 57712 or 578xx. The default flow control mechanism is to send pause frames when the on chip buffer (BRB) is reaching a certain level of occupancy. This is a performance targeted flow control mechanism. On 57711, 57711E, 57712 or 578xx one can enable another flow control mechanism to send pause frames in case where one of the host buffers (when in RSS mode) are exhausted. This is a "zero packet drop" targeted flow control mechanism.

Set the dropless_fc parameter to 1 as shown below to enable the dropless flow control mechanism feature on all 57711 or 57711E NetXtreme II NICs in the system. The parameters will also work on 57712 and 578xx devices with DCBX feature disabled or in case of DCB protocol has negotiated pause flow control with a link partner.

insmod bnx2x.ko dropless_fc=1

or

modprobe bnx2x dropless_fc=1

The optional parameter autogreeen can be used to force specific AutoGrEEEN behavior. By default, the driver will use the nvram settings per port, but if the module parameter is set, it can override the nvram settings to force AutoGrEEEN to either active (1) or inactive (2). The default value of 0 to use the nvram settings.

The optional parameter num_queues can be used to force number of RSS queues and override the default value which is equals to number of CPUs (limited by HW capabilities).

The optional parameter pri_map is used to map the skb-priority to a Class Of Service (CoS) in the HW. This 32 bit parameter is evaluated by the driver as 8 values of 4 bits each. Each nibble sets the desired HW queue number for that priority.

This parameter is only available in kernels which support mapping skb priorities to traffic classes and traffic classes to transmission queues. This means kernel 2.6.39 or newer. On the 5771x family three classes of service are available, but are always served in round robin manner allowing small bulk high priority traffic to be serviced before low priority heavy volume traffic by assigning it a separate COS and thus a separate hardware queue. More advanced COS features such as Strict Priority, Enhanced Transmission Selection and Priority Flow Control are unavailable as the hardware doesn't support them.

On the 57712 family two classes of service are available, with complete Data Center Bridging support (including SP, ETS and PFC).

On the 578xx family three classes of service are available, with complete Data Center Bridging support (including SP, ETS and PFC).

Configuring priorities to unavailable COSs will log an error and default to COS 0.

For example, set the pri_map parameter to 0x22221100 to map priority 0 and 1 to CoS 0, map priority 2 and 3 to CoS 1, and map priority 4 to 7 to CoS 2. Another example, set the pri_map parameter to 0x11110000 to map priority 0 to 3 to CoS 0, and map priority 4 to 7 to CoS 1.

The optional parameter tx_switching makes the L2 transmitter test for each transmitted packet whether packet is intended for the transmitting NIC. This is only relevant in multifunction mode, especially in virtualized environments. For example, if the destination mac address of the transmitted packet belongs to another function in the same port, and other conditions such as vlan are met, the packet is loopbacked instead of being transmitted on the wire. In certain cases the packet is replicated both to the loopback and the wire. An example for this is the receiver being in promiscuous mode. Important: enabling tx-switching has performance penalties, even if no tx-switching is taking place (testing whether the packets need to be loopbacked is the cause for the penalty). By default tx-switching is disabled.

modprobe bnx2x tx_switching=1

There are some more optional parameters that can be supplied as a command line argument to the insmod or modprobe command. These optional parameters are mainly to be used for debug and may be used only by an expert user.

The debug optional parameter poll can be used for timer based polling. Set the poll parameter to the timer polling interval on all NetXtreme II NICs in the system.

The debug optional parameter mrrs can be used to override the MRRS (Maximum Read Request Size) value of the HW. Set the mrrs parameter to the desired value (0..3) for on all NetXtreme II NICs in the system.

The debug optional parameter debug can be used to set the default msglevel on all NetXtreme II NICs in the system. Use ethtool -s to set the msglevel for a specific NetXtreme II NIC.

DEFAULT SETTINGS

Speed :
    According to nvram configuration, but in general Autonegotiation with all speeds advertised.

Flow control :
    According to nvram configuration, but in general Autonegotiation with rx and tx advertised. Note that for adapters/interfaces which do not support flow control autonegotiation, ie SFP+, driver default will be off if the nvram is set to autonegotiation.

MTU :
    1500 (range 46 - 9000)

Rx Ring size :
    4078/(number of RSS queues) (range 128 - 4078)

Tx Ring size :
    4078 (range (MAX_SKB_FRAGS+4) - 4078) MAX_SKB_FRAGS varies on different kernels and different architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.

Coalesce rx usecs :
    25 (range 0 - 3000)

Coalesce tx usecs :
    50 (range 0 - 12288)

MSI-X :
    Enabled (if supported by 2.6 kernel)

TSO :
    Enabled

WoL :
    According to nvram configuration for OOB WoL.

DUAL MEDIA SUPPORT
A dual media capable system connects two PHYs to a single MAC. These PHYs generally use different media types (for example SFP+ fiber and 10GBase-T twisted pair copper) and the dual media configuration requires that the user select a preference among the two PHYs. Supported preferences include manual selection and PHY priority selection. With manual selection, the user specifies that only one PHY should be configured and use to connect to the network. (For example, use the fiber PHY only, always ignore the copper PHY.) With PHY priority selection, the user specifies that either PHY may be used to connect to the network, but when both PHYs indicate link, the PHY with the higher priority will be used to connect to the network. (For example, with fiber PHY priority, if either the copper PHY or the fiber PHY has link, that PHY will be used to connect to the network. However, if both the fiber and copper PHYs have link, the fiber PHY will be used to connect to the network and the copper PHY will be ignored.) When PHY priority selection is used, the PHY which has been selected for network connectivity is referred to as the active PHY. When PHY manual selection is used, there are no special considerations when running ethtool since only one media type is used by the MAC and ethtool is able to control that media type as expected. However, since ethtool is currently not designed to manage the multiple physical interfaces enabled by Dual Media support, the following limitations will apply when ethtool is used on a system with PHY priority selection enabled: 1. Ethtool can be used to display the current physical media information for the active PHY. 2. Ethtool cannot be used to determine whether PHY manual selection or PHY priority selection is in use. This configuration information is available through system specific utilities provided by the vendor. 3. Ethtool can be used to control the current physical media configuration, but this will force the configuration back to PHY manual selection. 4. When ethtool is used to configure the active PHY, ethtool must be called twice, first to change AWAY from the active PHY, then to change BACK to the active PHY. (For example, if the active PHY is copper, ethtool must be first called to change the active PHY to fiber, forcing PHY manual selection to be enabled, then ethtool must be called again to change the active PHY to copper.) 5. Using ethtool to change from PHY priority selection to PHY manual selection only applies to the current session. When the driver is unloaded/reloaded or the system is rebooted, PHY selection will return to the default value. PHY selection defaults must be set outside of Linux with system specific utilities provided by the vendor.

AUTHOR
Eliezer Tamir - eliezert@broadcom.com

SEE ALSO
ifconfig(8), insmod(8), modprobe.conf(5), ethtool(8).