How to configure jumbo frames in Bright Cluster Manager using CMSH

    

This article describes how to configure jumbo frames in Bright Cluster Manager using the Cluster Management Shell (CMSH). Jumbo frames are enabled by changing the MTU (maximum Transmission Unit) on all the relevant switches and network interfaces. Bright makes it easy.

 

Let's get started.


First we'll change the MTU for the storagenet network inside of Bright Cluster Manager using the CMSH.

# cmsh
[atom]% network use storagenet
[atom->network[storagenet]]% show
Parameter                      Value
------------------------------ ------------------
Base address                   172.60.0.0
Domain Name                    storage
Dynamic range end              0.0.0.0
Dynamic range start            0.0.0.0
External network               no
Gateway                        0.0.0.0
IPv6                           no
MTU                            1500
Management allowed             no
Netmask bits                   16
Node booting                   no
Notes                          <0 bytes>
name                           storagenet

[atom->network[storagenet]]% set mtu 9216
[atom->network*[storagenet*]]% commit

[atom->network[storagenet]]% show
Parameter                      Value
------------------------------ ------------------
Base address                   172.60.0.0
Domain Name                    storage
Dynamic range end              0.0.0.0
Dynamic range start            0.0.0.0
External network               no
Gateway                        0.0.0.0
IPv6                           no
MTU                            9216
Management allowed             no
Netmask bits                   16
Node booting                   no
Notes                          <0 bytes>
name                           storagenet

Next we'll configure the relevant network interface on the machines that we don't want to reboot yet. In this case that's the head node itself.

# ifconfig eth4
eth4      Link encap:Ethernet  HWaddr 00:10:18:88:CD:40
inet addr:172.60.255.254  Bcast:172.60.255.255  Mask:255.255.0.0
inet6 addr: fe80::210:18ff:fe88:cd40/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:12447861 errors:0 dropped:0 overruns:0 frame:0
TX packets:9053229 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11381455036 (10.5 GiB)  TX bytes:12890467675 (12.0 GiB)
Interrupt:203 Memory:c7800000-c7ffffff

# ifconfig eth4 mtu 9216

# ifconfig eth4
eth4      Link encap:Ethernet  HWaddr 00:10:18:88:CD:40
inet addr:172.60.255.254  Bcast:172.60.255.255  Mask:255.255.0.0
inet6 addr: fe80::210:18ff:fe88:cd40/64 Scope:Link
UP BROADCAST MULTICAST  MTU:9216  Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
Interrupt:203 Memory:c7800000-c7ffffff

Finally, we'll set the MTU in the swicth that this network is connected to. In this example, its a Dell m8074 10gbe ethernet switch.

console>enable
console#configure
console(config)#interface range ethernet all
console(config-if)#mtu 9216

Here's how to test the end-to-end configuration from the head node.

We're done.

High Performance Computing eBook