I know many of you have been working on Oracle's Exalogic and other Engineered Systems. With partitions enabled now, things have gone multi dimension. But its fun. Isn't it ?
While you have some EoIB configurations together with InfiniBand partitions, the VNICs are not coming up and staying in WAIT-VHUB state ?
Chances are that you have forgot to add InfiniBand Gateway switches' Bridge-X port GUIDs to your
partition. These must be added as FULL members for EoIB to work properly. VHUB means a virtual hub in EoIB. Bridge-x is the access point for hosts to work over EoIB so thats why it must be a full member in
partition.
Step 1: Find out the port GUIDs of your bridge-x devices in IB Gateway switch.
# showgwports
INTERNAL PORTS:
---------------
Device Port Portname PeerPort PortGUID LID IBState GWState
---------------------------------------------------------------------------
Bridge-0 1 Bridge-0-1 4 0x0010e00c1b60c001 0x0002 Active Up
Bridge-0 2 Bridge-0-2 3 0x0010e00c1b60c002 0x0006 Active Up
Bridge-1 1 Bridge-1-1 2 0x0010e00c1b60c041 0x0026 Active Up
Bridge-1 2 Bridge-1-2 1 0x0010e00c1b60c042 0x002a Active Up
Step 2: Add these port GUIDs to the IB
partition associated with EoIB. Login to master SM switch for this task.
# smpartition start
# smpartition add -pkey <PKey> -port <port GUID> -m full
# smpartition commit
Enjoy !
