One of
the important new features of
the AK8 code is
the ability to use multiple IP addresses on
the same physical network port. This feature is called VNICs, or Virtual NICs. This allows us to no longer "burn" a whole port in a cluster when one cluster peer owns a network port. Traditionally, we have had to leave Net0 empty on controller 2, because it was used for managing controller 1. Vise-versa for Net1 on Controller 1. Then, if you have data going over 10GigE ports, you probably only had half of your ports running at any given time, and
the partner 10GigE port on
the other controller just sat there, doing nothing, unless
the first controller went down.
What a waste. Those days are over.
I want to thank and give a big shout-out to our good partner, OnX Enterprise Solutions, for allowing me to come into their lab and play around with their 7320 to do this demo. They let me make a big mess of their lab for
the day as I played around with VNICs. If you're looking for a partner who knows Oracle well and can also piece together a solution from multiple vendors to get you what you need, OnX is a good choice. If you would like to talk to your local OnX rep, you can contact Scott Gill at
[email protected] and he can point you in
the right direction for your area.
Here we go:
Here is what your Datalinks window looks like BEFORE you upgrade to AK8.
Here's what
the same screen looks like after you upgrade. See
the new box?
So here is my current network setup. I have my 4 physical interfaces setup each with an IP address. If I ping them, no problems.
So I can ping 180, 181, 251, and 252. However, if I try to ping 240, it does not work, as
the 240 address is not being used by any of these interfaces, right?Let's change that.
Here, I'm going to make a new Datalink by clicking
the Datalink "Plus sign" button. I will check
the VNIC box and tell it to use igb2, even though another interface is already using it.
Now, I will create a new Interface, and choose "v_dl2" for it's datalink.
My new network screen looks like this. A few things to take note of here. First, when I click
the "igb2" device, it only highlights dl2 and int2. It does not highlight v_dl2 or v_int2.I think it should, but OK, it looks like VNICs don't highlight when you click
the device. Second, note how
the underscore character in v_dl2 and v_int2 do not seem to show on this screen. You can see it plainly if you go in and edit them, but from here it looks like a space instead of an underscore. Just a cosmetic
bug, but something to be aware of.
Now, if I click
the VNIC datalink "v_dl2", on
the other hand, it DOES highlight
the device it belongs to, as it should. Seen here: Note that it did not, however, highlight int2 with it, even though int2 is connected to igb2. That's because we clicked v_dl2, which int2 has nothing to do with. So I'm OK with that.
So let's try pinging 240 now. Of course, it works great.
So I now make another VNIC, and call it v_dl3 using igb3, and v_int3 with an address of 241. I then setup three shares, using ports 251, 240, and 241.Remember that IP 251 and 240 both are using
the same physical port of igb2, and IP 241 is using port igb3.
Next, I copy a folder full of stuff over to all three shares at
the same time. I have analytics going so I can see
the traffic. My top chart is showing
the logical interfaces, and
the bottom chart is showing
the physical ports.Sure enough, look at
the igb2 and vnic1 interfaces. They equal
the traffic going over
the igb2 physical port on
the second chart. VNIC2, on
the other hand, gets igb3 all to itself.
This would work
the same way with 10Gig or Infiniband ports. You can now have multiple IP addresses and even completely different subnets sharing
the same physical ports. You may need to make route table entries for that. This allows us to use all of
the ports you paid for with no more waste.
Very, very cool.
One small "bug" I found when doing this. It's really not a
bug, it was designed to do this when VNICs were not around. But now that we have NVIC capability, they should probably change this. I've alerted
the engineering team about this and they're looking into it, so perhaps it will be fixed in a later code.
Here it is. Remember when we made
the new VNIC datalink, I specifically said to click on
the "Plus Sign" button to create it? I don't always do that. I really like to use
the drag-and-drop method to create my datalinks in
the network screen.HOWEVER, if you were to do that for building a VNIC, it will mess you up a little. Watch this.
Here, I'm dragging igb3 over to make a new datalink. igb3 is already being used by dl3, but I'm going to make this a VNIC, so who cares, right?
Well,
the ZFSSA does not KNOW you are going to make it a VNIC, now does it? So... it works as designed and REMOVES
the igb3 device from
the current dl3 datalink in
the background. See how it's now missing? At
the same time,
the dl3 datalink choice is missing from my list of possible VNICs for me to choose from!!!! Hey!!! I wanted to pick dl3. Why isn't it on
the list??? Well, it can't be on this list because dl3 no longer has a device associated with it. Bummer for you.
When you click cancel,
the device is still missing from dl3.
The fix is easy. Just edit dl3 by clicking
the pencil button, do absolutely nothing, and click "Apply".
The device will magically come back.
Now, make
the VNIC datalink by clicking
the "Plus Sign" button. Sure enough, once you check
the VNIC box, dl3 is a valid choice. No problem.
That's it for now. Have fun with VNICs.
