NSX, Jumbo Frames & Intel NUC Weirdness

I built a lab with Intel 6th generation NUC’s with the intent of creating a VMware NSX, vRealize Automation and vSAN learning lab. I ran into a bit of trouble when I was attempting to enable Jumbo Frames on the onboard I219-V ethernet adapter.

I tried setting the MTU using both the gui and esxcli and each time received the following error:

esxcli network vswitch standard set -m 9000 -v vSwitch0
Unable to set MTU to 9000 the following uplinks refused the MTU setting: vmnic0
I searched around to try to find others reporting problems with jumbo frames and the NUC and came a across a few mentions on Virtually Ghetto in the comments section and possible workarounds using USB 3.0 to ethernet adapters (Virtually Ghetto & DEV TTY).

So I ordered some Star Tech USB 3.0 Ethernet Adapters, but while I was waiting for them I decided to tinker around with NSX. I deployed the NSX appliance and performed host preparation–to my surprise VXLAN install was successful

NSX Install

My initial thought is that the VTEPs were probably created but not completely functional since I was unable to change the MTU (MTU 1600 is minimum recommended setting). I’m a rank NSX newbie but it would appear part of that host prep changes the MTU for the switch and VTEP vmk to 1600.

vmk

Ok, that is pretty odd–the gui and esxcli both failed setting MTU to 1600 but NSX did the job. Just for grins I went back to the gui and esxcli and tried to set the MTU of the other vmk’s (Management & VSAN)…….and it works. I can set the MTU to 9000 without error!

This all had me wondering if this was actually functional so I configured my mac with jumbo frames (MTU 9000) using a thunderbolt ethernet adapter. I first pinged my cisco switch with 8972 byte frame—that worked. Ok now I can test my hosts. All my hosts checked out, however, I can’t seem to get anything over 1572 bytes to work (MTU set to 1600). Not ideal but good enough to run NSX.

After ping testing, I’ve since built out some logical switches, a distributed logical router and an edge gateway—it all works as expected and I was able to reach the physical network from my logical switches and ping across VMs in different clusters with the same logical switch and transport zone.

Long story short—you can get the onboard I219-V NIC to functional well enough to run NSX. Now its time for some NSX fun!—including stranding my Virtual Center and figuring out how to remove my poorly conceived firewall rules that blocked it! Doh!

Special thanks to William Lam (http://www.virtuallyghetto.com/) and Jose Gomez (http://www.devtty.uk/). I’ve followed Williams blog for a long time now and especially liked his mac mini content. Like him, I am now an Intel NUC convert. I need to post something up about my lab soon.

 

 

 

4 thoughts on “NSX, Jumbo Frames & Intel NUC Weirdness

      1. Jose Gomes

        Sorry Andy,

        For some reason I didn’t get a notification regarding your comment.

        Anyway, I was referring to a comment here placed on one of William’s post. Here even pasted the commands used to set the MTU…

        Jose

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s