VSAN Cluster Running On Three Intel NUCs – Part 1 (The Build)

Back in November, I started seeing the “virtual” blogs light up with virtual enthusiasts installing VMware ESXi on Mac mini hardware. This increased as Apple released their new Mac mini, even though the cpu core count was reduced. Apple also announced that the new Mac mini hardware (ram and flash storage) would be soldiered to the board giving the user less flexibility. This pushed me to attempt the same idea but on Intel NUC hardware.
For those that are not familiar with the Intel NUC , it’s a small form factor PC that allows for user selected RAM, drives, and even two mini PCIe slots.
Website here :
http://www.intel.com/content/www/us/en/nuc/overview.html
The important part, is the ability to configure the mini PC in any way and continue to change it even after purchase. You can configure the NUC with four, eight, or sixteen gigabytes of RAM, a full 2.5 inch drive (spinner or SSD), and two internal mini PCIe slots. (half length and full length)

The idea for a mini VSAN cluster starts with a few requirements, mostly set by VMware.
– At least three physical PCs/Servers that contain the following:
1 – A SSD or flash device.
2 – A spinning drive.
3 – A drive to install ESXi separate from the above two drives.
4 – A Gigabit network adapter. Two is preferred and 10Gb makes it optimal.
5 – Enough CPU cores and RAM to run the ESXi software plus any Virtual Machines.

If you need extra info on VMware VSAN , click this link here —> http://bit.ly/1Attry9

The challenge:
How can I find three physical devices that use very little power, is stable enough to run a home lab without any unexpected down time, has all of the above requirements in a low cost, easy to obtain package.

Lets start with the hardware:
The Intel NUC i5 series (D54250WYKH) flavor provides two core, four threads @ 1.3Ghz with 2.6Ghz turbo processor. It has two SODIMM slots for up to 16GB RAM but it only accepts 1.35 Volt SODIMMs so make sure you watch what you buy. The 1.5 Volt SODIMMs will prevent the NUC from booting. The D54250WYKH has a taller case when compared to the D54250WYK flavor so you can slide in a 2.5 Drive. I found a few 2TB spinner drives but only the Samsung SpinPoint 2TB drive will fit into the NUC. The Western Digital 2TB 2.5 drive is too thick and you will need to bend metal to make it fit. If you plan to run the system 24/7 then I would suggest a WD Red 1TB or an enterprise drive that is made to spin all day, every day. It must be a SATA drive though so check your selection. The idea of the 4TB San Disk SSD in this spot would be amazing. The NUC has a Full length and half length mini PCIe port. The Full length will allow for a msata SSD. Samsung EVO msata SSDs tested great. Now the difficult part; where can I find a mini PCIe network adapter that will fit into a half length mini PCIe slot. My SSD is hogging the full length slot and SSDs in the half-length form factor are very limited. I even researched the idea of a mini PCIe extension ribbon cable.
This is where the Syba Mini PCI-e Gigabit Ethernet Adapter (SD-MPE24031) comes in. This adapter, with a bit of tweaking, fits perfectly under the msata SSD. I modified the adapter so the cat5 cable is wired directly to the adapter. I lose out on an activity LED but if you like flashy LEDs, you can watch them on your switch. The only thing left is to feed the cable outside of the NUC and your hardware is golden. You will need to throw in a 16GB or 32GB flash drive so you can install your Hypervisor to. I would suggest a good brand and a USB3 version just to make sure any Hypervisor bits do not corrupt and are accessed better than USB v2 speed. A 32GB or larger is suggested to handle a memory dump or provide room for logs etc. I’ve tested with a 8GB thumb and it works fine minus a few annoying messages about it being too small for persistent storage.

Quick hardware summary: (all of this times three for our VSAN cluster)
– Intel NUC D54250WYKH
– 8GB DDR3 SODIMM 1.35volt (two sticks of 8GB for 16GB)
– 2TB Samsung SpinPoint 2.5TB spinner
– 120GB Samsung EVO msata SSD
– Syba half length mini PCIe 1Gb Network Adapter (SD-MPE24031)
– 16GB-32GB flash drive / thumb drive to install the Hypervisor to.
– Mini HDMI to HDMI cable, keyboard, mouse, monitor, tools, brain etc…..

Some Photos of the goods:

Now the software build:
Start by downloading your favorite flavor of VMWare ESXi. I started with 5.5u2p3 build 2143827 because it was the current download at the time. Download ESXi Customizer. I used a copy from http://www.v-front.de/p/esxi-customizer.html .

Next, download the drivers:
— Intel Network Driver :
http://vmnick0.me/net-e1000e-2.3.2.x86_64.zip
–SATA Controller driver :
http://vmnick0.me/sata-xahci-1.10-1.x86_64.zip
Lastly, you need the Realtek driver for your Syba mini PCIe NIC.
http://vmnick0.me/realtekvib.zip
Now just unzip all of the above software, run the esxi Customizer GUI, select your downloaded ISO and the VIB, click RUN. Repeat as needed for each VIB being sure to include the Exported ISO as your new Import ISO so it includes the previous VIB you injected. Once you have your ISO you can burn it to an old fashioned CD or you can play with the many other methods; PXE, USB drive/flash drive….etc.
If you found a better or updated driver, please send me the link and I will update the page.
If you don’t trust any of the above software, you can download all of the software inside a Virtual Machine, create the custom ISO, then export it out of the VM and destroy the VM.

Now you are ready for the install:
Install ESXi to a thumb drive on each NUC, configure the IP Network settings, then load up your management GUI of choice and start playing.

Sneak Peek of LegoEVORack Version 1: Cooling and Top of Rack switch just needs to be installed…..