[SGVLUG] Xen

[Matthew Campbell]

Rae is right on.  The biggest problem was matching up the OS, the
Hypervisor, the Mobo and CPUs to all interoperate to support all the
technologies.  John pointed out the main sticking points.  I ended up
going back and forth over nearly a hundred components before I finally
found a set that would work the way I wanted them too.  It was not
mentally hard, just tedious.  The worst part was the emotional
roller-coaster of having to throw out your 'optimal' design again and
again as you found that the part you just added is not compatible with
some other bit.

I ended up, like John, going with a SuperMicro motherboard and Intel CPUs.

SUPERMICRO MBD-X8DAH+-F-O Dual LGA 1366 Intel 5520
Enhanced Extended ATX Dual Intel Xeon 5500 and 5600 Series Server
Motherboard

2 EVGA SuperClocked 01G-P3-1461-KR GeForce GTX 560 (Fermi) 1GB
256-bit GDDR5 PCI Express 2.0 x16 HDCP Ready SLI Support Video

12 Crucial Ballistix sport 4GB 240-Pin DDR3 SDRAM DDR3 1333 Desktop
Memory Model BL51264BA1339

OCZ RevoDrive X2 OCZSSDPX-1RVDX0220 PCI-E 220GB 4 x PCI
Express MLC Internal Solid State Drive (SSD)

10 Seagate Barracuda XT ST33000651AS 3TB 7200 RPM SATA 6.0Gb/s
3.5" Internal Hard Drive -Bare Drive

2 Intel Xeon E5645 Westmere-EP 2.4GHz LGA 1366 80W Six-Core Server
Processor BX80614E5645

Adaptec RAID 6805 2271200-R 6Gb/s SATA/SAS 8 internal
ports w/ 512MB cache memory Controller Card, Kit

2 Noctua NH-U9DX 1366 Dual Heat-pipe SSO Bearing Quiet CPU Cooler



When I got it put together I found that the northbridges were running
at 90c+.  I had decided to make sure the basics were working before
tackling cooling and was glad I did as the requirements changed from
the post-assemble run.

2 SilenX IXN-40C Copper Chipset Cooler

6 GELID Solutions FN-PX08-20 80mm Case Fan with Intelligent PWM
control

2 LOGISYS Computer SF120 120mm Extreme Quiet Rubber Fan


The thing is nearly silent.  Even with all 24 pipelines under full
load the CPU temperatures register as 'LOW'.  It consumes about 1/5
the power that the rack of servers it is replacing did.



Note that the SSD board (not drive) has about three times the
performance of the best drive formfactor SSDs.  On the other hand MoBo
support of the cards turned out to be problematic.  They work fine
under Fedora, but Windoh 7 has caused me tears.  I ended up using the
card on my desktop as a cache card and running W7 off a 7200RPM HD as
W7 would not reliably boot off the 10K RAID drives either.  What a
pathetic product.

Matt


On Sat, Sep 10, 2011 at 7:24 PM, John Kreznar <jek at ininx.com> wrote:
> In a message purporting to be from matti <mathew_2000 at yahoo.com> but
> lacking a digital signature, it is written:
>
>> I am still looking to see which MB/chipset would support Xen, and
>> attempting to find a list of chipsets which work with this.
>
> The most useful and authoritative answers to this that I found when I
> chose hardware for xen last year were [1,2].
>
> Also, USENET newwsgroup gmane.comp.emulators.xen.user is valuable.  I
> archive it all, and that's one place I search when I've got a question.
>
> An important distinction to keep in mind is whether an unmodified
> operating system can be run (full virtualization, requiring hardware
> support), or an operating system tailored for Xen has to be used
> (paravirtualization).  There's nothing wrong with paravirtual, but
> sometimes you want to run a system not having the paravirtual hooks.  I
> ran Linux paravirtual for years on an AMD Athlon.
>
> The second important distinction, assuming you have hardware to run full
> virtual, is whether the hardware supports IOMMU [3,4].  When I chose
> hardware for xen in early 2010, AMD was not yet delivering a product
> with IOMMU.  Intel VT-d technology was the only game in town.  I chose a
> Supermicro 5036T [3], which uses a X8SAX motherboard with an Intel VT-d
> CPU.
>
> With IOMMU, peripheral devices can be selectively "passed through" the
> xen hypervisor to a virtual machine.  IO memory (such as DMA) is then
> mapped by the hardware just like CPU-addressed memory, so the virtual
> machine can efficiently use the device as though it were directly
> connected.  I use this for sound and to pass through a subset of the USB
> ports.
>
> [1] http://wiki.xensource.com/xenwiki/HVM_Compatible_Processors
> [2] http://wiki.xensource.com/xenwiki/HVM_Compatible_Motherboards
> [3] http://en.wikipedia.org/wiki/X86_virtualization
> [4] http://wiki.xensource.com/xenwiki/VTdHowTo
>
> --
>  John E. Kreznar jek at ininx.com 9F1148454619A5F08550 705961A47CC541AFEF13
>
>



-- 
Matthew Campbell
Storage Solution Consultant
Storage Platform Engineering Management
IT | Kaiser Permanente