[Geek] It lives! My creation lives! <insert manical laugh here>

[azekeil]

I made the appropriate sacrifices to the blood god and the money god, and lo, they were pleased, for my new fileserver, behemoth, went together smoothly and worked on first switch on :)

Thanks to ev1ldonut who unpackaged everything and fitted the 15 drives to their docks while I tackled modding the case to fit the three sets of drive chassis: the bays all had little metal tabs to support individual 5.25" bay devices, so these all had to be pushed back out to make way for the drive chassis.

He even went to the chinese to get food for us as it had got after 9pm and I hadn't eaten.. :)

I now have 6.18TB of usable space in a 15x 500GB disk ZFS RAIDZ pool :)

Here is a picture of the outside of behemoth:



And here's one of the innards:



I did originally order a black case which was in stock when I ordered but as per usual Scan managed to cock up and it transpired there wasn't one when it came to it. Luckily they had the same model but in silver. I don't think it quite has the same impact an all-black case would have done, but I think it's fairly stylish, certainly compared to the string and gaffer tape old version ;)

Here's a list of the specs:

• AMD Athlon64 X2 3800+ (2.0GHz dual core) EnergyEfficient (35W)
• Asus M2NPV-VM with onboard gigabit LAN, 4 port SATA-II controller and VGA (Nvidia 6150)
• 2x Supermicro SAT2-MV8 PCI-X/PCI 8 port SATA-II controllers
• 1GB (2x512MB) CorsairTwinX XMS2 DDR-2 6400 (800MHz)
• 15x Seagate Barracuda 500GB 7200.10 (5 year warrantee) drives
• 1x Hitachi 160GB boot drive for the OS
• 3x Supermicro 5 bay SATA-II hotswappable chassis (each one fits in 3x 5.25" drive bays)
• 700W X-Power PSU with 4 +12V rails each supporting 16A

I also ordered 15 SATA cables. But I needn't have worried - each of the chassis came with 5 SATA cables each, and each of the SATA-II controller cards came with 8 SATA cables each. I now have about 30 spare SATA cables lying around. Anyone need any..? ;)

When I ordered the pieces I had relied on using an existing power supply, but after being persuaded by ev1ldonut I looked at the figures and erred on the side of caution and went out to Maplin and bought the power supply I'm now using.

I don't seem to be able to access the BIOS of the controller cards, but the drives don't spin up until the controller cards are initialised, each in turn, so the power draw is definitely spread out rather than a big hit on switch on.

I had also not really decided what to do about the boot drive, so I ended up going out to PC World and getting the cheapest SATA-II drive I could. I also didn't really have room for it in the case, so it's now screwed to the grills of the two 80mm fan ports on the back of the case. This doesn't really matter as there is a 250mm fan which I got ev1ldonut to reverse so it was extracting. I think there's plenty of ventilation.

I also went around and cable-tied the bugger out of all the cables in there. I'm quite proud of it all now :)

The only bugbear I have is that the controller cards can come unseated fairly easily, leading to them not being recognised. I'll just have to remember to reseat them if I take the server anywhere. Also, the reset switch disappears inside the case if pressed too hard. I'll have to get the glue gun to it.

Wake from LAN works fine, I have Solaris 10u3 installed and a ZFS pool already configured. I've even wired up the drive LEDs to the chassis so you can see the drives blinking as they're accessed.

I need to do some stability testing and figure out why the Solaris kernel updates refuse to install (but the other updates install fine) (update: the resolution looks like it's reply 6 here), but one of the first things I did after setting up the RAIDZ pool was do some performance tests with dd. I get ~35MB/s write and ~95MB/s read. This is pure software and Sun do suggest not having more than about 5 drives in a RAIDZ set for performance reasons, so I'm not entirely surprised at the slow writing. The drives themselves blinked on and off during the writing so I'm sure they have a lot more to give. It's still pretty good for what I want :)

I also measured power and it peaks around 270W; idling it is about 240W. Quite surprising really.

So, LAN anyone? Bring all your stuff. I need to fill up my behemoth ;)

Comments

[ev1ldonut.livejournal.com]

Ah, you got all the LEDs working then? :)

Nice job. I'm impressed at how neat you managed to get it, that mass of cables was... impressive. ;)

[theeighth.livejournal.com]

Awesome, I'll load up my mac mini with stuff and junks and bring it over :D

^_^

[theeighth.livejournal.com]

hmm.... me and my colleagues have been looking at your setup at work this morning. We have a question....

What is the cooling like on those drives at the front? We're afraid you might be decreasing your HD life by not having adequate cooling.

[azekeil.livejournal.com]

Each set of 5 drives has a 90x90x38mm fan drawing air through them. I am well aware of cooling :)

One of the other things I will be doing is monitoring the temperatures reported by the HDDs with hdparm (or Solaris' equivalent).

[theeighth.livejournal.com]

What's the noise like?

[azekeil.livejournal.com]

Well, it sounds kinda like a machine from a datacentre. A bit less volume perhaps.

But then it's going in a cupboard, so I don't really care :)

[gerald_duck]

Mmm… toasty!

You did budget for aircon, right? (-8

[azekeil.livejournal.com]

When I say cupboard I actually mean a coal cupboard. It's about 10ft high, 10ft wide and 4 ft deep. It's also unheated out there and there is adequate ventilation.

[ev1ldonut.livejournal.com]

The bays they're in have a high-rpm high-displacement fan on the back that draws a lot of air over them quickly. They're designed to operate this way and are quie effective. :)

Also, data released by google a while ago showed that temperature made no difference (within reason of course) to the life of their drives, and in fact drives that ran a bit warmer than normal tended to live slightly loinger! (google go through a lot of drives, and they use normal drives like the rest of us apparently)

[gerald_duck]

Are you paying for Solaris, by the way, or does the freebie have ZFS?

(Or is this a tactless question? :-p )

[azekeil.livejournal.com]

Hahaha.

The free version IS the paid-for version. You don't get any updates unless you register, and no support unless you register and pay. I have registered.

[woz.livejournal.com]

lol my feeble 200Gb pales into insignificance!

So, how do you cater for DR on a setup like that?

[azekeil.livejournal.com]

I don't :)

It's not for critical stuff, so I have N+1 redundancy. If I were really worried, I could just buy another at £1600. I'd sync them up with rsync. Then I'd locate it somewhere else with an internet connection, and continue to sync them up periodically with rsync.

[woz.livejournal.com]

You mean you're risking losing that much porn? So reckless! :P

[azekeil.livejournal.com]

Lol :)

I intend to repurpose beast with mirrored drives and use it as a backup device for critical stuff. ;)

[serena-lesley.livejournal.com]

That's just awesome.

[stuartl.livejournal.com]

Bah! You had to one up me! *rasps*

Looks uber cool, though. I plan to dual PSU mine when the drive population increases but even with its existing drive set (4 RAID + 1 boot) it weighs enough that it's not a practical one person lift. And I've got 6 slots remaining...

Mmmmmm.

Must. Not. Spend. Money.

(have no money that's not already allocated...)

[dr-atheist.livejournal.com]

Now which disk did I set to "master" again...

[stuartl.livejournal.com]

There is no Master and Slave any more, there's just Primary and Secondary.

;)

[malcygoff.livejournal.com]

and just to think that you were complaining of a lack of space the other week... Although, I'm suprised by the choice of 500gb drives. What drove the decision not to go for the 1TB drives available these days?

[azekeil.livejournal.com]

500GB drive = ~£63
1TB drive = ~£200

Cheapest £/GB is around the 500GB mark.