[Geek] New beast on the way

[azekeil]

It was just over 4 years ago that the first incarnation of beast was born. This time round I'm doing the hardware properly, and specc'd up for Solaris 10u3 so I can use ZFS. 15x 500GB SATA II disks in hot-swappable caddys.. yes please! :) The disks cost me just shy of a grand, the whole machine cost me around £1600.

Delivery on Thursday. Watch this space! :)

Comments

[gerald_duck]

Oooh — where did you get a chassis with that many drive bays so cheaply?

I've promised myself I'm not going to splash out on disc until 1TB drives cost under £100, but it's still useful to plan ahead. (-8

[azekeil.livejournal.com]

Haha. I decided to 'cheat':

1x Cheap tower case with 9x 5.25" bays
3x 3x 5.25" bay to 5 drive SATA caddy

[gerald_duck]

OK… so where are you getting an ATX power supply from with enough oomph to drive that lot? :-p

[thespirit3.livejournal.com]

Hey :)

Do bear in mind that ZFS is prone to going tits up and totally trashing your data. Also, stupid stuff like space used for each ZFS partition isn't reported properly, you just get the total usage for the main ZFS volume (this isn't a show stopper but *is* annoying!).

I'm sticking with Linux and LVM on software RAID for now - although like you went for the hot-swap SATA caddies. Nice :)

And yes - I do still read LJ from time to time ;)

[ev1ldonut.livejournal.com]

15 drives in that space? Good job it'll be in the cupboard then! That'll run nice and toasty I'll imagine, so the cooler air will be needed, and probably several big fans!

Also, YAY STORAGE! *starts trawling torrent sites*

Not really, I'll be frugal. ;)

[azekeil.livejournal.com]

I have a 650W supply that I've hardly used. I'll have to investigate this but this may be something I need to play with. Perhaps I'll use two power supplies - one for the HDDs and one for the motherboard etc?

[azekeil.livejournal.com]

Yeah, although recent mass storage reports seem to indicate that slightly warmer running is actually optimal for drives! (Can't remember which piece I saw that in right now). Each set of 5 caddies has a fan built-in, plus the case has a 250mm fan (which I'll need to set to extract..)

[azekeil.livejournal.com]

I will be testing out the system before I trust it with data :)

Plus, if Solaris 10 ZFS trashed data there'd be a bit of an outcry. I suspect something is not right with whatever trashed it..

If I can't get Solaris working or stable then linux is always the fallback :)

[ev1ldonut.livejournal.com]

Oh yes, I remember that. i think it was El Reg talking about stats released by google on their drive performance...

What other hardware will be in it? A power consumption calculator may be in order, as I imagine this could be a bit power hungry

[azekeil.livejournal.com]

I have a ASUS motherboard with socket AM2, a dual-core 2.0GHz AMD Energy-Efficient CPU and 1GB DDR2 RAM in 2x512MB sticks. The motherboard has onboard VGA and gigabit LAN. Oh, and the two 8-port SATA-II storage controller cards.

I think it will actually be fairly reasonable. Booting is going to be.. interesting. But when it's running / idle, I can set the disks to sleep after a period as well.

[ev1ldonut.livejournal.com]

*nodnod*
Aye, I'm just thinking about the PSU(s) and making sure it (they) is able to cope with the initial powerup requirements, and occasional sustained running of all drives simultaneously.

[dr-atheist.livejournal.com]

AFAIK, it's stable, but not bootable. For mass storage it seems to be the ideal solution. Apple have been looking at it recently too, which might bring the technology foward.

[air-bizkit.livejournal.com]

You have far too much money. Give some to me.

[thespirit3.livejournal.com]

Don't get me wrong - it's really quite impressive, and after a few bug fixes and updates it should be rock solid. However, currently, it will appear stable for a long time before suddenly causing a kernel panic and a total refusal to boot (along with trashed data beyond recovery). I believe it's stung a few large companies.

http://www.google.co.uk/search?q=ZFS+panic (299,000 hits on google)

I don't want to slate it, because it really is the ultimate FS - definately worth playing with - and I hear it's soon to be available in OSX and possibly Linux ... so I'm sure it'll mature soon if it hasn't already :)

[azekeil.livejournal.com]

Erm, is this for Solaris 10, not opensolaris or nevada or nightly builds..? I saw some bugs here relating to Solaris 10, but I haven't bothered to look into it more.

Can you provide me links to people complaining about problems with the 'production ready' code?

[azekeil.livejournal.com]

Not any more...

[azekeil.livejournal.com]

Yeah I saw that. But Sun sell the X4500 storage server based on ZFS.. how can they continue to do that if it's not stable now? This is confusing.

[dr-atheist.livejournal.com]

It's stable, no one in their right mind would comission a server with an unstable filesystem but their system disks wont be ZFS, just the datadisks.
If it were bootable it would be pretty sweet, as it seems to be a pretty damn good system.

[air-bizkit.livejournal.com]

damn....too late!

[azekeil.livejournal.com]

Yeah, booting ZFS I'm not too fussed about. OK, let's see how it goes. I'll do some torture tests before relying on it :)

[thespirit3.livejournal.com]

I believe this was normal Solaris 10 - I know nothing about Open Solaris or Nevada. On my last contract I moved a load of old sun systems to zones on new T2000's - the whole thing worked very well. I've since spoken to the IT manager who mentioned he was aware of ZFS panic issues (but not experienced any). A couple of my current colleagues who come from contracts with the finance groups in the city are also aware of ZFS panic issues - but I've not quizzed them as we're currently working on a HPUX project.

So, I don't have direct experience of this - just repeating what I've heard. I didn't have any crashes during my last contract - and my E450 at home running ZFS also hasn't had any issues, but then - it's generally sat there either switched off or idle :)

Thought it worth mentioning so you can do your own research before trusting important data to it. If you find this issue has been fixed (or was mis-reported in the first place) then I'd be interested to hear from you :D

[azekeil.livejournal.com]

Huh. OK. I'll test putting a significant amount of data on and doing things to the machine that I might want to do and see if I can get it to misbehave before I trust it.

BTW, just pointed some people towards HelloIO for some work :)

[azekeil.livejournal.com]

Also, SATA-II supports staggered spin-up.. nowhere in the stuff for the Supermicro cards does it mention that as well, but as it's SATA-II I'm going to hope it will also support that. This will hopefully circumvent the problem altogether, and probably extend the life of all the components in my system to boot :)