As of the last update, I had a nice setup with a couple of Supermicro cases; one used as a SAS JBOD box, and one containing a consumer Intel motherboard. I happened upon a good deal on another Supermicro server, this time in a 2U case with 24 hotswap 2.5″ SAS bays (connected to an expander) dual 1200W PSUs, and an X8DTH motherboard with two Xeon E5645 CPUs, 64GB of ECC RAM, a built-in LSI SAS HBA, and IPMI with remote KVM. I was just shopping around for a motherboard, and for the price the case was just a bonus.
I started out by moving the new motherboard into one of the 3U cases; I wasn’t sure I was going to use the 2U case at all, and the 3U case would let me use full-height cards in the 7(!) PCIe x8 slots in the new board. That setup was really unstable, though, and eventually I came to the conclusion that the PSUs in the 3U cases just weren’t up to the task.
So, I moved the board back to the 2U case and used both 3U cases as SAS JBOD boxes. I had to pick up a replacement GPU to fit the half-height slots of the 2U case, but the new setup is rock-solid. The end result has 54 drive bays, 24 threads, and 96GB of RAM. The 2U case has passive heatsinks on the Xeons, and uses midplane fans with ducting to cool everything. The stock fans were super loud, but replacing them with Noctua fans didn’t provide sufficient cooling. I ended up putting the stock fans back in and fiddling with a script to slow down the stock fans via SMBus. It’s not as quiet as I’d like, but it’s quiet enough. I replaced the home-built rack with a metal-frame one, albeit without sides or a top. Cable management is definitely a lot easier.