Christmas homelab update

26 December 2024
homelab,
proxmox,
server,
zfs

Hi blog! It's been a minute. 2024 zoomed by, but it was a great year for me. Landed my dream job, made some progress in the computer science degree, and found some great new friendships. I wish the same good fortune to anyone reading this.

The homelab has been rock solid, chugging away for the past year without any hiccups. Recovered from a week-long power outage after hurricane Helene with no issues. My most-used services over the past few months have been jellyfin, satisfactory game server, firefly finance, and a VM I use for practicing databases. I had fun with a remote gaming VM as well, with GPU passthrough and sunlight/moonlight to my hacked Switch. I'm also still using my guacamole server a decent amount for remoting into everything via web browser. I should have written posts about some of that stuff!

But alas, the itch to upgrade has come again. This time, my goals are simpler: I just want to 'downsize' my 3 nodes into one larger, more efficient, more powerful server. I had enough fun with the little Intel ARC 310 gpu in Proxmox that I want to incorporate a full-size GPU into this build as well, and start dabbling with LLMs.

So here's where I was last year: #

Old cluster

P330 SFF - i7 8700k, 64gb ddr4, 4tb nvme, 12tb hdd, intel arc 310, 10gbe
P320 Tower - i7 7700k, 64gb ddr4, 4tb nvme, 4tb sata ssd, 4x 12tb hdd, 10gbe
Thinkpad x270 - i7 7200u, 16gb ddr4, 512gb nvme, 5gbe

The p320 housed the big ZFS pool containing the fileserver and the datastore for Proxmox backup server. In addition to files and backups, it also ran the satisfactory game server and bittorrent server.

The p330 ran all my VMs and did GPU passthrough stuff. It also synced the fileserver and backup server datasets to that spare 12tb drive.

The x270 ran all the containers -- guacamole, caddy, firefly, tailscale, dyndns, and more.

The upgrade: #

Downsizing all this stuff requires an upsized chassis, yet my shelf is tiny. Enter the Racksource 4U shallow depth chassis. I thought this chassis looked pretty cool, and I was too impatient to wait for the jonsbo n5 to come out. **I will regret not selecting a chassis with hot swap bays later in this post.

I also chose to use lots of desktop parts for this build, as the Black Friday sales were great. Maybe I will swap to ECC ram later, but these parts will work great for this purpose.

Big pile of parts in boxes

I went with a 8-core, 16-thread 5700X and 128GB of ddr4 3200mhz. Grabbed the X570 Taichi for all the nvme slots.

Also grabbed some Noctua and beQuiet stuff to help my ears.

Let's build it: #

Here's what she looks like empty. You get 3x 120mm fans up front and 2x 60mm fans out back, with a support bar in the middle that you can mount about 3 drives to the underside of.

Empty case with fans

I went ahead and ripped all that out and installed the good fans.

Good fans installed

Installing the motherboard, PSU, and heatsink went well. My wife took over and completed some of this part.

My dog also brought us a rock incase we get super mad and need to smash something:

Helper brought a rock

I removed the metal plate covering up the m.2 slots and southbridge stuff. I installed both 4tb nvme ssds from the Thinkstations here for the ZFS mirror that will store the VMs. I also installed a 256gb nvme ssd in the 3rd slot for the boot drive.

Heatsink, board, psu

Installing drives: #

Next, I installed the 4x 12tb spinning disks. **This part sucked.

The drives go in this little cage here.

Drives in cage

Drives in cage 2

**Can you see what's about to become a huge problem?

There is absolutely no room between the drive cage and either the fans in front of it or the PSU connectors behind it.

Here's what it looks like with no fan installed and no PSU cables connected:

No space

No space 2

You'll need a super long screwdriver to reach screws at the bottom of the drive cage:

No space 3

After getting it all installed and working, I had one drive in the ZFS pool showing as dead or dying.

Upon ripping it all back apart, I realized that the sata cable at the bottom of the whole drive stack had been yanked out!

Loose cable

That was a huge pain the butt to reinstall. Next time, I will remember to wait for the jonsbo N5 to come out.

I went ahead and threw my old Nvidia 1070Ti in here for now. At this point, I only had to wait on a $1.29 piece of stamped metal to get here from China -- a full height bracket for my 10 gig SFP nic:

Finished

Never bolt the chassis cover back on without testing it first! Feast your eyes on the server RGB:

RGB

Teach new server about its ancestors #

I started learning all of this server stuff on a bunch of e-waste. I upgraded from pure trash to semi trash, collecting Thinkpads and Thinkstations along the way. I wanted to keep the Thinkspirit with this build, so I slapped a badge on the new server:

Stickers and emblems