Notes on building an AMD Threadripper machine and Seagate EXOS X16 incompatibility
Well the build of a monster machine is always interesting and being at the bleeding edge is a sheer joy (agony!). So if you are crazy enough to want to build a 32-core, 256-GB memory with 2TB SSD and 16TB hard disk (https://pcpartpicker.com/user/richtong/saved/#view=gH4K4D)
As always you can either work for a gigantic company where they just give you a monster machine in the cloud and go old school to build your own boss hardware. (To go next to your Ford GT40 🙂
Building up the Hardware
Ordering the parts isn’t easy and we haven’t gotten the second nVidia card installed, but here are notes if you are building ones of these things:
- On buying the hardware, I normally use Newegg for most purchases, just make sure that particularly for the hard drive, it comes direct from Newegg. These hardware folks need to properly pack things. Also PCPartpicker does a great job of making things easy to search for.
- When you assemble the machine, the AMD processor is actually super easy to install. You never actually touch the processor, there is an orange carrier, so you clip open the motherboard mount and literally slide the processor in. What a smart way to do it!
- I made a mistake ordering the NH-U14S. Be aware, that most of the times, when you pick it that you won’t get the right version. The specific version is called the NH-U14S TR-SP3 and last part is important . That is the custom version for the Ryder/Threadripper/Epyc line that uses the sTRX4 pin layout.
- Most the assembly is pretty easy, but you can move the fan by 3-6 mm, this didn’t turn out to be necessary since the MSI RTX 2080Ti Trio just fits. But you need to do this before you mount it.
- The pattern of the thermal grease is important, it is basically a 3 x 3 and then dots in between. Don’t use too much paste. Like half a BB worth. Maybe 1.5mm wide drop. The cooler comes with it’s own thermal grease.
- As an aside, although water cooling is seemly mandatory these days, this cooler is about as efficient and super quiet. It comes with a single 140mm fan, but if you want to get a little more cooling get another whisper quite NF-A15 so you can operate in push-pull mode.
- When you install the memory, you will need to take the cooler off as the memory is pretty tall. Having 32GB per stick means it is quite high, but the fan still fits fine inside the XL2 chassis.
- This is a good time to install the SSD. You need to remove the cooler and cover that’s between the PCI slots. Make sure that you take off the plastic so that the thermal sticky stuff is against the SSD. As an aside, this thing is so buried, if you are going to put two sticks down there, now is a good time.
- When you install the motherboard, the instructions don’t say it, but you want to find the brass standoff and put 9 of them down. You don’t want the motherboard against the bare metal!
- The screw at the edge of the mother board is nearly impossible to reach. You need to unscrew the top of the cooler that’s on top of the VRM and then you can screw that one in.
- The power supply is big, so you will need to remove the case fan at the borrow and reposition it at the top. You just need to remove one of the blocks at the top and it mounts up there easy.
- When you install the rest, it is easiest to take the front and back panels off and stand the machine upright. That’s because to make it neat, you should route the power and other cables to the backside of the motherboard
- The motherboard 24-pin is the tightest fit, so put this one in first. The route the two 8-pin power in and to the upper part of the motherboard.
- Install all the other wires before putting the graphics card in. The tough ones are for all the cooling fans. Note that there are four fans. There is the CPU cooler, the case fan in the front, then the back and finally the top. Good news is that there are fan headers all along the edge of the board, so it will be quite neat.
- Now you can look in the back of the case and connect the USB 3 header, the USB 2 header and then the LED light, power and reset from the front panel to the teeny tiny jumpers on the edge of the board.
- OK, you are nearly done, now tilt the case onto it’s side and install the graphics cards (they are so heavy it’s easy to insert them with the case on it’s side). Make sure the card release is clicked down before you start!
- We only did one graphics card (the other one is coming), but you take off the slot covers in the rear and put the double width one in. Then connect the auxiliary power. Note that the power supply doesn’t have enough independent wires, so of the three connected, one needs to use the 6-pin and the 8-pin fro one wire. That’s because with two GPUs, you need four total power connections.
- Tilt the case back up and finally put your hard drive into the carrier and then route the SATA and the power cable behind. There should be enough slack that you can actually pull out the carrier and the wires stay connected.
- Now remove the power things you don’t need. There should be too many of them, but if it’s a pain to disconnect, so easier to leave extras in but not as neat.
- Find an Ethernet connection because when your start the WiFi drivers don’t seem to work. You should also add the WiFi antenna at the back of it.
- Plug your keyboard and mouse into the USB Ports at the back.
- Connect with HDMI. Note that there are things like DisplayPort to HDMI cables. Don’t use these, they often times don’t translate properly. To connect to a 4K display you need an HDMI 1.2 cable that is 4K capable.
Running the software
Now it’s time to power it up. It’s software time:
- Plug it in. We also have a fire extinguisher just in case, so beware of smoke etc. (you can’t be too careful!), but at this stage even at no power, you should see the ASUS display screen at the back (this is cool btw) start and show you status. The various LED lights on the card and the memory will light up. These are good signs!
- Turn it on. This verifies the power button works by the way 🙂
- Once it boots, you should be in the BIOS. This is good. Wander around and you should see on the boot screen the processor name and the available memory.
- Buy a copy of Windows Pro. You need Pro and not Home because Home “only” support 128GB of memory. Download it onto a USB as we don’t have any optical drives in this build.
- The installation is pretty automatic and you should see something on the display. Hooray!
- But now it is update time, take all the Windows updates just to make sure the base system works.
- The ASUS software needs to get downloaded from the site. Note that for this motherboard, the core thing you need is called AI Suite, it is buried in the `Show All` under utilities. The most important thing it installs is EZ Update which you need to get all the ASUS driver updates. Note that this seems really buggy, seems to be missing various files when you reboot.
- Run EZ Update to get the latest.
- Go to nVidia and download GEForce Experience, this is the software that keeps the video drivers up to date. You should now be a full resolution of your monitor. If not, it’s either the video driver or in our case using the wrong cable (see above!)
- Download Prime95 and make sure it runs. You should start AISuite so you can monitor the processor temperatures. At least on our build. On this machine running at full bore, the CPU temperature reached 70C which is pretty remarkably low given how intense that benchmark is. It’s also something to see 64 windows open up for each thread. The Threadripper is 32 cores with 2 threads per core, so 64 windows open up.
- Once this is all running, you can go back to BIOS and try the overclock settings. The two key ones are the memory where you want to set it to DOCP and then auto for overclock. We got the base 3.7GHz Threadripper 3 to 4.2GHz across all the cores. That’s pretty amazing I would say for just air cooling and no other heroics.
The one incompatibility Seagate EXOS X16
The only incompatibility we were not able to solve with the backing hard disk. The new X16 is a 16TB drive (ok really it’s 15TB when using real 1024 is a KB rather than the funky measurement of 1000 is a “disk KB”
For some reason, while Windows Pro should be able to support drives up to 2 Ettabytes, it wouldn’t work. The Disk Manager has an “I/O error” when you try to initialize the drive to a GPT drive. Note that the older MBR only allows 2TB drives (for 512 byte per sector driver).
We did download the third party Minitools Partition Tool and it was able to create a GPT partition, but failed when trying to format.
Windows has a dizzying array of partition types, but the short answer is that we tried Simple and it didn’t work.
It worked fine with a 6TB drive we had around, but didn’t work with either 16TB Drives. Even downloading the Seagate diagnostic application showed that both drives worked, so it looks like software. This is a monster drive after all.