We’ve had good luck with the Haswell-E Xeon 1650 v3 with the ASUS X99-WS motherboard. We get a full 16 lanes of PCIe to four slots and it is very fast. Plus it has ECC for all the memory we have added. These have been great for machine learning, for computer simulation of cars and for compiling custom Linux builds.
But now with the big processors coming out it’s time to see what the alternatives are:
This is the logical successor, the new HEDT (high end desktop) line seems tailor made for machine learning and training. But we need something with all those lanes as we either put in four GPUs or lots of fast SSD disks on PCI Express. PC Parts Picker has a good list of available motherboards.
The other option is to look at the new Ryzen line or even their Epyx. This has an incredible number of cores and seems to work very well for highly threaded things and paired with the very fast nVidia GTX 1080 could really be contender particularly given the lanes.