Which system for 2x RTX 6000 blackwell max-q

SRC: r/LocalLLaMA|BY: Annual_Award1260

AI_Summary

User is choosing between two systems for running 2x RTX 6000 Blackwell Max-Q cards for AI workloads:

**Option 1:** Dual Xeon setup with 1TB DDR4 ECC RAM but PCIe 3.0 slots **Option 2:** Single i9-13900K with 128GB DDR5 ECC RAM but PCIe 5.0 slots

They're getting poor CPU offloading performance (1.8 tok/s) with the current 1TB RAM setup and asking whether the massive RAM capacity for offloading is worth it, or if they should prioritize the faster PCIe 5.0 bandwidth for GPU communication.

Original_Description

I am trying to decide which system to run these cards in.

1) Supermicro X10Dri-T, 2x E5-2699v4, 1TB ddr4 ecc ram (16x 64GB lrdimm 2400mhz), PCI-E 3.0 slots

2) Supermicro X13SAE-F, i9-13900k, 128GB ddr5 ecc ram (4x 32GB udimm 4800mhz), PCI-E 5.0 slots

For ssds I have 2x Micron 9300 Pro 15.36TB.

I haven't had much luck with offloading to the cpu/ram on the 1TB ddr4. Probably can tweak it up a little. For the large models running just on cpu I get 1.8 tok/s (still impressive they even run at all).

So question is: Is there any point in trying to offload to ram? or just go for the higher pci 5 speed?

Metadata

Score: 1
Comments: 8