Alternative idea for the adventurous lot - or extreme self-hosters: Buy an old Xeon datacenter server with 1.5tb ddr3 ram (separately). Ddr3 still costs around a dollar pr gig (a month ago), so this rig with two xeon and 24 * 64gb ddr3 ram will cost ~1900$ + transport: ~2k.
(OR an older pc motherboard with 4*64 channels or similar)
DDR3 is a bit slower, use more power, and the server is big/noisy, but the difference in price is unbeatable, and it’s a hell of a KVcache (for ai) if you put a gpu in it.
Note though, that power expenses are going up in the West.
Alternative idea for the adventurous lot - or extreme self-hosters: Buy an old Xeon datacenter server with 1.5tb ddr3 ram (separately). Ddr3 still costs around a dollar pr gig (a month ago), so this rig with two xeon and 24 * 64gb ddr3 ram will cost ~1900$ + transport: ~2k.
(OR an older pc motherboard with 4*64 channels or similar)
DDR3 is a bit slower, use more power, and the server is big/noisy, but the difference in price is unbeatable, and it’s a hell of a KVcache (for ai) if you put a gpu in it.
Note though, that power expenses are going up in the West.