Maybe instead of using expensive SSDs. A topology of many spanning disks in large ZFS clusters by using PCIe HDD controllers. Then link the machines via 10GBe could provide you the speed and performance you require require. I've set up a moderate size pool of 1Pb across 16 physical servers on 4 full size racks. This cost less than 50k. Electricity and cooling come from solar. Its the damn internet connection for people to access it that is the cost killer.
> Its the damn internet connection for people to access it that is the cost killer.
Nope. Any hosting provider will be located at a carrier neutral datacenter or the like. At any of these you will have access to low cost IP transit providers and Internet exchanges. You can buy 100G IP transit for $5k per month, so Internet cost isn't really an issue.
You know, there are some people that just like to do things themselves so that one day instead of buying from the big guy like everyone else, they become the big guy. You can build a distributed datacenter yourself for 500$ a month that can compete with the more expensive network until you grow your user base large enough to afford it. Sometimes VC magic is a bad thing because when you don't have that option, the only other one is to innovate.
Oh yea it's always easier with scale. The main point I was trying to make is that the hardware is actually pretty cheap compared to the network costs to link it. Especially when large companies dump 2 year old servers for pennies on the dollar because the electricity costs at their scale justify it.
Side note: I took that cluster and split it into 8. Then moved them to different geographical locations and where I could use friend or families residential connections and a cloudflare cluster to offset my cost in exchange for unlimited hosting. Very similar to Ceph actually
