Brilliant, but I wonder now that we know how filesystems work if we could redesign ceph to do the right thing. For instance a lot of work was made to schedule the important writes at the right time. Perhaps they could have handled these latency issues explicitly.