I'm confused by this comment, especially since you seem to work with super computers. One of the biggest challenges in ECP (exascale computing project) is disk IO. So much so that they were inventing complicated heterogeneous architectures. In fact many teams are just skipping disk all together and using in situ methods, only saving results. I would think that would be needed here.

But the question is what heavy processing computer doesn't have IO issues? Also, Blue Waters isn't really a GPU super computer like Summit and Sierra are (or the up coming Aurora and Frontier). It has 4228 nodes (out of ~27000 nodes) that have GPUs on them, and they only have one, and they are Keplers. Those aren't great GPUs and aren't going to do very well in parallel either. There's a big bottleneck in GPU IO. I think this program will not be utilizing GPUs very heavily. Worse, they don't have many CPUs per node. It's 8-16 cores and 32-64 GB memory per node. There's going to be a lot of time spent in communication.

I'll admit that BW doesn't seem like the best computer to the job, but you use what you got. I'll buy the argument that this is the wrong computer for the specific job, but what would you use besides a super computer? (I think Summit would be a good computer for this job)