Hi Igor - <div><br></div><div>You know what, you're absolutely correct. It must be the head-node bandwidth. As we keep working I'll try to update. </div><div><br></div><div>As for compression, well, it's on the wish list :-). </div>
<div><br></div><div><br><div class="gmail_quote">On Tue, Dec 7, 2010 at 10:24 AM, Igor Tatarinov <span dir="ltr"><<a href="mailto:igor@priceyeti.com">igor@priceyeti.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>Boris,</div><div><br></div>What kind of write throughput are you able to get? I would expect the network bandwidth at the NFS server node to be the bottleneck rather than the CPUs on that box. <div><br></div><div>An obvious question: are you compressing the data before you write it to NFS?<div>
<br></div><div>Thanks,</div><div>igor<br><br><div class="gmail_quote"><div><div></div><div class="h5">On Mon, Dec 6, 2010 at 4:36 PM, Boris Fain <span dir="ltr"><<a href="mailto:boris@freecurve.com" target="_blank">boris@freecurve.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div></div><div class="h5">
This may be useful to everyone. When launching a cluster on Amazon, if you have a lot of processes writing to a shared EBS volume <div>NFS does become a bottleneck. Everyone's output is different, so everyone will have to figure out their own limits. In our case we had to </div>
<div>limit the head node compute load to 2/8 - it simply could not handle more. We have 12 8-core (xlarge high-cpu) instances, and there's quite </div><div>a bit of writing from each MPI thread. <br clear="all"><br>-- <br>
<br>b. <br>
</div>
<br></div></div>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br></div></div>
</blockquote></div><br><br clear="all"><br>-- <br><br>b. <br>
</div>