[StarCluster] note for usage

[Boris Fain]

Hi Igor -

You know what, you're absolutely correct. It must be the head-node
bandwidth. As we keep working I'll try to update.

As for compression, well, it's on the wish list :-).


On Tue, Dec 7, 2010 at 10:24 AM, Igor Tatarinov <igor at priceyeti.com> wrote:

> Boris,
>
> What kind of write throughput are you able to get? I would expect the
> network bandwidth at the NFS server node to be the bottleneck rather than
> the CPUs on that box.
>
> An obvious question: are you compressing the data before you write it to
> NFS?
>
> Thanks,
> igor
>
> On Mon, Dec 6, 2010 at 4:36 PM, Boris Fain <boris at freecurve.com> wrote:
>
>> This may be useful to everyone. When launching a cluster on Amazon, if you
>> have a lot of processes writing to a shared EBS volume
>> NFS does become a bottleneck. Everyone's output is different, so everyone
>> will have to figure out their own limits. In our case we had to
>> limit the head node compute load to 2/8 - it simply could not handle more.
>> We have 12 8-core (xlarge high-cpu) instances, and there's quite
>> a bit of writing from each MPI thread.
>>
>> --
>>
>> b.
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster at mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
>>
>


-- 

b.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20101207/1c094dc4/attachment.htm