[StarCluster] IPython cluster plugin questions and suggested improvements
Olivier Grisel
olivier.grisel at ensta.org
Thu Oct 18 04:34:17 EDT 2012
2012/10/17 MinRK <benjaminrk at gmail.com>:
>
>
> Sure, if starcluster didn't set up NFS already. You can certainly use
> sshfs, or (much lighter) scp, of course. I'm not sure why you would in the
> context of StarCluster, though, which already has NFS all set up.
I have not checked which mount point is on NFS on the default setup of
starcluster. Is the home folder of each node mapped to the home folder
of the master by default?
Ideally I would only use NFS for small sharing configuration files and
maybe a development workspace with editable source code while having
each node use a the local disk for reading / writing large datafiles
that are dispatched from / saved to S3 buckets at the beginning / end
of the processing.
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
More information about the StarCluster
mailing list