[StarCluster] IPython cluster plugin questions and suggested improvements

[Justin Riley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hey Olivier,

1. I'm very close to releasing a new 12.04 AMI with IPython 0.13 on
it. In the mean time you can use the git version of StarCluster which
has support for 0.13 and create a custom SC AMI with latest IPython.

2. It definitely is possible to extend the plugin to do this, however,
this behavior should be off by default and configurable via the
StarCluster config.

3. Yes I'm leaning towards doing this by default given that it ties up
the entire SGE queue anyway.

I'm certainly up for a pull request for 2 and 3 if you have time. I'm
super busy at the moment playing catch up between finishing up next
version, building the 12.04 AMIs, and responding to mailing list but I
will eventually get to this. Feel free to file a bug for this as well...

HTH,

~Justin

On 10/16/2012 05:55 AM, Olivier Grisel wrote:
> Hi,
> 
> I have a couple of questions for the ipcluster plugin:
> 
> 1- Is there an (dev) 64bit AMI with the latest stable release of 
> IPython (0.13) instead of 0.12? 2- Would it be possible to extend
> the ipcluster plugin to have the nodes install the dev version of
> ipython instead of the system version on startup? For instance by
> cloning a tag or a specific branch (master) of the IPython github
> repo. 3- Would be possible to make ipcluster not use SGE for
> engine configuration and only use ssh instead, for instance as
> documented in this very short blog post: 
> http://graal.ift.ulaval.ca/jeanfrancisroy/2012/09/27/using-ipython-for-parallel-computing/
>
> 
The goal would be to speedup cluster startup by skipping the SGE
> setup.
> 
> If you think 2 and 3 are desirable contributions for the project I
> can work on 2 pull requests for those.
> 
> Best,
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlB+5DMACgkQ4llAkMfDcrnT2wCeKo0N4S5aQ2Cwn/p8unnltKLc
1+cAnRrsLIcF6DmP1rKZtyQI05K46aTm
=BAek
-----END PGP SIGNATURE-----