<br><br><div class="gmail_quote">On Wed, Oct 17, 2012 at 10:00 AM, Justin Riley <span dir="ltr"><<a href="mailto:jtriley@mit.edu" target="_blank">jtriley@mit.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<br>
Hey Olivier,<br>
<br>
1. I'm very close to releasing a new 12.04 AMI with IPython 0.13 on<br>
it. In the mean time you can use the git version of StarCluster which<br>
has support for 0.13 and create a custom SC AMI with latest IPython.<br></blockquote><div><br></div><div>Justin - we intend to cut 0.13.1 this weekend (rc3 here:Â <a href="http://archive.ipython.org/testing/0.13.1/" target="_blank">http://archive.ipython.org/testing/0.13.1/</a>)</div>
<div><br></div><div>Olivier - I wrote a few tiny plugins a while ago, when I needed to run dev zeromq/pyzmq/ipython on starcluster:</div><div><br></div><div><a href="https://github.com/minrk/starclusterplugins" target="_blank">https://github.com/minrk/starclusterplugins</a></div>
<div><br></div><div>I haven't used them in some time, so they may need a bit of fixing up if starcluster APIs have changed.</div><div><br></div><div>Among them is a generic installer (EggSetup) that would let you use easy_install (or pip with a tiny edit, though I specifically wanted to fetch eggs to avoid recompilation) to install any packages you may need at node startup.</div>
<div><br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
2. It definitely is possible to extend the plugin to do this, however,<br>
this behavior should be off by default and configurable via the<br>
StarCluster config.<br>
<br>
3. Yes I'm leaning towards doing this by default given that it ties up<br>
the entire SGE queue anyway.<br>
<br>
I'm certainly up for a pull request for 2 and 3 if you have time. I'm<br>
super busy at the moment playing catch up between finishing up next<br>
version, building the 12.04 AMIs, and responding to mailing list but I<br>
will eventually get to this. Feel free to file a bug for this as well...<br>
<br>
HTH,<br>
<br>
~Justin<br>
<div><br>
On 10/16/2012 05:55 AM, Olivier Grisel wrote:<br>
> Hi,<br>
><br>
> I have a couple of questions for the ipcluster plugin:<br>
><br>
> 1- Is there an (dev) 64bit AMI with the latest stable release of<br>
> IPython (0.13) instead of 0.12? 2- Would it be possible to extend<br>
> the ipcluster plugin to have the nodes install the dev version of<br>
> ipython instead of the system version on startup? For instance by<br>
> cloning a tag or a specific branch (master) of the IPython github<br>
> repo. 3- Would be possible to make ipcluster not use SGE for<br>
> engine configuration and only use ssh instead, for instance as<br>
> documented in this very short blog post:<br>
> <a href="http://graal.ift.ulaval.ca/jeanfrancisroy/2012/09/27/using-ipython-for-parallel-computing/" target="_blank">http://graal.ift.ulaval.ca/jeanfrancisroy/2012/09/27/using-ipython-for-parallel-computing/</a></div>
</blockquote><div><br></div><div>That blog post, while helpful and short, actually *overstates* the complexity of getting engines up and running, at least in a starcluster environment. Â With the local network already configured, there would actually be no SSH involved. Â No ipcluster config would be necessary, only local launchers would need to be used, and the single command `ipcluster engines [-n NUMBER] [--daemon]` on each node at startup would totally suffice.</div>
<div><br></div><div>-MinRK</div><div><br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br>
><br>
><br>
The goal would be to speedup cluster startup by skipping the SGE<br>
> setup.<br>
><br>
> If you think 2 and 3 are desirable contributions for the project I<br>
> can work on 2 pull requests for those.<br>
><br>
> Best,<br>
><br>
<br>
</div>-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v2.0.19 (GNU/Linux)<br>
Comment: Using GnuPG with Mozilla - <a href="http://www.enigmail.net/" target="_blank">http://www.enigmail.net/</a><br>
<br>
iEYEARECAAYFAlB+5DMACgkQ4llAkMfDcrnT2wCeKo0N4S5aQ2Cwn/p8unnltKLc<br>
1+cAnRrsLIcF6DmP1rKZtyQI05K46aTm<br>
=BAek<br>
-----END PGP SIGNATURE-----<br>
<div><div>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
</div></div></blockquote></div><br>