<br><br><div class="gmail_quote">On Wed, Oct 17, 2012 at 10:00 AM, Justin Riley <span dir="ltr"><<a href="mailto:jtriley@mit.edu" target="_blank">jtriley@mit.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<br>
Hey Olivier,<br>
<br>
1. I'm very close to releasing a new 12.04 AMI with IPython 0.13 on<br>
it. In the mean time you can use the git version of StarCluster which<br>
has support for 0.13 and create a custom SC AMI with latest IPython.<br></blockquote><div><br></div><div>Justin - we intend to cut 0.13.1 this weekend (rc3 here: <a href="http://archive.ipython.org/testing/0.13.1/" target="_blank">http://archive.ipython.org/testing/0.13.1/</a>)</div>
<div><br></div><div>Olivier - I wrote a few tiny plugins a while ago, when I needed to run dev zeromq/pyzmq/ipython on starcluster:</div><div><br></div><div><a href="https://github.com/minrk/starclusterplugins" target="_blank">https://github.com/minrk/starclusterplugins</a></div>
<div><br></div><div>I haven't used them in some time, so they may need a bit of fixing up if starcluster APIs have changed.</div><div><br></div><div>Among them is a generic installer (EggSetup) that would let you use easy_install (or pip with a tiny edit, though I specifically wanted to fetch eggs to avoid recompilation) to install any packages you may need at node startup.</div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
2. It definitely is possible to extend the plugin to do this, however,<br>
this behavior should be off by default and configurable via the<br>
StarCluster config.<br>
<br>
3. Yes I'm leaning towards doing this by default given that it ties up<br>
the entire SGE queue anyway.<br>
<br>
I'm certainly up for a pull request for 2 and 3 if you have time. I'm<br>
super busy at the moment playing catch up between finishing up next<br>
version, building the 12.04 AMIs, and responding to mailing list but I<br>
will eventually get to this. Feel free to file a bug for this as well...<br>
<br>
HTH,<br>
<br>
~Justin<br>
<div><br>
On 10/16/2012 05:55 AM, Olivier Grisel wrote:<br>
> Hi,<br>
><br>
> I have a couple of questions for the ipcluster plugin:<br>
><br>
> 1- Is there an (dev) 64bit AMI with the latest stable release of<br>
> IPython (0.13) instead of 0.12? 2- Would it be possible to extend<br>
> the ipcluster plugin to have the nodes install the dev version of<br>
> ipython instead of the system version on startup? For instance by<br>
> cloning a tag or a specific branch (master) of the IPython github<br>
> repo. 3- Would be possible to make ipcluster not use SGE for<br>
> engine configuration and only use ssh instead, for instance as<br>
> documented in this very short blog post:<br>
> <a href="http://graal.ift.ulaval.ca/jeanfrancisroy/2012/09/27/using-ipython-for-parallel-computing/" target="_blank">http://graal.ift.ulaval.ca/jeanfrancisroy/2012/09/27/using-ipython-for-parallel-computing/</a></div>
</blockquote><div><br></div><div>That blog post, while helpful and short, actually *overstates* the complexity of getting engines up and running, at least in a starcluster environment. With the local network already configured, there would actually be no SSH involved. No ipcluster config would be necessary, only local launchers would need to be used, and the single command `ipcluster engines [-n NUMBER] [--daemon]` on each node at startup would totally suffice.</div>
<div><br></div><div>-MinRK</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br>
><br>
><br>
The goal would be to speedup cluster startup by skipping the SGE<br>
> setup.<br>
><br>
> If you think 2 and 3 are desirable contributions for the project I<br>
> can work on 2 pull requests for those.<br>
><br>
> Best,<br>
><br>
<br>
</div>-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v2.0.19 (GNU/Linux)<br>
Comment: Using GnuPG with Mozilla - <a href="http://www.enigmail.net/" target="_blank">http://www.enigmail.net/</a><br>
<br>
iEYEARECAAYFAlB+5DMACgkQ4llAkMfDcrnT2wCeKo0N4S5aQ2Cwn/p8unnltKLc<br>
1+cAnRrsLIcF6DmP1rKZtyQI05K46aTm<br>
=BAek<br>
-----END PGP SIGNATURE-----<br>
<div><div>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
</div></div></blockquote></div><br>