<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Dan,<br>
<br>
The next version uses a ClusterManager object rather than the
cluster.* methods. The new ClusterManager is automatically loaded in
the shell as variable 'cm'. So with the latest github code it would
be:<br>
<br>
$ starcluster shell<br>
$ cm.run_plugin("shortreadplugin", "myfirstcluster")<br>
<br>
You can pretty much replace any cluster.* method call with cm.* if
you're using the latest github code.<br>
<br>
~Justin<br>
<br>
On 10/30/10 2:57 PM, Dan Tenenbaum wrote:
<blockquote
cite="mid:AANLkTimtbiE3PN86uN+twJpc5JSOBchTS8xaHBsZmZK2@mail.gmail.com"
type="cite">
<div>Looks like there are a number of files in this fork that
don't exist in the 0.91 that I downloaded.</div>
<div><br>
</div>
So just downloaded your whole fork and built that, then got this
in the shell:
<div><br>
</div>
<div>
<div>In [1]:
cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)</div>
<div>---------------------------------------------------------------------------</div>
<div>AttributeError Traceback (most
recent call last)</div>
<div><br>
</div>
<div>/Library/Python/2.6/site-packages/StarCluster-0.9999-py2.6.egg/starcluster/commands/shell.pyc
in <module>()</div>
<div>----> 1 </div>
<div> 2 </div>
<div> 3 </div>
<div> 4 </div>
<div>
5 </div>
<div><br>
</div>
<div>AttributeError: 'module' object has no attribute
'run_plugin'</div>
<div><br>
</div>
<div>In [2]: </div>
<div><br>
</div>
<div>Dan</div>
<div><br>
</div>
<br>
<div class="gmail_quote">
On Sat, Oct 30, 2010 at 11:45 AM, Dan Tenenbaum <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:dtenenba@fhcrc.org">dtenenba@fhcrc.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
Keeping this on the list.
<div>BTW, the new cluster.py has no syntax errors, so
something else must be happening.</div>
<div>Dan</div>
<div>
<div class="h5">
<div><br>
<br>
<div class="gmail_quote">On Sat, Oct 30, 2010 at 11:44
AM, Dan Tenenbaum <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:dtenenba@fhcrc.org" target="_blank">dtenenba@fhcrc.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt
0pt 0pt 0.8ex; border-left: 1px solid rgb(204,
204, 204); padding-left: 1ex;">Thanks. That gave
me a different error, below.
<div>The first time this happened, I did:<br>
sudo easy_install iptools, but it is still
happening.</div>
<div><br>
</div>
<div>Dan</div>
<div><br>
</div>
<div>
<div>$ starcluster shell</div>
<div>Leopard libedit detected.</div>
<div>ERROR: An unexpected error occurred while
tokenizing input</div>
<div>The following traceback may be corrupted or
invalid</div>
<div>The error message is: ('EOF in multi-line
statement', (405, 0))</div>
<div><br>
</div>
<div>---------------------------------------------------------------------------</div>
<div>ImportError
Traceback (most recent call last)</div>
<div><br>
</div>
<div>/Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/EGG-INFO/scripts/starcluster
in <module>()</div>
<div> 3 __requires__ = 'StarCluster==0.91'</div>
<div> 4 import pkg_resources</div>
<div>----> 5
pkg_resources.run_script('StarCluster==0.91',
'starcluster')</div>
<div> 6 </div>
<div>
7 </div>
<div><br>
</div>
<div>/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc
in run_script(self, requires, script_name)</div>
<div> 440 ns.clear()</div>
<div> 441 ns['__name__'] = name</div>
<div>--> 442
self.require(requires)[0].run_script(script_name,
ns)</div>
<div> 443 </div>
<div> 444 </div>
<div><br>
</div>
<div>/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc
in run_script(self, script_name, namespace)</div>
<div> 1165 )</div>
<div> 1166 script_code =
compile(script_text,script_filename,'exec')</div>
<div>-> 1167 exec script_code in
namespace, namespace</div>
<div> 1168 </div>
<div> 1169 def _has(self, path):</div>
<div><br>
</div>
<div>/Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/EGG-INFO/scripts/starcluster
in <module>()</div>
<div> 1 </div>
<div>----> 2 #!/usr/bin/python</div>
<div><br>
</div>
<div> 3 # EASY-INSTALL-SCRIPT:
'StarCluster==0.91','starcluster'</div>
<div><br>
</div>
<div> 4 __requires__ = 'StarCluster==0.91'</div>
<div> 5 import pkg_resources</div>
<div> 6
pkg_resources.run_script('StarCluster==0.91',
'starcluster')</div>
<div><br>
</div>
<div>/Users/dante/.starcluster/build/bdist.macosx-10.6-universal/egg/starcluster/cli.py
in <module>()</div>
<div> 36 </div>
<div> 37 </div>
<div>---> 38 </div>
<div> 39 </div>
<div> 40 </div>
<div><br>
</div>
<div>/Users/dante/.starcluster/build/bdist.macosx-10.6-universal/egg/starcluster/cluster.py
in <module>()</div>
<div> 10 </div>
<div> 11 </div>
<div>---> 12 </div>
<div> 13 </div>
<div> 14 </div>
<div><br>
</div>
<div>ImportError: cannot import name iptools</div>
<div>
<div>
<div><br>
</div>
<div><br>
<div class="gmail_quote">
On Sat, Oct 30, 2010 at 11:38 AM, Marc
Resnick <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:mresnick@mit.edu"
target="_blank">mresnick@mit.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin: 0pt 0pt 0pt 0.8ex;
border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;">The version
here<br>
<a moz-do-not-send="true"
href="http://github.com/jtriley/StarCluster/blob/experimental/starcluster/cluster.py"
target="_blank">http://github.com/jtriley/StarCluster/blob/experimental/starcluster/cluster.py</a><br>
may have those bugs fixed, as I
remember that one as well.<br>
<font color="#888888"><br>
<br>
Marc<br>
</font>
<div>
<div><br>
On Sat, Oct 30, 2010 at 2:31 PM,
Dan Tenenbaum <<a
moz-do-not-send="true"
href="mailto:dtenenba@fhcrc.org"
target="_blank">dtenenba@fhcrc.org</a>>
wrote:<br>
> I already did that. ;)<br>
> Now I'm getting another
error:<br>
> In [1]:
cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)<br>
>>>> Using private key
/Users/dante/.ec2/bioc-keypair.pem.rsa
(rsa)<br>
>
---------------------------------------------------------------------------<br>
> AttributeError
Traceback (most
recent call last)<br>
>
/Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc<br>
> in <module>()<br>
> ----> 1<br>
> 2<br>
> 3<br>
> 4<br>
> 5<br>
>
/Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc<br>
> in run_plugin(plugin_name,
cluster_tag, cfg)<br>
> 175 plugins = {}<br>
> 176
plugins[plugin_name] = plug<br>
> --> 177 plugins =
cl.load_plugins(plugins)<br>
> 178 master =
cl.master_node<br>
> 179 for p in plugins:<br>
>
/Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc<br>
> in load_plugins(self,
plugins)<br>
> 314 plugs = []<br>
> 315 for plugin in
plugins:<br>
> --> 316
setup_class =
plugin.get('setup_class')<br>
> 317
plugin_name =
plugin.get('__name__')<br>
> 318 mod_name
=
'.'.join(setup_class.split('.')[:-1])<br>
> AttributeError: 'str' object
has no attribute 'get'<br>
> Hmm....<br>
> Dan<br>
><br>
> On Sat, Oct 30, 2010 at 11:27
AM, Marc Resnick <<a
moz-do-not-send="true"
href="mailto:mresnick@mit.edu"
target="_blank">mresnick@mit.edu</a>>
wrote:<br>
>><br>
>> Yes, that's the bug I
told Justin about this Summer... I
think I<br>
>> patched it, but he may
have forgotten to pull from my
fork.<br>
>><br>
>> You can fix it by editing
the run_plugin method, and
replacing<br>
>><br>
>> cl =
get_cluster(cluster_tag)<br>
>><br>
>> with<br>
>><br>
>> cl =
get_cluster(cluster_tag, cfg)<br>
>><br>
>><br>
>><br>
>> Marc<br>
>><br>
>><br>
>> On Sat, Oct 30, 2010 at
2:20 PM, Dan Tenenbaum <<a
moz-do-not-send="true"
href="mailto:dtenenba@fhcrc.org"
target="_blank">dtenenba@fhcrc.org</a>>
wrote:<br>
>> > Hi Marc,<br>
>> > I am running 0.91.<br>
>> > I was doing a couple
things wrong. I figured them out
but I am still<br>
>> > having<br>
>> > this problem.<br>
>> > One thing I did
wrong was have the<br>
>> >
plugins=shortreadplugin<br>
>> > line under the
global section; it needed to be
under the<br>
>> > [cluster
smallcluster]<br>
>> > section.<br>
>> > When I start the
cluster now, I can see the output
from my plugin:<br>
>> >>>> Running
plugin shortreadplugin<br>
>> >>>> about to
echo<br>
>> > But, when I run the
shell and put in this command:<br>
>> >
cluster.run_plugin("shortreadplugin","myfirstcluster",cfg)<br>
>> > I get the same error
described below.<br>
>> > Dan<br>
>> ><br>
>> > On Sat, Oct 30, 2010
at 11:11 AM, Marc Resnick <<a
moz-do-not-send="true"
href="mailto:mresnick@mit.edu"
target="_blank">mresnick@mit.edu</a>>
wrote:<br>
>> >><br>
>> >> Hey Dan,<br>
>> >><br>
>> >> Are you running
the latest version? I saw that bug
a few months ago, I<br>
>> >> thought I
remember Justin fixing it.<br>
>> >><br>
>> >><br>
>> >> Marc<br>
>> >><br>
>> >> On Sat, Oct 30,
2010 at 2:00 PM, Dan Tenenbaum
<<a moz-do-not-send="true"
href="mailto:dtenenba@fhcrc.org"
target="_blank">dtenenba@fhcrc.org</a>><br>
>> >> wrote:<br>
>> >> > Hi,<br>
>> >> > I am not
much of a python programmer and
I'm trying to figure out the<br>
>> >> > plugin<br>
>> >> > system.<br>
>> >> > I created a
~/.starcluster/plugins directory
and put a file in it<br>
>> >> > called<br>
>> >> >
shortread.py.<br>
>> >> > Here are
its contents:<br>
>> >> ><br>
>> >> > from
starcluster.clustersetup import
ClusterSetup<br>
>> >> > from
starcluster.logger import log<br>
>> >> > class
ShortReadPlugin(ClusterSetup):<br>
>> >> > def
__init__(self, arg1):<br>
>> >> >
self.arg1 = arg1<br>
>> >> > <a
moz-do-not-send="true"
href="http://log.info"
target="_blank">log.info</a>('arg1
= %s' % arg1)<br>
>> >> > def
run(self, nodes, master, user,
user_shell, volumes):<br>
>> >> > <a
moz-do-not-send="true"
href="http://log.info"
target="_blank">log.info</a>("about
to echo")<br>
>> >> >
master.ssh.execute("echo 'hey'")<br>
>> >> > As you can
see, it's just a test.<br>
>> >> > In my
config, I put this at the bottom:<br>
>> >> > [plugin
shortreadplugin]<br>
>> >> > SETUP_CLASS
= ShortReadPlugin<br>
>> >> > arg1=foo<br>
>> >> > And above
in the [global] section I put:<br>
>> >> >
plugins=shortreadplugin<br>
>> >> > I'm a bit
confused about whether this is
right because in your<br>
>> >> > example:<br>
>> >> > <a
moz-do-not-send="true"
href="http://web.mit.edu/stardev/cluster/docs/plugins.html"
target="_blank">http://web.mit.edu/stardev/cluster/docs/plugins.html</a><br>
>> >> > ...you have
the SETUP_CLASS as
ubuntu.PackageInstaller. I'm not
sure<br>
>> >> > where<br>
>> >> > the
"ubuntu" fits into this. Is it
like java where I need to have a<br>
>> >> > folder<br>
>> >> > called
ubuntu where the class resides?<br>
>> >> > Anyway, I
fired up my cluster and went to
test this with "starcluster<br>
>> >> > shell"<br>
>> >> > and got the
following output:<br>
>> >> > In [5]:
cluster.run_plugin('shortreadplugin',
'myfirstcluster', cfg)<br>
>> >> ><br>
>> >> ><br>
>> >> >
---------------------------------------------------------------------------<br>
>> >> > TypeError
Traceback (most recent call<br>
>> >> > last)<br>
>> >> ><br>
>> >> ><br>
>> >> >
/Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cli.pyc<br>
>> >> > in
<module>()<br>
>> >> > ----> 1<br>
>> >> > 2<br>
>> >> > 3<br>
>> >> > 4<br>
>> >> > 5<br>
>> >> ><br>
>> >> ><br>
>> >> >
/Library/Python/2.6/site-packages/StarCluster-0.91-py2.6.egg/starcluster/cluster.pyc<br>
>> >> > in
run_plugin(plugin_name,
cluster_tag, cfg)<br>
>> >> > 170 def
run_plugin(plugin_name,
cluster_tag, cfg):<br>
>> >> > 171
ec2 = cfg.get_easy_ec2()<br>
>> >> > --> 172
cl = get_cluster(cluster_tag)<br>
>> >> > 173
cl.load_receipt()<br>
>> >> > 174
plug = cfg.get_plugin(plugin_name)<br>
>> >> > TypeError:
get_cluster() takes exactly 2
arguments (1 given)<br>
>> >> ><br>
>> >> > It's hard
for me to understand what's
happening here. I of course am<br>
>> >> > not<br>
>> >> > calling
get_cluster directly. Is there a
bug in run_plugin() or is<br>
>> >> > there<br>
>> >> > something
wrong in my setup, or both?<br>
>> >> > Thanks<br>
>> >> > Dan<br>
>> >> ><br>
>> >> >
_______________________________________________<br>
>> >> > StarCluster
mailing list<br>
>> >> > <a
moz-do-not-send="true"
href="mailto:StarCluster@mit.edu"
target="_blank">StarCluster@mit.edu</a><br>
>> >> > <a
moz-do-not-send="true"
href="http://mailman.mit.edu/mailman/listinfo/starcluster"
target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
>> >> ><br>
>> >> ><br>
>> >><br>
>> ><br>
>> ><br>
>><br>
><br>
><br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>