<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Paolo,<br>
<br>
Well it probably won't be the *next* version given that I need to
put out a patch release today/tomorrow but it should make it in to
the next major release.<br>
<br>
As far as the config goes the only thing that will change vs using a
plugin to do this is that the cluster template itself would now
support a user_data option which would allow you to specify
user_data in multiple templates easily rather than having to create
separate plugin sections for each template in order to specify
different scripts. We should also add a command line flag to the
'start' command that specifies a single user_data file to pass to
all nodes in the cluster.<br>
<br>
~Justin<br>
<br>
On 2/9/12 9:50 AM, Paolo Di Tommaso wrote:<br>
<span style="white-space: pre;">> Dear Justin, all, <br>
><br>
> I think this will be a quite useful enhancement to
StarCluster. <br>
><br>
> Actually I tried to solve the problem writing a custom
plugin, but the main limitation was that I had to define the
data/file to pass as user_data in the plugin definition block (as
a plugin parameter) of the configuration file, and this was really
unhandy. <br>
><br>
><br>
> So I'm really happy to read that next version will include
this feature. <br>
><br>
><br>
> Thanks,<br>
> Paolo<br>
> <br>
><br>
><br>
> On Feb 8, 2012, at 9:00 PM, Justin Riley wrote:<br>
></span><br>
<blockquote type="cite">Awesome, thanks! Yea, it may not be
universal but at least it allows<br>
standard shell scripts (and apparently YAML) to work as expected
where<br>
it previously did not so I think it's worth while including. I'm
just<br>
speculating but my guess is scripts are probably the most common<br>
user-data.<br>
<br>
Once we have the code in and tested the next step is to document
how<br>
to use user-data with StarCluster and add it to the user manual...<br>
<br>
~Justin<br>
<br>
On 02/07/2012 09:38 PM, Joydeep Sen Sarma wrote:<br>
>>> ok - i think the patch is pretty simple - let me file
a pull<br>
>>> request. i had added the user_data param as well ..<br>
>>><br>
>>> i think the '#' strategy works for yaml as well
(looking one<br>
>>> message below). i was just afraid it wasn't
universal.<br>
>>><br>
>>> On Tue, Feb 7, 2012 at 11:03 PM, Justin Riley<br>
>>> <<a class="moz-txt-link-abbreviated" href="mailto:justin.t.riley@gmail.com">justin.t.riley@gmail.com</a>
<a class="moz-txt-link-rfc2396E" href="mailto:justin.t.riley@gmail.com"><mailto:justin.t.riley@gmail.com></a>><br>
>>> wrote:<br>
>>><br>
>>> Hi Paolo/Joydeep,<br>
>>><br>
>>> You're correct that StarCluster currently uses the
user-data to<br>
>>> store node-aliases which at the moment kills the
ability to utilize<br>
>>> user-data scripts. It is possible to run user-scripts
on the entire<br>
>>> cluster fairly easily, however, using a StarCluster
plugin. I'm<br>
>>> happy to provide an example or perhaps include it in
a future<br>
>>> release if you're interested.<br>
>>><br>
>>> The reason for storing the alias in user-data is that
I needed some<br>
>>> way to 'stick' a name to a node at launch time. The
new tagging API<br>
>>> for EC2 is useful but it's still not possible to
assign names at<br>
>>> launch time. My current approach is to use the
user-data at launch<br>
>>> time, then later fetch it from user-data once the
instance is up<br>
>>> and apply an alias tag. This is useful for example if
things crash<br>
>>> somewhere in the middle - the nodes are guaranteed to
be named the<br>
>>> same thing the next time around because of the
user-data.<br>
>>><br>
>>> Your solution of effectively 'commenting' the alias
line in<br>
>>> user-data so that other scripts can run is not a bad
solution<br>
>>> actually. I'd be willing to accept a pull request for
this and<br>
>>> include the functionality in a future release. We
need to add a<br>
>>> USER_DATA config parameter to the cluster config that
would let<br>
>>> users point to a user-data script.<br>
>>><br>
>>> Would you be interested in contributing this back?
I'm happy to<br>
>>> help you along with the fork/pull request process if
you're<br>
>>> willing. There's also a contribute guide that gives
an overview of<br>
>>> the process:<br>
>>><br>
>>>
<a class="moz-txt-link-freetext" href="http://web.mit.edu/star/cluster/docs/latest/contribute.html">http://web.mit.edu/star/cluster/docs/latest/contribute.html</a><br>
>>><br>
>>> ~Justin<br>
>>><br>
>>><br>
>>> On 01/24/2012 11:41 AM, Joydeep Sen Sarma wrote:<br>
>>>> It doesn't.<br>
>>>><br>
>>>> Currently the user data is used by starcluster
itself to store<br>
>>> node aliases.<br>
>>>><br>
>>>> I had a similar requirement as yours - but in my
case - the user<br>
>>>> data was restricted to shell scripts. So what i
did for my use<br>
>>>> case was<br>
>>> hack<br>
>>>> the starcluster code to use the first line of the
user data -<br>
>>> prepended<br>
>>>> with a '#' for node aliases - and the rest of the
user data could<br>
>>>> be used as a shell script (filled in when
launching the cluster<br>
>>>> from the configuration).<br>
>>>><br>
>>>> it seemed pretty hacky to me - so didn't try to
contribute back.<br>
>>>><br>
>>>> there may be other ways of achieving this
outcome. for example -<br>
>>>> one could take the cluster-name and the instance
alias (node001<br>
>>>> etc.) and then map that to a location in s3<br>
>>>> (s3://urbucket/cluster-name/node001) and then
store instance<br>
>>>> specific user data in that location. the boot
scripts can get the<br>
>>>> cluster name and the instance alias from ec2 apis
and then use<br>
>>>> that to locate the instance data and get rolling.<br>
>>>><br>
>>>> On Tue, Jan 24, 2012 at 9:22 PM, Paolo Di Tommaso
<br>
>>>> <<a class="moz-txt-link-abbreviated" href="mailto:Paolo.DiTommaso@crg.eu">Paolo.DiTommaso@crg.eu</a>
<a class="moz-txt-link-rfc2396E" href="mailto:Paolo.DiTommaso@crg.eu"><mailto:Paolo.DiTommaso@crg.eu></a><br>
>>> <<a class="moz-txt-link-freetext" href="mailto:Paolo.DiTommaso@crg.eu">mailto:Paolo.DiTommaso@crg.eu</a>
<a class="moz-txt-link-rfc2396E" href="mailto:Paolo.DiTommaso@crg.eu"><mailto:Paolo.DiTommaso@crg.eu></a>>><br>
>>> wrote:<br>
>>>><br>
>>>> Dear all,<br>
>>>><br>
>>>> Does StarCluster provide any support for EC2 user
data feature?<br>
>>>><br>
>>>> <a class="moz-txt-link-freetext" href="http://www.turnkeylinux.org/blog/ec2-userdata">http://www.turnkeylinux.org/blog/ec2-userdata</a><br>
>>>><br>
>>>><br>
>>>> I would like to use this mechanism to specify
some extra<br>
>>>> instance configuration at node boot time.<br>
>>>><br>
>>>> If no, is there any other alternative?<br>
>>>><br>
>>>><br>
>>>> Thank you, Paolo Di Tommaso<br>
>>>><br>
>>>> Software Engineer Comparative Bioinformatics
Group Centre de<br>
>>>> Regulacio Genomica (CRG) Dr. Aiguader, 88 08003
Barcelona, Spain<br>
>>>><br>
>>>><br>
>>>> _______________________________________________
StarCluster<br>
>>>> mailing list <a class="moz-txt-link-abbreviated" href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a>
<a class="moz-txt-link-rfc2396E" href="mailto:StarCluster@mit.edu"><mailto:StarCluster@mit.edu></a><br>
>>> <<a class="moz-txt-link-freetext" href="mailto:StarCluster@mit.edu">mailto:StarCluster@mit.edu</a>
<a class="moz-txt-link-rfc2396E" href="mailto:StarCluster@mit.edu"><mailto:StarCluster@mit.edu></a>><br>
>>>>
<a class="moz-txt-link-freetext" href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> _______________________________________________
StarCluster<br>
>>>> mailing list <a class="moz-txt-link-abbreviated" href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a>
<a class="moz-txt-link-rfc2396E" href="mailto:StarCluster@mit.edu"><mailto:StarCluster@mit.edu></a> <br>
>>>>
<a class="moz-txt-link-freetext" href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> This body part will be downloaded on demand.<br>
<br>
</blockquote>
<span style="white-space: pre;">>>
_______________________________________________<br>
>> StarCluster mailing list<br>
>> <a class="moz-txt-link-abbreviated" href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
>> <a class="moz-txt-link-freetext" href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
><br>
><br>
> _______________________________________________<br>
> StarCluster mailing list<br>
> <a class="moz-txt-link-abbreviated" href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
> <a class="moz-txt-link-freetext" href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a></span><br>
<br>
<br>
</body>
</html>