<div dir="ltr">Thanks Rayson! <br></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 26, 2015 at 7:49 AM, Rayson Ho <span dir="ltr"><<a href="mailto:raysonlogin@gmail.com" target="_blank">raysonlogin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><span class=""><div>On Tue, Jan 13, 2015 at 12:35 PM, Jian Feng <span dir="ltr"><<a href="mailto:freedafeng@gmail.com" target="_blank">freedafeng@gmail.com</a>></span> wrote:</div></span><div class="gmail_quote"><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>But on the other hand, your idea to assign two security groups can solve my problem too. I just did not know we can assign multiple sg to an instance. (I did not see this functionality in the UI.) I guess we can only do this programmatically? Could you or anyone please elaborate how this is done? <br></div></div></blockquote><div><br></div></span><div>The standard EC2 python API supports passing in a list of security groups when you request for EC2 instances -- you will notice that "security_groups" & "security_group_ids" (for VPC) both accept a list of groups:</div><div><br></div><div><a href="https://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.run_instances" target="_blank">https://boto.readthedocs.org/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.run_instances</a><br></div><div><br></div><div>In fact, I hacked the code in StarCluster so that it uses both my pre-created security group and the group that it creates during the StarCluster bootstrap process, and the code change was minimal.</div><div><div class="h5"><div><br></div>Rayson<br><br>==================================================<br>Open Grid Scheduler - The Official Open Source Grid Engine<br><a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br><div><a href="http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html" target="_blank">http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html</a></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><br></div>Thanks!<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 13, 2015 at 7:51 AM, Ramon Ramirez-Linan <span dir="ltr"><<a href="mailto:rlinan@navteca.com" target="_blank">rlinan@navteca.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>For question 1</div><div>What do you think about adding an option where the user can specify a SG to server as a template for the Cluster's SG?</div><div>So if I have a production SG (psg) StarCluster can copy that group psg_sc, that way the workflow of destroying a cluster will be safer since psg_sc only exist for the purpose of that cluster. I think this option would be easier to implement.</div><div><br></div><div>For question 2,</div><div>I would also like to see this feature, what I have been doing for now is to bake the compute node storage in the AMI</div><div><br></div><div><br></div><div><br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 12, 2015 at 11:45 AM, Rayson Ho <span dir="ltr"><<a href="mailto:raysonlogin@gmail.com" target="_blank">raysonlogin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hi Jian,<div><br></div><div>StarCluster puts information in the cluster security group to identify whether a cluster exists or not (you may have encountered the message " !!! ERROR - Cluster 'sc' already exists.". While we can rename or clear the tags added to a security group, forcing StarCluster to use a predefined security group can affect the current code we have.</div><div><br></div><div>IMO, a better way to satisfy your need is to add your security group (the one that's used as the traffic source in the production security group) together with the security group create by Starcluster to each cluster node. This way, only 1 or 2 functions in the existing StarCluster code need to be changed.</div><div><br></div><div>One the other hand, forcing StarCluster to use a predefined security group can cause issues as StarCluster's cleanup code won't be able to delete your group, because it is referenced in the production security group. While StarCluster probably shouldn't touch your predefined security group at all if one is supplied, it does change how StarCluster cleans up the security group when you terminate the cluster.</div><div><br></div><div>Finally, an added benefit is that you can document what that security group is for in your audit log, and also apply that security group to any instances that need to access the production cluster. This way, you can have multiple StarClusters and/or other services access the production cluster, while at the same time keeping the traffic from flowing through each other.</div><div class="gmail_extra"><br clear="all"><div><div>Rayson<br><br>==================================================<br>Open Grid Scheduler - The Official Open Source Grid Engine<br><a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br><a href="http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html" target="_blank">http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html</a></div></div><div><br></div><div><br></div>
<br><div class="gmail_quote">On Tue, Jan 6, 2015 at 3:52 PM, Jian Feng <span dir="ltr"><<a href="mailto:freedafeng@gmail.com" target="_blank">freedafeng@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Dear StarCluster devs,<br><br></div>As a loyal starcluster user, I am wondering if it's ok for me to submit a feature request to meet needs of more people and businesses. I'd like to see<br><br></div>1. security group separated from cluster creation. In AWS the sg and instances are two totally independent things. Security group and live without any instances. This is convenient because we'd like to reuse the security group for different clusters. This is especially true when we need to fine tune the security rules for different type of work. In my case, I need my starcluster cluster to access the production cluster data. So I will have to change the production security group rules every time I create a new star cluster. This is not efficient and not easy to be fully automated. <br></div><div> It's totally fine to keep the current setting but give people an option to reuse an existing security group. <br></div><div><br></div>2. local ebs disk storage for each node. I see a great amount applications have this needs. This is crucial for any distributed persistent data storage. <br><br></div><div>I understand that starcluster is open source so people can tailor it to their own needs. The thing is some functionalities can help people to improve productivity a lot so they might be worth adding.<br></div><div><br></div>Thanks!<br></div>
<br>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br></div></div>
<br>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>