<div dir="ltr"><div><div><div><div>Hi Rayson and Ramon,<br><br>Thanks for the clarifications and comments!<br><br></div>Sorry for replying late -- was stuck in some production issues. <br><br></div>I understand that starcluster uses security group as the cluster identification so that it knows which ec2 instances belongs to which cluster. This is totally fine and I couldn't find any better way to ID the cluster instances. What I was proposing is to decouple the (creation/deletion) of cluster instances and the security group. It still holds that there is at most one cluster using any specific security group. The only difference is that the security group won't be deleted when we terminate a cluster, and it won't (or does not need to) be created when we create a cluster. <br><br></div>But on the other hand, your idea to assign two security groups can solve my problem too. I just did not know we can assign multiple sg to an instance. (I did not see this functionality in the UI.) I guess we can only do this programmatically? Could you or anyone please elaborate how this is done? <br><br></div>Thanks!<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 13, 2015 at 7:51 AM, Ramon Ramirez-Linan <span dir="ltr"><<a href="mailto:rlinan@navteca.com" target="_blank">rlinan@navteca.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>For question 1</div><div>What do you think about adding an option where the user can specify a SG to server as a template for the Cluster's SG?</div><div>So if I have a production SG (psg) StarCluster can copy that group psg_sc, that way the workflow of destroying a cluster will be safer since psg_sc only exist for the purpose of that cluster. I think this option would be easier to implement.</div><div><br></div><div>For question 2,</div><div>I would also like to see this feature, what I have been doing for now is to bake the compute node storage in the AMI</div><div><br></div><div><br></div><div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 12, 2015 at 11:45 AM, Rayson Ho <span dir="ltr"><<a href="mailto:raysonlogin@gmail.com" target="_blank">raysonlogin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Jian,<div><br></div><div>StarCluster puts information in the cluster security group to identify whether a cluster exists or not (you may have encountered the message " !!! ERROR - Cluster 'sc' already exists.". While we can rename or clear the tags added to a security group, forcing StarCluster to use a predefined security group can affect the current code we have.</div><div><br></div><div>IMO, a better way to satisfy your need is to add your security group (the one that's used as the traffic source in the production security group) together with the security group create by Starcluster to each cluster node. This way, only 1 or 2 functions in the existing StarCluster code need to be changed.</div><div><br></div><div>One the other hand, forcing StarCluster to use a predefined security group can cause issues as StarCluster's cleanup code won't be able to delete your group, because it is referenced in the production security group. While StarCluster probably shouldn't touch your predefined security group at all if one is supplied, it does change how StarCluster cleans up the security group when you terminate the cluster.</div><div><br></div><div>Finally, an added benefit is that you can document what that security group is for in your audit log, and also apply that security group to any instances that need to access the production cluster. This way, you can have multiple StarClusters and/or other services access the production cluster, while at the same time keeping the traffic from flowing through each other.</div><div class="gmail_extra"><br clear="all"><div><div>Rayson<br><br>==================================================<br>Open Grid Scheduler - The Official Open Source Grid Engine<br><a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br><a href="http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html" target="_blank">http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html</a></div></div><div><br></div><div><br></div>
<br><div class="gmail_quote">On Tue, Jan 6, 2015 at 3:52 PM, Jian Feng <span dir="ltr"><<a href="mailto:freedafeng@gmail.com" target="_blank">freedafeng@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Dear StarCluster devs,<br><br></div>As a loyal starcluster user, I am wondering if it's ok for me to submit a feature request to meet needs of more people and businesses. I'd like to see<br><br></div>1. security group separated from cluster creation. In AWS the sg and instances are two totally independent things. Security group and live without any instances. This is convenient because we'd like to reuse the security group for different clusters. This is especially true when we need to fine tune the security rules for different type of work. In my case, I need my starcluster cluster to access the production cluster data. So I will have to change the production security group rules every time I create a new star cluster. This is not efficient and not easy to be fully automated. <br></div><div> It's totally fine to keep the current setting but give people an option to reuse an existing security group. <br></div><div><br></div>2. local ebs disk storage for each node. I see a great amount applications have this needs. This is crucial for any distributed persistent data storage. <br><br></div><div>I understand that starcluster is open source so people can tailor it to their own needs. The thing is some functionalities can help people to improve productivity a lot so they might be worth adding.<br></div><div><br></div>Thanks!<br></div>
<br>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br></div></div>
<br>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>