[StarCluster] dealing with security groups

[Justin Riley]

Hi Igor,
> I was really impressed how easy it was to use StarCluster - thanks! 
> :) but I can't figure out how to work around EC2 security groups to 
> make it work for us.
Thanks, glad you like it :D
> We need to be able to mount an NFS share (which is an EBS volume). I 
> understand that StarCluster can attach and share an EBS volume but we 
> already have our EBS volumes attached and used by other nodes. So 
> instead of attaching an EBS volume we need to mount an existing NFS 
> share. Does this make sense? I wonder how many people have a similar 
> set up? (We also need access to our NIS server and possibly other 
> services)

This makes sense and you'll certainly have to manage mounting the 
external NFS share in a plugin.

> By default, new EC2 security groups have all of the ports closed 
> (except ssh) so one thing I could do is to open the relevant ports at 
> cluster startup.

Yes, you'll also have to add the relevant lines to /etc/fstab and run 
the 'mount' command for each share on each node off the top of my head. 
Modifying fstab and running 'mount' will need to be done in a plugin.

> Alternatively, I'd rather not deal with security groups at all. 
> 'default' would work fine for us. Unfortunately, it looks like the 
> code assumes that a cluster has its own security group with a certain 
> name. So I guess that's not an option, right?

Currently each cluster has it's own security group. This group is mostly 
used for accounting but is also useful if you want to apply a firewall 
to each individual cluster launched. The latest github code has support 
for applying security group permissions after the group is created which 
might meet your needs, however, currently the permission settings do not 
support specifying group-group permissions. I can add this fairly easily 
though...

Also, this use case suggests it would be handy to be able to specify 
additional security groups (such as 'default') to add each of the 
cluster nodes to in addition to their own (@sc-*). This would remove the 
need to add permissions since they'd all be in a common group (e.g. 
default).

> If I follow the first approach, I would need to do something like this:
>    ec2-authorize default -p <nfs-port> -o @sc-mycluster
> right? or do I also need to allow access from sc-cluster to default?
>
> Do I need to revoke these permissions when the cluster shuts down or 
> will EC2 take care of that as long as StarCluster deletes the security 
> group (does it?)

When the cluster shuts down, StarCluster deletes the cluster's security 
group so I'd imagine that would take care of the group-group permission 
but we'll need to test this to make sure that's the case. If the group 
is deleted the rule can't apply but it may still be defined. If this is 
the case the rule would have to be cleaned up.

~Justin