[Starcluster] StarCluster instances not coming up

[Justin Riley]

Hi Rob,

Thanks for your interest in StarCluster and sorry that you're having problems. 
I've cc'd our mailing list on this one so that others will benefit this 
thread, hope you don't mind. I would also like to invite you to join the list 
for issues in the future.

Regarding your issue, I'm guessing you meant to say 'starcluster -s' got stuck 
waiting for the cluster to come up? If so, you should check that you have port 
22 open on 0.0.0.0/0 in your default security group.  You can modify your 
security group permissions using the AWS management console or the ElasticFox 
firefox plugin. 

The reason this permissions is needed is that StarCluster does its magic via 
an ssh connection to each instance and therefore needs to be able to connect 
on port 22 from your machine. Typically users set this up after following the 
Amazon EC2 Getting Started Guide so that they have some way of logging into 
their system(s).

Let me know if this fixes the issue for you.

The next version of StarCluster, coming soon, will not have this problem 
because it creates security groups for each cluster and automatically 
configures the firewall settings for ssh. Stay tuned.

~Justin



On Saturday 30 January 2010 8:48:43 pm rentzsch at biochem.ucl.ac.uk wrote:
> Dear Justin,
> 
> I'm currently experimenting with using a remote EC2 cluster to generate
> data for my PhD project (bioinformatics). First of all thanks for starting
> this off, I think it will potentially save me a lot of time (and if it's
> only to find the EC2 thing or EC2+cluster thing isn't stable enough yet,
> from the Amazon site especially).
> 
> I just had a first go but 'starcluster -m' got stuck waiting for the
> cluster to come up. I checked with the Management Console and both
> instances were pending for >15m, so I terminated them and Ctrl-C'ed
> StarCluster. I'm not experienced at all with using the AWS so far and
> before I even try to find out what's happening here (it might be a bug in
> the current AMI init code?) maybe you can have a look at the attached text
> file, which is a full console dump from one of the instances.
> 
> Two lines strike me as potentially interesting, although the second seems
> to occur during shutdown (so when I terminated the instance) and might be
> related to an Ubuntu bug (Google brings st like that up when entering the
> error msg) no matter if running in Xen or not. The first one might be more
> interesting:
> 
> /etc/init.d/rc: 390: /etc/rcS.d/S50ec2-ssh-host-key-gen: Permission denied
>  * Starting system log daemon...
> 
> halt: Unable to iterate IDE devices: No such file or directory
> System halted.
> 
> I'd be grateful if you could have a quick look!
> 
> All the best,
> Rob
>