[StarCluster] making a leaner AMI

[Dan Tenenbaum]

Hi all,

I'm trying to get my cluster instances to start up faster.

I saw the cookbook for creating a StarCluster AMI from an Alestic Ubuntu
10.10 AMI:
http://starcluster.scripts.mit.edu/~starcluster/wiki/index.php?title=StarCluster_AMI_Cookbook_Ubuntu_10.04

This is great!

What I'd like to know is, which of the packages in the apt-get install line
(and the pip install line) are essential for me?

I do NOT need SunGridEngine, nor am I doing any numerical work in python. I
will need be using openmpi with R.

(I realize I may need to change the StarCluster code so it doesn't try and
launch/set up SunGrid Engine on the cluster nodes. I would like to avoid
having to fork, though--could this maybe be made into a config option? I
could send you a patch...)

So, which packages in that list are essential for StarCluster to operate
with openmpi? And which ones can be safely *NOT* installed?

I would especially like to get rid of packages that do things at startup.
I'd like the cluster instances to come up quickly. I have done some
extensive testing (I can share the results with anyone who is interested)
and a "bare" alestic ubuntu AMI starts up in about 30 seconds, and the
StarCluster AMI starts up in about 3 minutes.

BTW, another suggestion for improving startup time would be to increase the
frequency with which StarCluster polls to see if the instances have started
up. It currently polls every 60 seconds. Hopefully with the leanest possible
AMI, we could maybe poll every 15 seconds. Could I submit another patch for
this? ;)

Thanks,
Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20101028/4a4f7536/attachment.htm