[StarCluster] Creating a StarCluster AMI

[MacMullan, Hugh]

Hi Everyone:

The old (and a couple of newer) 'cookbooks' are on the StarCluster GitHub wiki, at https://github.com/jtriley/StarCluster/wiki. You could probably put together a newer 'cookbook' from those.

But basically:

GE (OGS) in /opt/sge6-fresh. Creating CentOS 7 (and maybe 6.5?) I needed to get the latest binary version from http://sourceforge.net/projects/gridscheduler/files/ ... there's probably a compat-glibc workaround, but ... that was simple enough. Just untar/gz and move the newly-created directory to /opt/sge6-fresh.

Root key access needs to be enabled, as well, so /etc/cloud/cloud.cfg needs to have 'disable_root: false'. /etc/ssh/sshd_config should have 'PermitRootLogin without-password'.

For the portmap issue in more recent versions of Ubuntu & RedHat/CentOS it's a pretty simple fix:

echo 'exit 0' > /etc/init.d/portmap
chmod +x /etc/init.d/portmap

That gets the NFS working, assuming that the standard NFS server software is installed.

Beyond that, it just depends what you're doing with your cluster. If you're doing MPI, you'll need that. I think the OEM packages (CentOS/RedHat & Ubuntu) versions work with SGE now (compiled with '--with-sge').

I recently (last night, actually!) took the Official StarCluster Ubuntu 12.04 HVM AMI and did 'do-release-upgrade' from an imagehost. Don't forget to 'apt-get update && apt-get upgrade' first. A bunch of packages had issues, but just removing and re-installing the whole set of them (maybe 20) seems to have worked fine, and got me straight to 14.04, with Enhanced Networking, although the latest driver doesn't work on 14.04 (14.04 was called 'too new' by the developers of the ixgbevf driver. :)

I've also recently created CentOS 6.5 & RHEL 7 AMIs, all HVM images, with the Enhanced Networking attribute set (what I was really after here). If anyone is interested in package lists or further details for any of that stuff, feel free to let me know. Still working on documents.

-Hugh

-----Original Message-----
From: starcluster-bounces at mit.edu [mailto:starcluster-bounces at mit.edu] On Behalf Of Pierre Mariani
Sent: Wednesday, September 17, 2014 1:47 PM
To: starcluster at mit.edu
Subject: Re: [StarCluster] Creating a StarCluster AMI

> Hi All,
>
> I need to generate an ami that comply with the company golden image and
> would like it to be used in a star cluster , SGE environment.
>
> Are there any instructions as for how to bake a starcluster ami rather than
> using those that are there ?
>
> What is the minimum packages and configuration that is required to be
> installed on an image so it can be used as a Starcluster node ?

Disclosure: I am a recent user of StarCluster and may be missing
something important.

I went through the same issue very recently, and this is what I found out:

The most relevant documentation I was able to find is
http://star.mit.edu/cluster/docs/0.95.5/manual/create_new_ami.html. It
is helpful for some aspects, but unfortunately it doesn't go into
details of what should be installed on the AMI.

So I ended up reading StarCluster code at
https://github.com/jtriley/StarCluster/blob/0.95.5/starcluster/. This
is helpful for getting an idea of which packages the framework is
expecting to find on the AMI, although an actual list would be nicer
to work with.

But, a more serious issue is that the code is making some low level
assumptions about these packages.

Let's take the example of NFS. The code is assuming that NFS is being
started with /etc/init.d/nfs start, and /etc/init.d/portmap start.
In the case of Ubuntu, it was correct up to 13.04. But in 14.04, it's
different. These files don't exist anymore. There are new files that
seem to replace them, but I don't know if there are more fundamental
changes beside just the file name.

I tried to work around this particular NFS issue by customizing the
AMI and symlinking the actual NFS files to what StarCluster was
expecting, but I bumped into another issue right after that.

At that point I felt I was on a wild goose chase and decided to fork,
update and maintain my own version of the Ubuntu 12.04 AMI and work
with that. (13.04 has reached EOL, 14.04 isn't officially supported
yet, so 12.04 is best choice available).

I don't know what StarCluster contributors' opinions are on this.
Personally, I think that it is unmaintainable to manually handle the
many different ways to interface with NFS, package managers and other
required packages across multiple distributions and multiple versions
of each distribution. Ideally StarCluster could leverage a 3rd party
library that did that, but I don't know if such a library exists.

I would be very interested in reading the experiences and opinions of
other members of the community on this.

Thank you.
Pierre
_______________________________________________
StarCluster mailing list
StarCluster at mit.edu
http://mailman.mit.edu/mailman/listinfo/starcluster