<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Here's some help with creating an
Ubuntu 14.04 AMI that works as a Starcluster node(master/worker).
Below are directions for creating your own Ubuntu 14.04 LTS AMI. I
didn't create a public AMI because I didn't want to step on the
toes of the developers and I didn't want to be responsible for
sanitizing private information off of an AMI that is made
publically available. I also give the disclaimer that these
directions "appear" to work to fix all issues with NFS and
portmapper -- but I only did light testing (e.g. start/terminate
clusters and add & remove nodes - SGE & NFS seem to work
as they should). Also the fixes I mention below I collected based
on my own experience and others contributions to the mailing list.<br>
<br>
The portmapper issue isn't how I solved the issue on the version
of Starcluster that I use in production because I don't have a
problem editing to make the fix because there are some other
tweaks I also edit the code for (like adding new C4 instance types
in). That said it seems that the fix that creates a file
/etc/init.d/portmap with "exit 0" seems to work -- my guess is the
calls to portmap aren't really needed. The initial part of the
script runs update and installs updates to the distribution, then
it installs nfs-server and copies the init.d file to nfs as it is
called by Starcluster scripts. Python is installed and then SGE
is downloaded, unzipped and renamed to /sge6-fresh so Starcluster
scripts work as advertised.<br>
<br>
How to make your own Ubuntu 14.04 LTS AMI:<br>
---------------------------------------<br>
1) Find your AMI look <a
href="http://cloud-images.ubuntu.com/locator/ec2/">here</a> for
all Ubuntu 14.04 LTS AMIs for your region -- grab the AMI ID for
my example I used AMI ID: ami-36f8b15e -- its us-east-1,
14.04LTS, 64bit, ebs-ssd<br>
-- make sure to select and EBS backed EC2 AMI (not
instance-store)<br>
<br>
2) a) Login to AWS Console -- click on "launch an instance"<br>
b) select "Community AMIs" and put the AMI id from step one in
to find it (here I used "ami-36f8b15e"), once it is found <br>
click "Select"<br>
c) Select instance type you want (choose something small
because we are just creating an AMI to use) - click NEXT<br>
d) For "configure instance details" section -- make sure the
instance is placed in your default VPC -- just so you CAN access
it if <br>
need be <br>
e) Under the "Advanced Details" section click the
triangle/arrow so that it expands. Paste in the following in the
"User data" section. It <br>
runs the bash script that will install needed packages for
making Starcluster work along with fixing issues with NFS and
portmapper. <br>
This ASSUMES you are not using the StarCluster master node
as StarCluster "master" AND StarCluster "controller". If you want
to do<br>
this add "easy_install StarCluster" (no quotes included)
after the last line "chmod +x /etc/init.d/portmap". Adding this
line will<br>
install StarCluster on the EC2 you are launching. After
you have pasted in the script in "User data" - click NEXT<br>
#!/bin/bash<br>
apt-get update<br>
apt-get dist-upgrade -y<br>
apt-get install nfs-kernel-server nfs-common rpcbind libgssglue1
libnfsidmap2 libtirpc1 -y<br>
cp /etc/init.d/nfs-kernel-server /etc/init.d/nfs<br>
apt-get install sysv-rc-conf -y<br>
sysv-rc-conf nfs on<br>
/etc/init.d/nfs restart<br>
apt-get install python-setuptools python-dev -y<br>
cd /opt/<br>
wget <a class="moz-txt-link-freetext"
href="http://dl.dropbox.com/u/47200624/respin/ge2011.11.tar.gz">http://dl.dropbox.com/u/47200624/respin/ge2011.11.tar.gz</a><br>
gunzip < ge2011.11.tar.gz | tar xvf -<br>
mv ge2011.11/ sge6-fresh/<br>
cd /<br>
echo 'exit 0' > /etc/init.d/portmap<br>
chmod +x /etc/init.d/portmap<br>
<br>
f) Click Next - so that you are in the "Tag Instance" section
and Name your instance. I used "Ubuntu 14.04 Starcluster ready" <br>
g) Click Next - so you are in "Configure Security Group"
section -- here either select a pre-existing Security Group that
you can <br>
SSH into -- OR -- Create a new security group that
allows you to SSH into it from "My IP" (which is your IP).<br>
h) Click Review and Launch<br>
i) Click Launch <br>
j) Select an existing Keypair you have access too (so you can
SSH into the instance) and click the check box and click "Launch<br>
Instances" OR create a new keypair, download that keypair
and then click "Launch Instances".<br>
<br>
3) Wait about 5-10 minutes and "Status Checks" should change from
"Initializing" to " 2/2 checks passed" -- You can login to the
instance and make sure issuing the command <br>
"sudo vi /etc/init.d/portmap" opens the file and shows the
command "exit 0". This means the script completed and you can
logout.<br>
<br>
4) Stop the running instance by selecting "Actions - Instance
State - Stop"<br>
<br>
5) Select "Actions - Image - Create Image" to create the AMI --
name it something like "Ubuntu 14.04 Starcluster node AMI" and
then select "Create Image" You will see "Create Image request
received. View pending image ami-XXXX" note that AMI ID -- and
that's the AMI ID you will use to create your Ubuntu 14.04
Starcluster master and worker nodes.<br>
<br>
6) Once the AMI ID ami-XXXX - has Status "available" you can
terminate the EC2 Instance we launched with the User data script
because it is no longer needed. Use that AMI ID ami-XXXX in <br>
the StarCluster config file AND/OR when you want to launch
master and worker nodes that are Ubuntu 14.04 LTS. <br>
<br>
Potential Issues:<br>
---------------<br>
Step 1: DON'T choose "instance store" -- this isn't an EBS backed
EC2 so when it is stopped all adjustments to OS are lost. Use
something that says "ebs" in the instance type. <br>
ebs = magnetic disk, ebs-ssd = the general purpose ssd
storage, ebs-ios = the Provisioned IOPS ssd storage.<br>
Step 3: WAIT at least 5-10 minutes before stopping the EC2
Instance -- also it is really good idea to login to the EC2 and
check that /etc/init.d/portmap says "exit 0" so you <br>
know that the script ran.<br>
Virtualization type issue -- Specifically your AMI is PV or HVM
and you need the other type.<br>
1) Go to Snapshots -- locate snapshot of AMI ID ami-XXX - note
the "Description" has AMI ID in its text. Select that snapshot.<br>
2) Select "Actions - Create Image" -- note in "Virtualization
type" section you can select "Paravirtual" or "Hardware-assisted
virtualization" -- so select virtualization type you<br>
need, Name the AMI, and select the other features you want
then click "Create". This creates the AMI of the opposite
virtualization type without any hassle. Using the <br>
snapshot to create and AMI of the virualization of your
choice is WAY EASIER than the method I had used previously. <br>
<br>
Good Luck!<br>
-Jennifer<br>
<br>
<br>
On 2/16/15 12:27 PM, Cedar McKay wrote:<br>
</div>
<blockquote cite="mid:0B989CFC-4D58-43AC-A7D8-5FDF8D8DC9BB@uw.edu"
type="cite">
<pre wrap="">Some people, like Jennifer, have managed to get 14.04 AMIs working. It would be a real service if those people could share them.
Also, does anyone have a reasonable explanation for why the official version is so slow in coming? Is there some sort of issue?
Thanks,
Cedar
</pre>
<blockquote type="cite">
<pre wrap="">On Jan 29, 2015, at 6:41 AM, Pablo Escobar <a class="moz-txt-link-rfc2396E" href="mailto:pescobar001@gmail.com"><pescobar001@gmail.com></a> wrote:
Hi Jennifer
thank you very much for your help. It's really useful.
regards,
Pablo.
2015-01-28 19:10 GMT+01:00 Jennifer Staab <a class="moz-txt-link-rfc2396E" href="mailto:jstaab@cs.unc.edu"><jstaab@cs.unc.edu></a>:
I installed Starcluster on Ubuntu 14.04 LTS and it is working fine for me. I didn't use the guide you mentioned, just a little guidance from emails on mailing list between Jacob and Hugh regarding installation (see here) and some sleuthing to get it to work. Hugh also provides a guideline/cookbook for creating new Starcluster AMI's here on the mailing list and said that just upgrading the existing AMI for 12.04 to 14.04 worked for him. To me that seems like the easiest way to handle this whole issue is to use existing Ubuntu 12.04 Starcluster AMI and just upgrade it to 14.04. I wasn't able to do this, so below is what worked for me. I am sure there are other ways to create a new Ubuntu 14.04 Starcluster AMI.
</pre>
</blockquote>
</blockquote>
<br>
<blockquote cite="mid:0B989CFC-4D58-43AC-A7D8-5FDF8D8DC9BB@uw.edu"
type="cite">
<blockquote type="cite">
<pre wrap="">--------------------------------------------------------------------------------------
1) Locate the AMI ID of the latest version of Ubuntu 14.04 for your region (see here). Start up your Ubuntu 14.04 AMI -- login and run update and apply all the security and distribution updates.
2) Install packages "python-dev" and "python-setuptools" -- You are welcome to install other packages, but those are the only ones needed if to do the Starcluster install. A list of all python packages can be found here .
3) You need to install NFS server so install package "nfs-kernel-server". Installing this package should install its dependencies (for me that included: nfs-common rpcbind libgssglue1 libnfsidmap2 libtirpc1). One issue I found was that in previous versions the startup script located in /etc/init.d/ was named "nfs" this installed version it's named "nfs-kernel-server". The easiest fix is to create a symbolic link to "/etc/init.d/nfs" (ln -s /etc/init.d/nfs-kernel-server /etc/init.d/nfs). Test by issuing the command "/etc/init.d/nfs restart" if it restarts then all is well.
4) Install Starcluster with "easy_install Starcluster"
5) You need to get a copy of the Open Grid Scheduler and place it in "/opt/sge6-fresh/" -- this is what Hugh and Jacob discuss in their emails on the mailing list. The easiest way I was able to do this was to use a running Ubuntu 12.04 Starcluster AMI (ami-765b3e1f) and copy "/opt/sge-fresh" from the 12.04 version to my Ubuntu 14.04 version. There are other ways to do this see their emails here.
6) Ubuntu 14.04 replaces portmap with rpcbind -- so this was an issue for me because the Starcluster software expects "portmap" to exist. So this is where I had to "edit" the code a bit.There are other ways to deal with the portmap issue, look here for an alternative solution. This is just how I handled the issue: locate the "node.py" portion of the code in the installed StarCluster version, mine was in "/usr/local/lib/python2.7/dist-packages/StarclusterXXX.egg/starcluster/node.py" On line 731 in the function "mount_nfs_shares()" you see line "self.ssh.execute('/etc/init.d/portmap start')". This won't work on Ubuntu 14.04 since "portmap" was replaced with "rpcbind". One way to fix this issue without breaking the code for other OS versions still using portmap is by replacing "self.ssh.execute('/etc/init.d/portmap start')" with a "try/except" statement so that if portmap doesn't work it calls rpcbind command instead; thus, handling the issue with portmap on Ubuntu 14.04. B!
</pre>
</blockquote>
<pre wrap=""> elow is how I changed the code in node.py:
</pre>
<blockquote type="cite">
<pre wrap="">FIND line 731:
self.ssh.execute('/etc/init.d/portmap start')
REPLACE with:
try:
self.ssh.execute('/etc/init.d/portmap start')
except:
self.ssh.execute('service rpcbind restart')
7) Stop this instance and take an AMI of it. Use that AMI for nodes on your Starcluster clusters. If it bothers you to have Starcluster installed on the AMI you are using in your Starcluster clusters skip steps 2 and 4 BUT be certain to edit the version of Starcluster software you are using to create your Ubuntu 14.04 clusters to deal with portmap/rpcbind issue as described in step 6.
A few things to note:
1) I only use starcluster commands to create clusters, terminate them, addnodes and removenodes (on-demand & spots) -- and this is what has been working for me. I haven't tested the other functionality, so there could be other issues like the portmap one that I don't know about. For the other functionality (like stopping instances, saving AMIs, etc), I use AWS directly to perform those functions.
2) The version of StarCluster that I used was 0.95.6 -- as available via easy_install. These issues (or ones similar to them) have been discussed and a few fixed in some of the development branches of Starcluster available on github see here and here.
3) I had some issue installing other software, but I am guessing most of that has been resolved now that it has been so long since the initial release of Ubuntu 14.04.
Good Luck,
Jennifer
On 1/27/15 3:59 PM, Pablo Escobar wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi all,
I plan to use starcluster with ubuntu14.04. I will adapt the ami following this guide
<a class="moz-txt-link-freetext" href="https://www.youtube.com/watch?v=2RBupgpi_ec">https://www.youtube.com/watch?v=2RBupgpi_ec</a>
Is anyone in the list using starcluster with ubuntu 14.04 in production? Is it working fine or is there any drawback? is there any problem I should be aware before start using starcluster with ubuntu14.04?
I know this has been asked before, but is there any plan to release a ubuntu14.04 official ami?
thanks in advance for any advice.
best regards,
Pablo.
_______________________________________________
StarCluster mailing list
<a class="moz-txt-link-abbreviated" href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a>
<a class="moz-txt-link-freetext" href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a>
</pre>
</blockquote>
<pre wrap="">
_______________________________________________
StarCluster mailing list
<a class="moz-txt-link-abbreviated" href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a>
<a class="moz-txt-link-freetext" href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a>
</pre>
</blockquote>
<pre wrap="">
_______________________________________________
StarCluster mailing list
<a class="moz-txt-link-abbreviated" href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a>
<a class="moz-txt-link-freetext" href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a>
</pre>
</blockquote>
<br>
</body>
</html>