[StarCluster] access instance storage

[Justin Riley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Sebastian,

Unfortunately this is not possible when creating a new HVM image using
the ebsimage command. This is a limitation imposed by EC2's
'create-image' API call. The 'create-image' call does not support
specifying a block device mapping that's different from the image host's
block device mapping. This is why you don't see the ephemeral storage on
your new 11.04 HVM image; the original alestic HVM didn't have it in the
block device mapping. To make matters worse the EC2 API documentation
specifically states that an HVM image must be created from an existing
HVM instance in order for the AMI to be HVM-compatible:

http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/using_cluster_computing.html#using_cluster_computing_hvm_ami_running_instance

There may be a way around this though. It is possible to manually
specify a block device mapping when launching the instances. This would
allow StarCluster to override the default block device mapping specified
by a given AMI and always provide ephemeral storage even if the original
AMI creator did not include it when registering the image. I'll
experiment with this and see what I can come up with. I've created a
github issue to keep track of this:

https://github.com/jtriley/StarCluster/issues/46

Thanks for reporting,

~Justin

On 08/08/2011 09:30 AM, sebastian rooks wrote:
> Hello Justin,
> 
> I would like to use the instance storage (2x 840G disks) that comes 
> with the cluster compute instance type. This is again for an Ubuntu 
> 11.04 hvm AMI.
> 
> The scratch space is not mounted on /mnt: root at master:/mnt/sgeadmin#
> df -h Filesystem            Size  Used Avail Use% Mounted on 
> /dev/sda1             7.9G  4.0G  3.5G  54% / none
> 12G  144K   12G   1% /dev none                   12G     0   12G   0%
> /dev/shm none                   12G   60K   12G   1% /var/run none
> 12G     0   12G   0% /var/lock /dev/xvdz1             20G  618M   19G
> 4% /home
> 
> The block devices are indeed not available on my AMI:
> 
> ec2-describe-images ami-ab5197c2 -K ~/.starcluster/pk-....pem -C 
> ~/.starcluster/cert-....pem IMAGE	ami-ab5197c2	11-04-hvm	available
> private		x86_64	machine   ebs	hvm BLOCKDEVICEMAPPING	/dev/sda1
> snap-8f055aee	8
> 
> Nor are they on Alestic's public one:
> 
> ec2-describe-images ami-1cad5275 -K ~/.starcluster/pk-....pem -C 
> ~/.starcluster/cert-....pem IMAGE	ami-1cad5275
> 099720109477/hvm/ubuntu-images/ubuntu-natty-11.04-amd64-server-20110426
> 099720109477	available	public		x86_64	machine ebs	hvm 
> BLOCKDEVICEMAPPING	/dev/sda1		snap-b1ad2dde	8
> 
> Would it be possible to add the local instance storage during the
> AMI creation with ebsimage ? 
> http://docs.amazonwebservices.com/AWSEC2/2011-01-01/UserGuide/index.html?Using_AddingDefaultLocalInstanceStorageToAMI.html
>
>  Regards,
> 
> Sebastian _______________________________________________ StarCluster
> mailing list StarCluster at mit.edu 
> http://mailman.mit.edu/mailman/listinfo/starcluster

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5AJGkACgkQ4llAkMfDcrnmmQCdESKjN3GWw3edHajdDJb4ukc1
ZEMAnRetpFfu+ayuaat1wboKsyjOoVlh
=9BgL
-----END PGP SIGNATURE-----