[StarCluster] access instance storage

[Justin Riley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Also, for your immediate needs, you can launch an instance of the
Alestic HVM 11.04 AMI manually and specify the block mapping including
the ephemeral storage device(s). Using ebsimage on an instance launched
with a custom block device mapping should create the new AMI with an
identical block device mapping. Let me know if you need help with this.
Basically the launch command should look something like:

ec2-run-instances <ami id> -k <your keypair name> --block-device-mapping
'/dev/sdb1=ephemeral0' --block-device-mapping '/dev/sdc1=ephemeral1'
- --block-device-mapping '/dev/sdd1=ephemeral2' --block-device-mapping
'/dev/sde1=ephemeral3'

~Justin

On 08/08/2011 02:01 PM, Justin Riley wrote:
> Hi Sebastian,
> 
> Unfortunately this is not possible when creating a new HVM image
> using the ebsimage command. This is a limitation imposed by EC2's 
> 'create-image' API call. The 'create-image' call does not support 
> specifying a block device mapping that's different from the image
> host's block device mapping. This is why you don't see the ephemeral
> storage on your new 11.04 HVM image; the original alestic HVM didn't
> have it in the block device mapping. To make matters worse the EC2
> API documentation specifically states that an HVM image must be
> created from an existing HVM instance in order for the AMI to be
> HVM-compatible:
> 
> http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/using_cluster_computing.html#using_cluster_computing_hvm_ami_running_instance
>
>  There may be a way around this though. It is possible to manually 
> specify a block device mapping when launching the instances. This
> would allow StarCluster to override the default block device mapping
> specified by a given AMI and always provide ephemeral storage even if
> the original AMI creator did not include it when registering the
> image. I'll experiment with this and see what I can come up with.
> I've created a github issue to keep track of this:
> 
> https://github.com/jtriley/StarCluster/issues/46
> 
> Thanks for reporting,
> 
> ~Justin
> 
> On 08/08/2011 09:30 AM, sebastian rooks wrote:
>> Hello Justin,
> 
>> I would like to use the instance storage (2x 840G disks) that comes
>>  with the cluster compute instance type. This is again for an
>> Ubuntu 11.04 hvm AMI.
> 
>> The scratch space is not mounted on /mnt:
>> root at master:/mnt/sgeadmin# df -h Filesystem            Size  Used
>> Avail Use% Mounted on /dev/sda1             7.9G  4.0G  3.5G  54% /
>> none 12G  144K   12G   1% /dev none                   12G     0
>> 12G   0% /dev/shm none                   12G   60K   12G   1%
>> /var/run none 12G     0   12G   0% /var/lock /dev/xvdz1
>> 20G  618M   19G 4% /home
> 
>> The block devices are indeed not available on my AMI:
> 
>> ec2-describe-images ami-ab5197c2 -K ~/.starcluster/pk-....pem -C 
>> ~/.starcluster/cert-....pem IMAGE	ami-ab5197c2	11-04-hvm	available 
>> private		x86_64	machine   ebs	hvm BLOCKDEVICEMAPPING	/dev/sda1 
>> snap-8f055aee	8
> 
>> Nor are they on Alestic's public one:
> 
>> ec2-describe-images ami-1cad5275 -K ~/.starcluster/pk-....pem -C 
>> ~/.starcluster/cert-....pem IMAGE	ami-1cad5275 
>> 099720109477/hvm/ubuntu-images/ubuntu-natty-11.04-amd64-server-20110426
>>
>> 
099720109477	available	public		x86_64	machine ebs	hvm
>> BLOCKDEVICEMAPPING	/dev/sda1		snap-b1ad2dde	8
> 
>> Would it be possible to add the local instance storage during the 
>> AMI creation with ebsimage ? 
>> http://docs.amazonwebservices.com/AWSEC2/2011-01-01/UserGuide/index.html?Using_AddingDefaultLocalInstanceStorageToAMI.html
>
>>  Regards,
> 
>> Sebastian _______________________________________________
>> StarCluster mailing list StarCluster at mit.edu 
>> http://mailman.mit.edu/mailman/listinfo/starcluster
> 
> _______________________________________________ StarCluster mailing
> list StarCluster at mit.edu 
> http://mailman.mit.edu/mailman/listinfo/starcluster

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5ALGQACgkQ4llAkMfDcrkdkQCfafIHF1jdFFG5XlIz4ogsqdMH
1AkAn3pT4Sm1b4jA+Bdh4EIiDxDboKDE
=ZK3g
-----END PGP SIGNATURE-----