[Starcluster] Bug VolumeCreator _request_instance

[Justin Riley]

Hi Joachim,

Thanks for reporting this. Adding an --instance-type option to the 
createvolume
command is on my todo list now and this should fix your issue.

For the record though, there's no need to use a 64bit image to create a new
volume. The reason I chose m1.small as a default is because this instance is
*only* used to mount the volume and automate the partitioning/formatting. For
this there really isn't any advantage to using a 64bit image and using 
m1.small
is your cheapest option.

I would recommend using a 32bit alestic image (alestic.com), or *any* 32bit
image, in the eu-west-1 zone with the createvolume command. This avoids 
needing
a different instance type. Regardless, I'll be adding checks for invalid
instance_type/image_id combos though in the VolumeCreator class in addition to
allowing instance_type to be passed to the createvolume action at the command
line.

Also, I've decided to replicate the StarCluster AMIs in all availability zones
so that folks outside the us-east-1 zone do not have to migrate the AMIs
themselves. The whole point of StarCluster is to abstract most of the details
of using EC2 to create HPC clusters in the cloud. Having users migrate the AMI
themselves to other zones is suboptimal. I'll send a notice to the list in the
next week or so when the us-west/eu-west/ap-souteast AMIs are available

Hope that helps and thanks again for reporting.

~Justin

Quoting Joachim Van den Bogaert <joachim at ccl.kuleuven.be>:

> Hi Justin,
>
>
>
> I tried to create a volume with the following command:
>
>
>
> joachim at tm1:~$ starcluster createvolume 100 eu-west-1b --image-id
> ami-6be5cf1f
>
>
>
>>>> Using keypair sge
>
>>>> Requesting host instance in zone eu-west-1b to attach volume to...
>
>>>> Creating security group @sc-volumecreator...
>
>>>> No instance in group @sc-volumecreator for zone eu-west-1b, launching
> one now.
>
>
>
> volume.py:186 - ERROR - exception thrown: EC2ResponseError: 400 Bad Request
>
> <?xml version="1.0"?>
>
> <Response>
>
>  <Errors>
>
>    <Error>
>
>      <Code>InvalidParameterValue</Code>
>
>      <Message>The requested instance type's architecture (i386) does not
> match the architecture in the manifest for ami-6be5cf1f (x86_64)</Message>
>
>    </Error>
>
>  </Errors>
>
>  <RequestID>4ff61a79-6a82-480e-b1dd-e26d8965eb70</RequestID>
>
> </Response>
>
>
>
> I was looking for an architecture parameter, but I couldn't find one
>
>
>
> (volume.py line 32 - 56)
>
>
>
> def _request_instance(self, zone):
>
>        for i in self.security_group.instances():
>
>            if i.state in ['pending','running'] and i.placement == zone:
>
>                log.info("Using existing instance %s in group %s" % \
>
>                         (i.id,self.security_group.name))
>
>                self._instance = Node(i, self._key_location, 'vol_host')
>
>                break
>
>        if not self._instance:
>
>            log.info("No instance in group %s for zone %s, launching one
> now." % \
>
>                     (self.security_group.name, zone))
>
>            self._resv = self._ec2.run_instances(image_id=self._image_id,
>
>                instance_type='m1.small',
>
>                min_count=1, max_count=1,
>
>                security_groups=[self.security_group.name],
>
>                key_name=self._keypair,
>
>                placement=zone)
>
>            instance = self._resv.instances[0]
>
>            self._instance = Node(instance, self._key_location, 'vol_host')
>
>        s = Spinner()
>
>        log.info("Waiting for instance %s to come up..." %
> self._instance.id)
>
>        s.start()
>
>        while not self._instance.is_up():
>
>            time.sleep(15)
>
>        s.stop()
>
>        return self._instance
>
>
>
> Apparently the code in line 43
>
>
>
> instance_type='m1.small'
>
>
>
> throws an error if you try to create a volume with a 64-bit StarCluster AMI,
> since you can't launch an m1.small with 64 bit images.
>
>
>
> I am in the eu zone, so I can't use the AMI defined in
> image_id=static.BASE_AMI_32.
>
> With the 64-bit image I use instead, an extra parameter could prevent this
> error
>
>
>
> After I changed the line into
>
>
>
> instance_type='m1.large'
>
>
>
> the volume was created successfully.
>
>
>
>
>
>
>
> Kind regards,
>
> Joachim
>
>
>
>
>
>
>
>