[Starcluster] create_image.py vs. createimage

Thomas Deselaers deselaers at vision.ee.ethz.ch
Thu Apr 15 05:54:53 EDT 2010


Hi Justin,

Thanks a lot for your prompt reply.

I just played a bit with the new version to create a new image.

I already solved a few problems but now I am at a point where I don't
know what to do anymore.

Could you give me a pointer what I should change?

I manually started an instance (64bit image) and modified it a bit.
Then I tried to run the commands below.

You also find the output of starcluster.


Thanks in advance

export EC2_PRIVATE_KEY="/home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa"
export EC2_CERT=/home/desthoma/work/text/aws/cert-XXXXXXXXXXXXXXXXXXXXXXXXXXXXxx.pem

starcluster createimage i-73eXXXXX myStarcluster starclusterAMIBucket

StarCluster - (http://web.mit.edu/starcluster)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster at mit.edu

>>> Using private key /home/desthoma/work/text/aws/id_ECKeypair-us-east.rsa (rsa)
---------------------------------------------------------------------------
AuthenticationException                   Traceback (most recent call last)

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
in <module>()
      3 __requires__ = 'StarCluster==0.9999'
      4 import pkg_resources
----> 5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
      6
      7

/home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
in run_script(self, requires, script_name)
    487         ns.clear()
    488         ns['__name__'] = name
--> 489         self.require(requires)[0].run_script(script_name, ns)
    490
    491

/home/desthoma/work/lib/python/setuptools-0.6c11-py2.5.egg/pkg_resources.py
in run_script(self, script_name, namespace)
   1212             )
   1213             script_code = compile(script_text,script_filename,'exec')
-> 1214             exec script_code in namespace, namespace
   1215
   1216     def _has(self, path):

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/EGG-INFO/scripts/starcluster
in <module>()
      4 import pkg_resources
      5 pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
----> 6
      7
      8

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
in main()
    750         sys.exit(0)
    751     try:
--> 752         sc.execute(args)
    753     except exception.BaseException,e:
    754         log.error(e.msg)

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/cli.pyc
in execute(self, args)
    359             'bucket': bucket,
    360         })
--> 361         icreator = image.EC2ImageCreator(**kwargs)
    362         icreator.create_image()
    363

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
in __init__(self, instance, aws_access_key_id, aws_secret_access_key,
aws_user_id, bucket, ec2_cert, ec2_private_key, prefix,
remove_image_files, **kwargs)
     51         }
     52         if self.host:
---> 53             self.config_dict['arch'] = self._get_arch()
     54
     55     @print_timing

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/image.pyc
in _get_arch(self)
     72
     73     def _get_arch(self):
---> 74         conn = self.host.ssh
     75         arch = conn.execute(
     76             'python -c "import platform; print
platform.architecture()[0]"'

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/node.pyc
in ssh(self)
    147             self._ssh = ssh.Connection(self.instance.dns_name,
    148                                        username=self.user,
--> 149                                        private_key=self.key_location)
    150         return self._ssh
    151

/home/desthoma/work/lib/python/StarCluster-0.9999-py2.5.egg/starcluster/ssh.pyc
in __init__(self, host, username, password, private_key,
private_key_pass, port)
     77                     raise TypeError, "You have not specified a
password or key."
     78
---> 79             self._transport.connect(username = username, pkey = pkey)
     80
     81     def _load_rsa_key(self, private_key, private_key_pass=None):

/var/lib/python-support/python2.5/paramiko/transport.pyc in
connect(self, hostkey, username, password, pkey)
    987             else:
    988                 self._log(DEBUG, 'Attempting public-key auth...')
--> 989                 self.auth_publickey(username, pkey)
    990
    991         return

/var/lib/python-support/python2.5/paramiko/transport.pyc in
auth_publickey(self, username, key, event)
   1214             # caller wants to wait for event themselves
   1215             return []
-> 1216         return self.auth_handler.wait_for_response(my_event)
   1217
   1218     def auth_interactive(self, username, handler, submethods=''):

/var/lib/python-support/python2.5/paramiko/auth_handler.pyc in
wait_for_response(self, event)
    172             if issubclass(e.__class__, PartialAuthentication):
    173                 return e.allowed_types
--> 174             raise e
    175         return []
    176





On Wed, Apr 14, 2010 at 21:20, Justin Riley <jtriley at mit.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Thomas,
>
> The github code's createimage command is the recommended way, however, I
> haven't made a new release out of this code yet. You're welcome to try
> out the latest code; it's very close to ready. The config has been
> enhanced quite a bit past the previous 0.90.1 version so you'll
> certainly want to look at the docs in github:
>
> http://github.com/jtriley/StarCluster/tree/master/docs/sphinx
>
> The *.rst files are the raw text documents. These docs are also
> available on the web here:
>
> http://web.mit.edu/stardev/cluster/docs/
>
> I'm in the process of cleaning up the code in github and finishing up
> some documentation for a new release that supports multiple clusters,
> multiple ebs volumes, cluster 'plugins' that allow users to write their
> own custom setup routines on top of starcluster's defaults, automated
> EBS volume creation (including partition/format), and more.
>
> If you could also report any issues (either bugs or usability issues)
> with the github code to this list I'd appreciate it.
>
> Thanks!
>
> ~Justin
>
>
>
> On 04/14/2010 11:39 AM, Thomas Deselaers wrote:
>> Dear all,
>>
>> I am a new (since today) StarCluster user and I already managed to get
>> a few things done (start my "own" cluster, run jobs, etc). The
>> documentation is really fine (and I know SGE from the past).
>>
>> Now I would like to modify the starcluster AMIs to my needs.
>>
>> In the mailinglist I found various  references to a create_image.py
>> script, which I cannot find: neither in the mist recent stable version
>> (0.90.1) nor in the git-version.
>>
>>
>> In the git version, I found the command `createimage' for starcluster
>> (starcluster -h), and assume that this is the replacement. Now I am
>> wondering if this is the recommended way at the moment or if there are
>> better choices.
>>
>> Best, and thanks for making star cluster available.
>> thomas
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.14 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkvGFZYACgkQ4llAkMfDcrmc1QCdEWkhSUtP9K2s6xw8oOmtzJ+5
> pDMAnjRV4yBGjRvgRgrmUJLoNSeBLLuz
> =diB3
> -----END PGP SIGNATURE-----
>
>



-- 
http://thomas.deselaers.de




More information about the StarCluster mailing list