[Starcluster] Eucalyptus support

[Justin Riley]

Hi Nicolas,

I've cc'd the list on this one, hope you don't mind. 

Before modifying starcluster to connect to Eucalyptus I decided to try it out 
manually using ipython shell and boto. I wasn't able to get it to work using 
your config settings for some reason, however, I set up my own single-node 
eucalyptus cluster today locally and I'm happy to report that boto seems to 
work well with my local eucalyptus cluster.

Keep in mind that I haven't done any rigorous testing outside of just playing 
around with a few of the boto API calls and there very well may be other 
issues lurking. Initial tests seem to indicate that it's possible....we'll 
see. I'm adding in configuration support now and will make the necessary 
modifications to awsutils.py in the github code so that we can test this out 
further.

Stay tuned,

~Justin

On Sunday 21 March 2010 6:01:22 pm you wrote:
> Dear Justin,
> 
> Many thanks for the Starcluster ISOs that you provided recently.
> 
> I've successfully managed to bundle and upload the x86_64 iso to a UEC
> (eucalyptus) ami. I've also been successful in getting the instance
> running but I'm facing two problems:
> 
> 1) I cannot ssh into the instance using e.g.
> 
>  ssh -i ~/.ssh/blah.priv sgeadmin at XXX.XXX.XXX.XXX
> 
> I've also tried "ubuntu" and "root" as usernames but still no luck.
> Please note that I can login to other ubuntu-based amis started with
> this keypair with no problem, so there's nothing wrong with my
> keypairs or with group permissions and such (e.g. ec2-authorize
> default -P tcp -p 22 -s 0.0.0.0/0, etc)
> 
> 
> 2) I installed the latest starcluster code using git and then I tried
> to modify the python code in order to make it work with Eucalyptus.
> I've discovered some extra parameters that I can pass to
> boto.connect_ec2 and  boto.connect_s3 in order to make them connect to
> my UEC front-end server rather than Amazon, and the code compiles
> allright. I've also set up a config file with my UEC credentials.
> 
> However, when I run e.g. "starcluster start smallcluster test" I'm
> getting the following error:
> 
> Traceback (most recent call last):
>   File "/usr/local/bin/starcluster", line 5, in <module>
>     pkg_resources.run_script('StarCluster==0.9999', 'starcluster')
>   File
>  "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/pytho
> n/pkg_resources.py", line 442, in run_script
>     self.require(requires)[0].run_script(script_name, ns)
>   File
>  "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/pytho
> n/pkg_resources.py", line 1167, in run_script
>     exec script_code in namespace, namespace
>   File
>  "/Library/Python/2.6/site-packages/StarCluster-0.9999-py2.6.egg/EGG-INFO/s
> cripts/starcluster", line 6, in <module>
> 
>   File "build/bdist.macosx-10.6-universal/egg/starcluster/cli.py",
> line 546, in main
>   File "build/bdist.macosx-10.6-universal/egg/starcluster/cli.py",
> line 157, in execute
>   File "build/bdist.macosx-10.6-universal/egg/starcluster/config.py",
> line 291, in get_cluster
>   File "build/bdist.macosx-10.6-universal/egg/starcluster/cluster.py",
> line 148, in __init__
>   File "build/bdist.macosx-10.6-universal/egg/starcluster/awsutils.py",
> line 51, in __init__
> TypeError: __init__() takes exactly 4 arguments (8 given)
> 
> I understand that this error that I'm getting is due to the extra
> arguments that I've added but, unfortunately, my python skills are not
> that good and the _init__() is defined in multiple places. So, for
> example, I'm not sure whether I have to make a global change in "class
> EasyAWS(object)" or to each individual class (e.g. class
> EasyEC2(EasyAWS)). In addition, I'm not certain that more UEC related
> parameters aren't needed in other parts of the code.
> 
> To this end, I've decided to ask for your help in case that you can
> enlighten me further. I'm attaching the file awsutils.py where I've
> made the changes and the config file that I'm using, so perhaps you
> can try it out for yourelf in our UEC server.
> 
> 
> Many thanks,
> 
> 
> Nicholas