[StarCluster] Using StarCluster with Hudson Continuous Integration

[Justin Riley]

Hi Vaughan,

My apologies for taking so long to respond. StarCluster uses Python's
os.path.expanduser to determine the user's home directory and in this
case it's returning '/root' for some reason. My guess is Hudson runs as
root and runs the CI script as the Hudson user but doesn't clear the
environment so that $HOME still points to /root which causes
os.path.expanduser to return the wrong value. You might try exporting
$HOME explicitly to the right place before running StarCluster and see
if that helps.

If this doesn't work please feel free to join #starcluster on IRC and we
can try to debug this:

http://webchat.freenode.net/?channels=starcluster

~Justin

On Thu, Jun 14, 2012 at 09:11:50PM +0000, Vaughan, Garrison wrote:
>    Hello,
> 
>    I'm currently working as an intern with NASA at the Goddard Space Flight
>    Center. We have decided to move
>    some regression tests we run on our own high performance computer system
>    into the cloud.
>    I have been assigned the task of setting up a continuous integration
>    server and setting up the scripts which will
>    instantiate clusters that will be used for the builds/regression test. I
>    was shown StarCluster by a system architect here
>    and needless to say, it's an extremely impressive tool!
> 
>    I am currently using a CI server called Hudson which is running smoothly
>    on a micro AMI instance which mimics the OS image we run on our cluster.
>    The Hudson Server runs under a user (user Hudson) on the instance and can
>    run scripts as that user on the system when triggered.
>    The point is to, when triggered, create a cluster (using StarCluster) that
>    will mimic the cluster we run here,
>    and use that cluster to build and do a regression test against a clients
>    code repository, and subsequently tear the
>    cluster down upon completion.
> 
>    I have StarCluster installed under user Hudson, and when I shell into user
>    Hudson, StarCluster works without a hitch.
>    My problem occurs when I try to run any StarCluster command through the
>    Hudson Web Servlet. I have a job going on Hudson
>    which, when triggered, will execute a shell command (currently
>    "starcluster help") under user Hudson. After setting up the $PATH to
>    StarCluster
>    through the Hudson UI, the starcluster command is recognized.
> 
>    But for some reason, which I've been struggling with all day, I get this
>    error as output when I run any starcluster command:
> 
>    #######Hudson Console Output#######
> 
>  Started by user admin
>  [workspace] $ /bin/bash -xe /tmp/hudson6888230102812979297.sh ## [workspace] is a directory under user Hudson. When the same command
>                                                                                                                                 ## which is in the /tmp/hudsonxxxx.sh script (currently "starcluster help") is
>                                                                                                                                 ## run from this directory through ssh, it runs just fine.
> 
>  + starcluster help                                                                                             ##This is the command in /tmp/hudsonxxxxx.sh. A whoami shows this command running under user Hudson.
>  !!! ERROR - /root/.starcluster *must* be a directory                              ##This is what has me baffled, why does anything in /root need to be referenced/exist?
>                                                                                                                                 ##Again, this isn't the case when this command is run directly under user Hudson (through ssh).
>   !!! ERROR - /root/.starcluster *must* be a directory
>  Traceback (most recent call last):
>    File "/usr/local/bin/starcluster", line 9, in <module>
>      load_entry_point('StarCluster==0.93.3', 'console_scripts', 'starcluster')()
>    File "/usr/local/lib64/python2.6/site-packages/StarCluster-0.93.3-py2.6.egg/starcluster/cli.py", line 310, in main
>      logger.configure_sc_logging()
>    File "/usr/local/lib64/python2.6/site-packages/StarCluster-0.93.3-py2.6.egg/starcluster/logger.py", line 129, in configure_sc_logging
>      backupCount=2)
>    File "/usr/lib64/python2.6/logging/handlers.py", line 112, in __init__
>      BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
>    File "/usr/lib64/python2.6/logging/handlers.py", line 64, in __init__
>      logging.FileHandler.__init__(self, filename, mode, encoding, delay)
>    File "/usr/lib64/python2.6/logging/__init__.py", line 827, in __init__
>      StreamHandler.__init__(self, self._open())
>    File "/usr/lib64/python2.6/logging/__init__.py", line 846, in _open
>      stream = open(self.baseFilename, self.mode)
>  IOError: [Errno 13] Permission denied: '/root/.starcluster/logs/debug.log'
>  [DEBUG] Skipping watched dependency update for build: Test Build #30 due to result: FAILURE
>  Finished: FAILURE
> 
>  .starcluster exists at /var/lib/hudson/.starcluster and is shown as the default path to the config file when "starcluster help" is run on user Hudson (through ssh).
>  I have passed that directory as a  (starcluster -c "path_to_config") argument in the Hudson script, and still get the same error.
>  Permissions for /var/lib/hudson/.starcluster are rwxr-xr-x.
> 
>  Any help would be greatly appreciated!
> 
>  Thanks for the great work,
>  Garrison

> _______________________________________________
> StarCluster mailing list
> StarCluster at mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://mailman.mit.edu/pipermail/starcluster/attachments/20120731/4557b1f5/attachment.bin