Hi Rayson,<div><br></div><div>Thanks for your answer and helpful material. I´ll read it carefully.</div><div><br></div><div>All the best,</div><div><br>Sergio<br><br><div class="gmail_quote">On Wed, May 30, 2012 at 8:12 PM, Rayson Ho <span dir="ltr"><<a href="mailto:raysonlogin@gmail.com" target="_blank">raysonlogin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Sergio,<br>
<br>
It would be great if you can develep a web interface for StarCluster -<br>
and I can help you test the code...<br>
<br>
For queue status, Grid Engine (or "Open Grid Scheduler" - which is the<br>
project that maintains the open source grid engine code base) has<br>
XML-qstat, which was originally developed by Chris from the BioTeam,<br>
and now it is available at:<br>
<br>
<a href="http://olesenm.github.com/xml-qstat/" target="_blank">http://olesenm.github.com/xml-qstat/</a><br>
<a href="http://olesenm.github.com/xml-qstat/screenshots.html" target="_blank">http://olesenm.github.com/xml-qstat/screenshots.html</a><br>
<br>
There is also a web portal developed by Sun, but I don't think anyone<br>
is using that one - at least not with any modern versions of Grid<br>
Engine anyway.<br>
<br>
Outside of Grid Engine, there is a lightweight web portal that handles<br>
job submission, file upload, queue status, etc for OpenPBS (not Torque<br>
- it was not tested with Torque, AFAIK), and the project is called<br>
PBSWeb:<br>
<br>
<a href="http://webdocs.cs.ualberta.ca/~pinchak/PBSWeb/" target="_blank">http://webdocs.cs.ualberta.ca/~pinchak/PBSWeb/</a><br>
<br>
I talked to Paul Lu (the author) 11 years ago, and he said that PBSWeb<br>
was developed by him & his grad students. He said that PBSWeb is<br>
available under the GPL, and he was happy to accept changes from other<br>
people. According to Paul, internally inside PBSWeb, the web front-end<br>
invokes an SSH connection to the submission machine for each qsub, and<br>
this was how he handled user switching (ie. the Apache user has to<br>
have access to the secret keys of all the users - or else you would<br>
see root or the Apache User as the submission user for all the jobs).<br>
<br>
(BTW, Paul Lu was a student of Songnian Zhou, who was the founder of<br>
Platform Computing, and Songnian wrote some of the initial Platform<br>
LSF code when he was at UC Berkeley.)<br>
<br>
However, as I am not a web programming guy, I didn't (and still don't)<br>
know much about web technologies (Yes, I am using Gopher... just<br>
kidding :-D ). Another thing to look at is the S3 multi-part upload,<br>
GridFTP, and multi-session SCP upload methods:<br>
<br>
<a href="http://www.elastician.com/2010/12/s3-multipart-upload-in-boto.html" target="_blank">http://www.elastician.com/2010/12/s3-multipart-upload-in-boto.html</a><br>
<a href="http://www.globus.org/toolkit/docs/latest-stable/gridftp/" target="_blank">http://www.globus.org/toolkit/docs/latest-stable/gridftp/</a><br>
<a href="http://www.psc.edu/index.php/hpn-ssh" target="_blank">http://www.psc.edu/index.php/hpn-ssh</a><br>
<br>
As you might know TCP is too slow to handle large file uploads, so<br>
those technologies really help. I hope the web portal is flexible<br>
enough to use at least those technologies by not hard coding to<br>
specific way of doing the upload.<br>
<br>
If you need changes in Grid Engine to integrate with the portal, then<br>
I can help. Other than that, but then I am not a web guy and thus I am<br>
not sure what's the best technology to use now in 2012 to tie these<br>
things together - and may be in this area you know better than I do!<br>
<br>
Rayson<br>
<br>
================================<br>
Open Grid Scheduler / Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
<br>
Scalable Grid Engine Support Program<br>
<a href="http://www.scalablelogic.com/" target="_blank">http://www.scalablelogic.com/</a><br>
<div><div class="h5"><br>
<br>
<br>
On Wed, May 30, 2012 at 10:17 AM, Sergio Mafra <<a href="mailto:sergiohmafra@gmail.com">sergiohmafra@gmail.com</a>> wrote:<br>
> Hi fellows,<br>
><br>
> One thing that is missing on a Cluster deployment is, in IMHO, a web site<br>
> that could serve as a front side to the final users. This site should has<br>
> some functionality like: Cluster Admin (on/off, cluster health),<br>
> Upload/Download of files, queue status and others.<br>
><br>
> I have a project to build one like this but I´d like to know first if this<br>
> is available in our community?<br>
><br>
> All the best,<br>
><br>
> Sergio<br>
><br>
><br>
</div></div>> _______________________________________________<br>
> StarCluster mailing list<br>
> <a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
> <a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
><br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
--<br>
==================================================<br>
Open Grid Scheduler - The Official Open Source Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
</font></span></blockquote></div><br></div>