<div dir="ltr"><br><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Sergio Mafra</b> <span dir="ltr"><<a href="mailto:sergiohmafra@gmail.com">sergiohmafra@gmail.com</a>></span><br>
Date: Mon, Aug 19, 2013 at 3:47 PM<br>Subject: Re: [StarCluster] Fwd: Integration of MPICH2 plugin with SGE<br>To: Hyokun Yun <<a href="mailto:yun3@purdue.edu">yun3@purdue.edu</a>><br><br><br><div dir="ltr">Hi Hyokun,<div>
<br></div><div>Here we go:</div><div><br></div><div>1. <span style="font-size:13px;font-family:arial,sans-serif">This indicates, that you application tries to use a node in the cluster, which wasn't granted to this job by OGE.</span></div>
<div><span style="font-size:13px;font-family:arial,sans-serif">2. OGE works well but I guess that this is more for OpenMPI (the default MPI of StarCluster)... Which version of MPICH2 are you using? Is it the last one.. 1.4? Did you compile your app using this version?</span></div>
<div>3. MIT StarCluster has changed the default allocation strategy from $round_robin to $fill_up on this last release.</div><div>4. The only thing that can be related to the AMI is the Mpich2 version. </div><div><br></div>
<div>All best,</div><div><br></div><div>Sergio</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Aug 19, 2013 at 3:10 PM, Hyokun Yun <span dir="ltr"><<a href="mailto:yun3@purdue.edu" target="_blank">yun3@purdue.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Sergio,<br><br><br>Thanks for the advice! <br><br>I have read the document, but why would daemons reject the task if it is configured $fill_up?<br>
</div>Shouldn't OGE work for both choices? The document doesn't say I should not use $fill_up.<br>
<div class="gmail_extra"><br></div><div class="gmail_extra">I think I gave $round_robin a try, but I will try once again and let you know whether I had success.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">
Also, is it possible that this is a problem specific to the AMI I am using?<br></div><div class="gmail_extra"><br><br></div><div class="gmail_extra">Best,<br>Hyokun Yun<br></div><div class="gmail_extra"><br><br></div><div class="gmail_extra">
<div><div>
<br><div class="gmail_quote">On Mon, Aug 19, 2013 at 10:38 AM, Sergio Mafra <span dir="ltr"><<a href="mailto:sergiohmafra@gmail.com" target="_blank">sergiohmafra@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><div dir="ltr">Hi Hyokun,<br><div class="gmail_quote"><div dir="ltr"><div><br></div><div>I´m a user of MPICH2 and OGE.</div><div><br></div><div>It seems that you´re using $fill_up instead of $round_robin. If so, try to change it to $round_robin with <span style="background-color:rgb(250,250,250);color:rgb(62,67,73);font-family:Consolas,'andale mono','lucida console',monospace;font-size:14px;line-height:17px">$ qconf -mp orte</span></div>
<div>You can learn more here: <a href="http://star.mit.edu/cluster/docs/latest/plugins/sge.html#using-the-plugin" target="_blank">http://star.mit.edu/cluster/docs/latest/plugins/sge.html#using-the-plugin</a></div><div><br>
</div><div>Let me know if this help you.</div>
<div><br></div><div>All best.</div><div><br></div><div>Sergio</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Mon, Aug 19, 2013 at 1:53 AM, Hyokun Yun <span dir="ltr"><<a href="mailto:yun3@purdue.edu" target="_blank">yun3@purdue.edu</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><div>Dear starcluster users,</div><div><br></div><div><br></div><div>I am experiencing a problem using MPICH2 plugin with SGE.</div>
<div><br></div><div>I am using the following image: ami-52a0c53b which uses Ubuntu 12.04</div>
<div><br></div><div>When I use mpich2 plugin, it seems like mpich2 and SGE are not tightly integrated: when I execute my script using qsub, I get the following error message.</div><div><br></div><div>error: executing task of job 1 failed: execution daemon on host "node001" didn't accept task</div>
<div>error: executing task of job 1 failed: execution daemon on host "node002" didn't accept task</div><div>error: executing task of job 1 failed: execution daemon on host "node003" didn't accept task</div>
<div>error: executing task of job 1 failed: execution daemon on host "nodef004" didn't accept task</div><div><br></div><div>It runs fine when I simply execute 'mpirun' myself, instead of relying on SGE.</div>
<div>Also, the same script runs fine as well when I use OpenMPI instead of MPICH2. That's why I suspect it is MPICH2 & SGE integration issue.</div><div><br></div><div>The problem is that I need multi-thread support, and it is by default disabled in OpenMPI. I also prefer to use MPICH2 instead of OpenMPI.</div>
<div><br></div><div>I was able to reproduce the problem when I restarted the cluster from scratch. Would any of you please take a look on the problem by trying the same image with MPICH2 plugin?</div><div><br></div><div>
<br></div><div>Thanks,</div><div>Hyokun Yun</div>
</div>
<br></div></div>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br></div>
</div><br></div>
</div></div><br>_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu" target="_blank">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br></blockquote></div><br><br clear="all"><br></div></div><span><font color="#888888">-- <br><b>Hyokun Yun </b>( <a href="http://www.stat.purdue.edu/~yun3" target="_blank">http://www.stat.purdue.edu/~yun3</a> )<div>
<div>Ph.D Candidate</div><div>Department of Statistics</div>
<div>Purdue University</div></div><div><br></div>
</font></span></div></div>
</blockquote></div><br></div>
</div></div></div><br></div>