<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div>
<div>
<div><br>
</div>
<div>Hi Rayson</div>
<div><br>
</div>
<div>
<div>I have written a plugin to run the intel installer from a tgz file on s3. It takes a while but seems to work.</div>
</div>
<div><br>
</div>
<div>I am using one of the stock public images:</div>
<div>[0] ami-3393a45a us-east-1 starcluster-base-ubuntu-13.04-x86_64 (EBS)</div>
<div><br>
</div>
<div>Should I be using the HVM image? I understood they are only needed for GPU computing?</div>
<div><br>
</div>
<div>How can I tell if I have the enhanced networking driver setup correctly? In the paravirtual machine lspci etc show nothing?</div>
<div><br>
</div>
<div>Thanks for the suggestions!</div>
<div><br>
</div>
<div>
<div>
<p class="MsoNormal" style="color: rgb(0, 0, 0); margin-top: 0in; margin-right: 0in; margin-bottom: 0.0001pt; margin-left: 0in; line-height: 17px; font-size: 11pt; font-family: Calibri, sans-serif; ">
<b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; ">David</span></b><b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; "> Stuebe</span></b></p>
<p class="MsoNormal" style="color: rgb(0, 0, 0); margin-top: 0in; margin-right: 0in; margin-bottom: 0.0001pt; margin-left: 0in; line-height: 17px; font-size: 11pt; font-family: Calibri, sans-serif; ">
<b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; ">Scientist & Software Engineer </span></b><span class="Apple-style-span" style="line-height: normal; "><b><span lang="EN-GB" style="font-size: 8pt; color: navy; font-family: Arial, sans-serif; ">–
RPS ASA</span></b></span><b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; "><br>
<br>
</span></b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; ">55 Village Square Drive <br>
South Kingstown, RI 02879-8248<br>
<br>
</span><b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; ">Tel: </span></b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; ">+1 (401) 789-6224</span></p>
<p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-bottom: 10pt; margin-left: 0in; ">
<span lang="EN-GB"><b style="color: rgb(0, 0, 0); font-family: Arial, sans-serif; font-size: 8pt; line-height: 12px; "><span style="color: navy; ">Email: </span></b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; "><a href="mailto:David.Stuebe@rpsgroup.com" style="color: blue; text-decoration: underline; ">David.Stuebe@rpsgroup.com</a></span></span><span style="color: rgb(0, 0, 0); font-size: 12pt; line-height: 18px; font-family: 'Times New Roman', serif; "><br>
</span><b style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 11pt; line-height: 17px; "><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; ">www:</span></b><span lang="EN-GB" style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; "> <a href="http://www.asascience.com/" style="color: blue; text-decoration: underline; ">asascience.com</a> | <a href="http://www.rpsgroup.com/" style="color: blue; text-decoration: underline; ">rpsgroup.com</a></span><span style="color: rgb(0, 0, 0); font-size: 12pt; line-height: 18px; font-family: 'Times New Roman', serif; "><br>
<br>
</span><i style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 11pt; line-height: 17px; "><span style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; "><o:p></o:p></span></i></p>
<p class="MsoNormal" style="color: rgb(0, 0, 0); margin-top: 0in; margin-right: 0in; margin-bottom: 10pt; margin-left: 0in; line-height: 17px; font-size: 11pt; font-family: Calibri, sans-serif; ">
<i><span style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; ">A member of the RPS Group plc</span></i></p>
</div>
</div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Rayson Ho <<a href="mailto:raysonlogin@gmail.com">raysonlogin@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Fri, 16 May 2014 09:13:03 -0400<br>
<span style="font-weight:bold">To: </span>David Stuebe <<a href="mailto:dstuebe@asascience.com">dstuebe@asascience.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:starcluster@mit.edu">starcluster@mit.edu</a>" <<a href="mailto:starcluster@mit.edu">starcluster@mit.edu</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [StarCluster] MPICH Fabric<br>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div dir="ltr">
<div>How are you deploying the Intel Cluster Compiler Suite? If you are using a custom AMI, then make sure that you have the AWS enhanced networking NIC driver setup correctly, and also make sure that your instances are all in a placement group, and in a VPC
(those should be set by StarCluster if you are using the latest stable version).<br>
<br>
</div>
We benchmarked AWS enhanced networking on the C3 family a few months ago, and the latency is around 20% better on a pair of C3.8xlarge instances in a placement group with AWS enhanced networking enabled:<br>
<br>
<a href="http://blogs.scalablelogic.com/2014/01/enhanced-networking-in-aws-cloud-part-2.html">http://blogs.scalablelogic.com/2014/01/enhanced-networking-in-aws-cloud-part-2.html</a><br>
<div>
<div>
<div class="gmail_extra"><br clear="all">
<div>Rayson<br>
<br>
==================================================<br>
Open Grid Scheduler - The Official Open Source Grid Engine<br>
<a href="http://gridscheduler.sourceforge.net/" target="_blank">http://gridscheduler.sourceforge.net/</a><br>
<a href="http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html" target="_blank">http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html</a></div>
<br>
<br>
<div class="gmail_quote">On Thu, May 15, 2014 at 3:18 PM, David Stuebe <span dir="ltr">
<<a href="mailto:DStuebe@asascience.com" target="_blank">DStuebe@asascience.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>
<div>
<div><br>
</div>
<div>Hi Starcluster</div>
<div><br>
</div>
<div>Do anyone have advice on what fabric to use when running on AWS?</div>
<div><br>
</div>
<div>I know the interconnect is supposed to be 10-gigE but my model is more dependent on latency than throughput.</div>
<div><br>
</div>
<div>I have had to use the Intel Cluster Compiler Suite rather than the built in OpenMPI. Hoping to resolve those issues and compare the two – I am interested to see the performance differences…</div>
<div><br>
</div>
<div>Currently the model actually has a negative performance curve as I add processors past a single node.</div>
<div><br>
</div>
<div>
<div>Model performance on running on Amazon…</div>
<div>C3.8Xlarge - 1 instance, 32 cores</div>
<div> ! IINT <span style="white-space:pre-wrap"></span>SIMTIME(UTC) <span style="white-space:pre-wrap"></span>FINISH IN
<span style="white-space:pre-wrap"></span>SECS/IT PERCENT COMPLETE </div>
<div>!8396282 2014-03-18T00:18:02.000000 0000:07:54:19 0.1103 | |</div>
<div><br>
</div>
<div>C3.8Xlarge - 2 instance, 64 cores</div>
<div> !8395221 2014-03-18T00:00:21.000000 0000:20:27:58 0.2843 | |</div>
<div><br>
</div>
<div>C3.8Xlarge - 3 instance, 96 cores</div>
<div>!8395273 2014-03-18T00:01:13.000000 0007:18:33:19 2.5918 | |</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>
<p class="MsoNormal" style="color:rgb(0,0,0);margin:0in 0in 0.0001pt;line-height:17px;font-size:11pt;font-family:Calibri,sans-serif">
<b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; " lang="EN-GB">David</span></b><b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; " lang="EN-GB"> Stuebe</span></b></p>
<p class="MsoNormal" style="color:rgb(0,0,0);margin:0in 0in 0.0001pt;line-height:17px;font-size:11pt;font-family:Calibri,sans-serif">
<b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; " lang="EN-GB">Scientist & Software Engineer </span></b><span style="line-height:normal"><b><span style="font-size: 8pt; color: navy; font-family: Arial, sans-serif; " lang="EN-GB">–
RPS ASA</span></b></span><b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; " lang="EN-GB"><br>
<br>
</span></b><span style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; " lang="EN-GB">55 Village Square Drive <br>
South Kingstown, RI 02879-8248<br>
<br>
</span><b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; " lang="EN-GB">Tel: </span></b><span style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; " lang="EN-GB"><a href="tel:%2B1%20%28401%29%20789-6224" value="+14017896224" target="_blank">+1
(401) 789-6224</a></span></p>
<p class="MsoNormal" style="margin:0in 0in 10pt"><span lang="EN-GB"><b style="color:rgb(0,0,0);font-family:Arial,sans-serif;font-size:8pt;line-height:12px"><span style="color:navy">Email: </span></b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; "><a href="mailto:David.Stuebe@rpsgroup.com" style="color:blue;text-decoration:underline" target="_blank">David.Stuebe@rpsgroup.com</a></span></span><span style="color: rgb(0, 0, 0); font-size: 12pt; line-height: 18px; font-family: 'Times New Roman', serif; "><br>
</span><b style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:11pt;line-height:17px"><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; " lang="EN-GB">www:</span></b><span style="font-size: 8pt; line-height: 12px; color: navy; font-family: Arial, sans-serif; " lang="EN-GB"> <a href="http://www.asascience.com/" style="color:blue;text-decoration:underline" target="_blank">asascience.com</a> | <a href="http://www.rpsgroup.com/" style="color:blue;text-decoration:underline" target="_blank">rpsgroup.com</a></span><span style="color: rgb(0, 0, 0); font-size: 12pt; line-height: 18px; font-family: 'Times New Roman', serif; "><br>
<br>
</span><i style="color:rgb(0,0,0);font-family:Calibri,sans-serif;font-size:11pt;line-height:17px"><span style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; "><u></u><u></u></span></i></p>
<p class="MsoNormal" style="color:rgb(0,0,0);margin:0in 0in 10pt;line-height:17px;font-size:11pt;font-family:Calibri,sans-serif">
<i><span style="font-size: 8pt; line-height: 12px; color: black; font-family: Arial, sans-serif; ">A member of the RPS Group plc</span></i></p>
</div>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</span>
</body>
</html>