[StarCluster] AWS Enhanced Networking

[Sergio Mafra]

Nice to know that.
In my case, I just want to take advantage of Enhanced Networking since my
jobs demands that.
In the near future, when you think your AMI is ready and if you want to
share... just tell us.

All best,

Sergio


On Thu, Jan 9, 2014 at 11:25 PM, Rayson Ho <raysonlogin at gmail.com> wrote:

> As far as I know, not yet. I just created my own AMI using
> non-standard procedure...
>
> Both the Enhanced Networking feature & the new GPU instance family
> require new drivers / AMIs. Thus if one wants to take full advantage
> of the new features in the c3, i2, and g2 instance families, a new set
> of AMIs (with the Intel or NVidia drivers installed) need to be
> created.
>
> I just came back from my vacation a few days ago, and I will need to
> get back on the SC IRC and see what remaining work needs to be done.
>
> Rayson
>
> ==================================================
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>
>
> On Thu, Jan 9, 2014 at 7:59 PM, Sergio Mafra <sergiohmafra at gmail.com>
> wrote:
> > Hi Rayson,
> >
> > Is there any AMI with the Intel ixgbevf Virtual Function drive ready for
> > StarCluster?
> > I think that this should be part of StarCluster´s public AMIs.
> >
> > All best,
> >
> > Sergio
> >
> >
> > On Wed, Jan 8, 2014 at 3:52 PM, Rayson Ho <raysonlogin at gmail.com> wrote:
> >>
> >> The Enhanced Networking feature provides much better latency for all
> >> instance types that support it. For the c3.8xlarge instance type,
> >> Enhanced Networking allows the instance to utilize over 95% of the
> >> 10Gbit Ethernet bandwidth:
> >>
> >>
> >>
> http://blogs.scalablelogic.com/2013/12/enhanced-networking-in-aws-cloud.html
> >>
> >>
> http://blogs.scalablelogic.com/2014/01/enhanced-networking-in-aws-cloud-part-2.html
> >>
> >> Rayson
> >>
> >> ==================================================
> >> Open Grid Scheduler - The Official Open Source Grid Engine
> >> http://gridscheduler.sourceforge.net/
> >> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
> >>
> >>
> >>
> >> On Mon, Nov 25, 2013 at 12:42 PM, Rayson Ho <raysonlogin at gmail.com>
> wrote:
> >> > Thanks Sergio, I check the docs the day when c3 was announced, there's
> >> > no mention of placement group support for c3. I think I was reading
> >> > the older version of the doc.
> >> >
> >> > Yuichi, we can't take advantage of SR-IOV (yet) as we don't run in a
> >> > VPC.
> >> >
> >> > Rayson
> >> >
> >> > ==================================================
> >> > Open Grid Scheduler - The Official Open Source Grid Engine
> >> > http://gridscheduler.sourceforge.net/
> >> > http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
> >> >
> >> >
> >> > On Mon, Nov 25, 2013 at 9:39 AM, Yoshiara, Yuichi
> >> > <yoshiara at amazon.co.jp> wrote:
> >> >>
> >> >> Hi Sergio,
> >> >>
> >> >>
> >> >>
> >> >> There is a new feature called Enhanced Networking which is available
> in
> >> >> C3 instances.
> >> >>
> >> >> Please read through the link below and see if it helps you.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> http://aws.amazon.com/ec2/faqs/#What_networking_capabilities_are_included_in_this_feature
> >> >>
> >> >>
> >> >>
> >> >> Yuichi Yoshiara
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> From: starcluster-bounces at mit.edu [mailto:
> starcluster-bounces at mit.edu]
> >> >> On Behalf Of Sergio Mafra
> >> >> Sent: Monday, November 25, 2013 8:16 PM
> >> >> To: Rayson Ho; starcluster at mit.edu
> >> >> Subject: Re: [StarCluster] MIT StarCluster and the new C3 instances
> >> >>
> >> >>
> >> >>
> >> >> Hi Rayson,
> >> >>
> >> >>
> >> >>
> >> >> I noticed that StarCluster didn´t allocate a placement group for C3
> >> >> instances and did it by myself in the code.
> >> >>
> >> >> But this was not helpfull since C3 instances uses high throughput
> >> >> insted of the 10 giga network.
> >> >>
> >> >> In my tests, using C3 or CC2 in a cluster config where MPI is
> required
> >> >> will be equal to same processing time.
> >> >>
> >> >> Hope that AWS put them on 10 Giga net in the near future.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> Compute optimized
> >> >>
> >> >> c3.8xlarge
> >> >>
> >> >> 64-bit
> >> >>
> >> >> Intel Xeon E5-2680
> >> >>
> >> >> 32
> >> >>
> >> >> 108
> >> >>
> >> >> 60.00
> >> >>
> >> >> 640 GB (2 x 320 SSD)
> >> >>
> >> >> -
> >> >>
> >> >> 240
> >> >>
> >> >> High
> >> >>
> >> >> Compute optimized
> >> >>
> >> >> cc2.8xlarge
> >> >>
> >> >> 64-bit
> >> >>
> >> >> Intel Xeon E5-2670
> >> >>
> >> >> 32
> >> >>
> >> >> 88
> >> >>
> >> >> 60.50
> >> >>
> >> >> 3370 GB (4 x 840)
> >> >>
> >> >> -
> >> >>
> >> >> 240
> >> >>
> >> >> 10 Gigabit
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Mon, Nov 18, 2013 at 7:13 PM, Sergio Mafra <
> sergiohmafra at gmail.com>
> >> >> wrote:
> >> >>
> >> >> Hi Rayson,
> >> >>
> >> >>
> >> >>
> >> >> I´ll try to do that.
> >> >>
> >> >>
> >> >>
> >> >> All best,
> >> >>
> >> >>
> >> >>
> >> >> Sergio
> >> >>
> >> >>
> >> >>
> >> >> On Mon, Nov 18, 2013 at 1:27 PM, Rayson Ho <raysonlogin at gmail.com>
> >> >> wrote:
> >> >>
> >> >> It's in the development branch:
> >> >>
> >> >> https://github.com/jtriley/StarCluster/pull/325
> >> >>
> >> >> It's a few lines of code, and you can merge the changes into your
> >> >> local version very easily.
> >> >>
> >> >> Rayson
> >> >>
> >> >> ==================================================
> >> >> Open Grid Scheduler - The Official Open Source Grid Engine
> >> >> http://gridscheduler.sourceforge.net/
> >> >> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
> >> >>
> >> >>
> >> >>
> >> >> On Mon, Nov 18, 2013 at 8:29 AM, Sergio Mafra <
> sergiohmafra at gmail.com>
> >> >> wrote:
> >> >> > Hi everyone,
> >> >> >
> >> >> > I attended AWS Re:Invent this year and liked a lot. Right place to
> >> >> > go.
> >> >> > AWS announced there the new C3 family of HPC instances.
> >> >> > I´m anxious to test them in my models but I understand that
> >> >> > StarCluster must
> >> >> > take aware of them in the code.
> >> >> > So when will be the C3s available?
> >> >> >
> >> >> > All the best,
> >> >> >
> >> >> > Sergio Mafra
> >> >> >
> >> >>
> >> >> > _______________________________________________
> >> >> > StarCluster mailing list
> >> >> > StarCluster at mit.edu
> >> >> > http://mailman.mit.edu/mailman/listinfo/starcluster
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >>
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20140110/71db9d03/attachment.htm