[StarCluster] IPython parallel DirectView not distributing to starcluster nodes.

MinRK benjaminrk at gmail.com
Wed Jul 1 14:14:54 EDT 2015


Can you perhaps share a code sample? Have you verified that all the engines
are registered with the Client (`Client.ids`) before submitting the tasks?

-MinRK

On Wed, Jul 1, 2015 at 9:09 AM, Austin So <austin.so at tomabio.com> wrote:

> I’ve been trying to figure out what I’m doing wrong here, and if it is an
> issue within the starcluster config file. I’ve exhausted all possible
> implementations in my code that I could think of.
>
> During set up, 255 engines have been recorded to have been set-up by
> starcluster upon launch that are available to IPcluster.
>
> Within IPython Notebooks, I’m trying to distribute a function across all
> my nodes and engines (5 at r3.8xlarge).
>
> So when the line of code is running, I’m looking at qhost, and I see that
> only the Master is showing a CPU load. I look at the Cloud Metrics, and I
> see that only the Master is showing a CPU load. At the suggestion of a
> friend, I returned a socket.fqdn() call to identify if the results were
> processed by the master or one of the nodes. All results returned were
> generated by the Master.
>
> Any hints to identify where the source of the problem lies would be
> greatly appreciated.
>
> Best
>
> Austin
>
>
>
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster at mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20150701/5f7bea0f/attachment.htm


More information about the StarCluster mailing list