[StarCluster] IPython parallel DirectView not distributing to starcluster nodes.

Austin So austin.so at tomabio.com
Sat Jul 4 18:00:01 EDT 2015


No error messages. I assumed it was that file since it was a config file, but what do I know except things get buggered up.

If you or anyone for that matter is actually interested in replicating, then these are the steps:

1. create 5-node cluster via starcluster using the HVM AMI.
2. login via IP[y] Notebook.
3. open and execute the previously attached notebook without setting the number of engines within IP[y]
	- determine the number of unique domain names being utilized
4. close/halt and set the number of engines in IP[y] to whatever you feel like
5. reopen and execute the previously attached notebook.
	- determine the number of unique domain names being utilized.

If you do not honestly see a difference in the domain name utilization, let me know. Or if you think that this is a spurious concern that doesn’t alter performance of the cluster, then I’d be interested in knowing because it is clear that by not assigning engines in IP[y], that I get utilization across all nodes.

A.



> On Jul 4, 2015, at 2:23 PM, MinRK <benjaminrk at gmail.com> wrote:
> 
> Setting the engines in the notebook definitely does not rewrite the ipcluster_config file. What error do you see?
> 
> On Sat, Jul 4, 2015 at 12:39 PM, Austin So <austin.so at tomabio.com <mailto:austin.so at tomabio.com>> wrote:
> Note to self: do not set engines on IP[y] notebook as this rewrites the ipcluster_config and apparently messes everything up.
> 
> A.
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20150704/8d0d88b9/attachment-0001.htm


More information about the StarCluster mailing list