[StarCluster] Cannot find a module

[Work]

Hi Ya,

When you execute it, do check that you are using the home directory since this is the only one shared by all modes and master by default. 

Also, I notice you use iron python. To simplify things, can you run using sshmaster and sshnode. This may give you additional indication on what is going on. 

I am unsure of a solution, yet these seem reasonable steps to check. 

I hope this helps narrow down the issue. 

              Jacob



Sent from my iPhone

On Aug 18, 2014, at 2:49 PM, Ya Xue <yxue at measinc.com> wrote:

> Hi,
>  
> I have a script that needs to import a module and the module name is given as an input parameter. It works well on a local machine but generates “ImportError” with StarCluster. Here is a simple script that can reproduce the error:
>  
> error.py:
>  
> def imp(fname):
>     if fname[-3:] == '.py':
>         fname = fname[:-3]
>     exec('import ' + fname + ' as params')
>     print params.alpha
>     f = open(fname+'_alpha.txt', 'w')
>     f.write('%f' % params.alpha)
>     f.close()
>  
> If I run it on the StarCluster master, no problem:
>  
> In [1]: run error.py
>  
> In [3]: imp('p1.py')
> 1
>  
> However, on the cluster:
>  
> In [4]:  from IPython.parallel import Client
>  
> In [5]: c = Client()
>  
> In [6]: v = c[:]
>  
> In [7]: files = ['p1.py','p2.py','p3.py']
>  
> In [8]: r = v.map(imp, files)
>  
> In [9]: r[0]
> [0:apply]:
> ---------------------------------------------------------------------------
> ImportError                               Traceback (most recent call last)<string> in <module>()
> /home/ya/Code/error.py in imp(fname)
>       4     if fname[-3:] == '.py':
>       5         fname = fname[:-3]
> ----> 6     exec('import ' + fname + ' as params')
>       7     print params.alpha
>       8     f = open(fname+'_alpha.txt', 'w')
> <string> in <module>()
> ImportError: No module named p1
>  
> Does anyone know how to fix it?
>  
> Thanks,
> Ya
> 
> This email (including any attachments) may contain confidential information intended solely for acknowledged recipients. If you think you have received this information in error, please reply to the sender and delete  all copies from your system. Please note that unauthorized use, disclosure, or further distribution of this information is prohibited by the sender. Note also that we may monitor email directed to or originating from our network. Thank you for your consideration and assistance. |
> <p3.py>
> <p2.py>
> <p1.py>
> <error.py>
> _______________________________________________
> StarCluster mailing list
> StarCluster at mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20140819/8b88b2f2/attachment.htm