[StarCluster] Cannot find a module

[Ya Xue]

Hi,

I have a script that needs to import a module and the module name is given as an input parameter. It works well on a local machine but generates "ImportError" with StarCluster. Here is a simple script that can reproduce the error:

error.py:

def imp(fname):
    if fname[-3:] == '.py':
        fname = fname[:-3]
    exec('import ' + fname + ' as params')
    print params.alpha
    f = open(fname+'_alpha.txt', 'w')
    f.write('%f' % params.alpha)
    f.close()

If I run it on the StarCluster master, no problem:

In [1]: run error.py

In [3]: imp('p1.py')
1

However, on the cluster:

In [4]:  from IPython.parallel import Client

In [5]: c = Client()

In [6]: v = c[:]

In [7]: files = ['p1.py','p2.py','p3.py']

In [8]: r = v.map(imp, files)

In [9]: r[0]
[0:apply]:
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)<string> in <module>()
/home/ya/Code/error.py in imp(fname)
      4     if fname[-3:] == '.py':
      5         fname = fname[:-3]
----> 6     exec('import ' + fname + ' as params')
      7     print params.alpha
      8     f = open(fname+'_alpha.txt', 'w')
<string> in <module>()
ImportError: No module named p1

Does anyone know how to fix it?

Thanks,
Ya

This email (including any attachments) may contain confidential information intended solely for acknowledged recipients. If you think you have received this information in error, please reply to the sender and delete all copies from your system. Please note that unauthorized use, disclosure, or further distribution of this information is prohibited by the sender. Note also that we may monitor email directed to or originating from our network. Thank you for your consideration and assistance. |
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20140818/5ffe393e/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: p3.py
Type: application/octet-stream
Size: 31 bytes
Desc: p3.py
Url : http://mailman.mit.edu/pipermail/starcluster/attachments/20140818/5ffe393e/attachment.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: p2.py
Type: application/octet-stream
Size: 31 bytes
Desc: p2.py
Url : http://mailman.mit.edu/pipermail/starcluster/attachments/20140818/5ffe393e/attachment-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: p1.py
Type: application/octet-stream
Size: 31 bytes
Desc: p1.py
Url : http://mailman.mit.edu/pipermail/starcluster/attachments/20140818/5ffe393e/attachment-0002.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: error.py
Type: application/octet-stream
Size: 246 bytes
Desc: error.py
Url : http://mailman.mit.edu/pipermail/starcluster/attachments/20140818/5ffe393e/attachment-0003.obj