[StarCluster] 100 nodes cluster

[Paolo Di Tommaso]

Hi All, 

I've tried to setup a cluster with 100 nodes with quite powerful machines (Hi-Mem double extra large configuration) but it ended in a total failure. 

The overall configuration process was extremely slow. Five instances blocked in pending state for more than 10 minutes so I had to terminate them manually . 

Also other machines returns some error codes, for example mounting the /home and other SGE components. 

I had to stop the initialization phase manually after more than 30 minutes, because it seem to hung. 


I'm not blaming about StarCluster, it is really a nice piece of software. The problem really seems to be the Amazon infrastructure that has lot of latencies and unreliable behaviors. 


What is your opinion about that? Is there anyone running successfully a "big" cluster using the StarCluster tool? 




Thank you, 

Paolo Di Tommaso
Software Engineer 
Comparative Bioinformatics Group
Centre de Regulacio Genomica (CRG)
Dr. Aiguader, 88
08003 Barcelona, Spain