<div dir="ltr">Hi, <div>I just started using star cluster. It's fantastic. </div><div><br></div><div>One issue I am seeing though is that there is some difficulty getting disks up and mounted in a reasonable place. I was wondering if anyone has written a plugin for handling this. I have some work-arounds in terms of a quick and dirty plugin but I don't know how general purpose it would be. </div>
<div><br></div><div>What I would like is the following:</div><div><br></div><div>I want to use EBS volumes for everything except for scratch space as I want to be able to shutdown (to save money) and bring back up just as if it were non-virtual hardware even with spot instances. I want it to survive (no persisted data loss) having my spot instances being seized due to a spot price surge. I believe this is possible. I am wrong? </div>
<div><br></div><div>I would like /home NFS cross-mounted EBS. </div><div><br></div><div>I'd like a "restart" command to find the right EBS volumes and mount them in the usual places. In the case of a fresh "start", I'd like it to create new EBS volumes. I believe the current EBS volume attachment functionality (in config file), assumes that they already exists and that you know their id. </div>
<div><br></div><div>I believe that when you change instance types from say r3.2xlarge to m3.xlarge, the storage devices (/dev/xdvv etc) change names so this requires some knowledge of what the devices will be names for each instance. </div>
<div><br></div><div>Also, I want to avoid over-mounting a device on a directory with files and subdirectories inside. </div><div><br></div><div>I can probably do all of this with a DIY solution but I figured everyone must be dealing with similar issues. Is anyone working on a general plugin for this?</div>
<div><br></div><div>I can imagine a simple plugin (similar to an stab file) that looks like this in the config file</div><div>[volumesetup]</div><div>/home 100GB cross</div><div>/data 100GB cross</div><div>/hdfs1 200GB separate</div>
<div>/hdfs2 200GB separate</div><div><br></div><div>with the remaining local storage being automatically assigned to /scratch1 /scratch2 etc depending on the number of partitions. </div><div><br></div><div>Anyone have any ideas of suggestions?</div>
<div>Dave</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div> </div>
</div>