Hi Manal,<div><br></div><div>I did tried a similar thing last week and experienced a similar failure to mount an EBS disk via StarCluster.</div><div><br></div><div>If you take a look at the log file, you will see what had happened. In my case, the debug log was located at ~/.starcluster/logs/debug.log </div>
<div><br></div><div>It appears that when StarCluster attached an EBS volume, the actual partition name it got was different than what StarCluster expected to mount.</div><div><br></div><div>For example, if you look at /proc/partitions table, you will see the actual partition.</div>
<div>Then, take a look at the debug.log and see what device StarCluster was trying to mount.</div><div><br></div><div>I think this issue happened if you manually tried to mount an EBS volume several times and then, switched to StarCluster to make it automated.</div>
<div>I haven't tried but if you start it from a fresh image, it would work.</div><div><br></div><div>Regards,</div><div>- Chansup<br><br><div class="gmail_quote">On Mon, May 21, 2012 at 3:44 PM, Ron Chen <span dir="ltr"><<a href="mailto:ron_chen_123@yahoo.com" target="_blank">ron_chen_123@yahoo.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Manal,<br>
<br>
Note: I am only a developer of Open Grid Scheduler, the open source Grid Engine. I am not exactly a EC2 developer yet, and may be there are better ways to do it in StarCluster.<br>
<br>
Did you format your EBS? Like a new harddrive, you need to fdisk & format it before you can use it.<br>
<br>
- So first, logon to the EC2 Management Console. Then go to your EBS Volumes.<br>
<br>
- Then check the state, if it is in-use then it is already attached to an instance. If it is available, then StarCluster has not attached it yet.<br>
<br>
- After you are sure it is attached, the Attachment section should show something similar to the following:<br>
<br>
Attachment: i-39586e5f (master):/dev/sdf1 (attached)<br>
<br>
<br>
<br>
And now you need to partition the disk.<br>
<br>
- If you see /dev/sdf1 above, you need to partition /dev/xvdf as the AMIs have the xvd drivers:<br>
<br>
# fdisk /dev/xvdf<br>
<br>
<br>
Then you can format the disk using mkfs.<br>
<br>
# mkfs -t ext4 /dev/xvdf1<br>
<br>
<br>
So finally, you can mount the disk, and if you specify the volume in the StarCluster config correctly, then it will be mounted next time you boot StarCluster.<br>
<br>
-Ron<br>
<br>
<br>
<br>
<br>
<br>
________________________________<br>
From: Manal Helal <<a href="mailto:manal.helal@gmail.com">manal.helal@gmail.com</a>><br>
To: Justin Riley <<a href="mailto:jtriley@mit.edu">jtriley@mit.edu</a>><br>
Cc: <a href="mailto:starcluster@mit.edu">starcluster@mit.edu</a><br>
Sent: Monday, May 21, 2012 7:41 AM<br>
Subject: Re: [StarCluster] newbie problems<br>
<br>
<br>
Hello, <br>
I hate being a headache, but this didn't go smooth as I was hoping, and I appreciate your support to get moving, <br>
<br>
I finally successfully attached the volume I created, but didn't see where it should be on the cluster, and how my data will be saved from session to session, <br>
<br>
The volume I created is a 30 GB, I first mounted it to /mydata, and didn't see this when I started the cluster, this is what I get:<br>
<br>
root@ip-10-16-3-102:/dev# fdisk -l<br>
<br>
Disk /dev/sda: 8589 MB, <a href="tel:8589934592" value="+18589934592">8589934592</a> bytes<br>
255 heads, 63 sectors/track, 1044 cylinders, total 16777216 sectors<br>
Units = sectors of 1 * 512 = 512 bytes<br>
Sector size (logical/physical): 512 bytes / 512 bytes<br>
I/O size (minimum/optimal): 512 bytes / 512 bytes<br>
Disk identifier: 0x00000000<br>
<br>
Device Boot Start End Blocks Id System<br>
/dev/sda1 * 16065 16771859 8377897+ 83 Linux<br>
<br>
Disk /dev/xvdb: 901.9 GB, 901875499008 bytes<br>
255 heads, 63 sectors/track, 109646 cylinders, total 1761475584 sectors<br>
Units = sectors of 1 * 512 = 512 bytes<br>
Sector size (logical/physical): 512 bytes / 512 bytes<br>
I/O size (minimum/optimal): 512 bytes / 512 bytes<br>
Disk identifier: 0x00000000<br>
<br>
Disk /dev/xvdb doesn't contain a valid partition table<br>
<br>
Disk /dev/xvdc: 901.9 GB, 901875499008 bytes<br>
255 heads, 63 sectors/track, 109646 cylinders, total 1761475584 sectors<br>
Units = sectors of 1 * 512 = 512 bytes<br>
Sector size (logical/physical): 512 bytes / 512 bytes<br>
I/O size (minimum/optimal): 512 bytes / 512 bytes<br>
Disk identifier: 0x00000000<br>
<br>
Disk /dev/xvdc doesn't contain a valid partition table<br>
<br>
root@ip-10-16-3-102:/dev# df -h<br>
Filesystem Size Used Avail Use% Mounted on<br>
/dev/sda1 7.9G 5.1G 2.5G 68% /<br>
udev 12G 4.0K 12G 1% /dev<br>
tmpfs 4.5G 216K 4.5G 1% /run<br>
none 5.0M 0 5.0M 0% /run/lock<br>
none 12G 0 12G 0% /run/shm<br>
/dev/xvdb 827G 201M 785G 1% /mnt<br>
<br>
<br>
<br>
no 30GB volume attached, then I terminated and followed the suggestions in this page:<br>
<br>
<a href="http://web.mit.edu/star/cluster/docs/latest/manual/configuration.html" target="_blank">http://web.mit.edu/star/cluster/docs/latest/manual/configuration.html</a><br>
<br>
making it mount to /home thinking it will be used in place of the /home folder, and this way all my installations and downloads will be saved after I terminate the session, <br>
<br>
however, when I started the cluster this is what I get:<br>
<br>
root@ip-10-16-24-98:/home# df -h<br>
Filesystem Size Used Avail Use% Mounted on<br>
/dev/sda1 7.9G 5.1G 2.5G 68% /<br>
udev 12G 4.0K 12G 1% /dev<br>
tmpfs 4.5G 216K 4.5G 1% /run<br>
none 5.0M 0 5.0M 0% /run/lock<br>
none 12G 0 12G 0% /run/shm<br>
/dev/xvdb 827G 201M 785G 1% /mnt<br>
<br>
<br>
There is no 30 GB volume as well, and neither / nor /mnt are getting bigger, <br>
<br>
here is what I am having in my config file:<br>
<br>
[cluster mycluster]<br>
VOLUMES = mydata<br>
<br>
<br>
[volume mydata]<br>
# attach vol-c9999999 to /home on master node and NFS-shre to worker nodes<br>
VOLUME_ID = vol-c9999999 #(used the volume ID I got from the AWS console)<br>
MOUNT_PATH = /home #(not sure if this is true or not, I used /mydata in the first run and didn't work as well)<br>
<br>
also when I was running before attaching the volume, I had starcluster put and starcluster get commands working very well. After attaching the volume, I had them working and saying 100% complete on my local machine, but when I log in to the cluster, I find the paths where I was uploading the files to, empty, no files went through! I am not sure if this is related to attaching the volume and whether there should be anything I need to do <br>
P.S. I noticed in the ec2 command line tools to attach a volume to an instance, I should define the volume ID, the instance ID and the device ID (/dev/sdf), same as found in the aws online console. However, the mount path in the starcluster configuration file, doesn't seem to be a device ID that should have been (/dev/sdf) for linux as far as I understand. Not sure where to define this in starcluster if this is the missing point, <br>
<br>
I appreciate your help very much, <br>
<br>
thanks again, <br>
<br>
Manal<br>
<br>
<br>
_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
<br>
_______________________________________________<br>
StarCluster mailing list<br>
<a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a><br>
<a href="http://mailman.mit.edu/mailman/listinfo/starcluster" target="_blank">http://mailman.mit.edu/mailman/listinfo/starcluster</a><br>
</blockquote></div><br></div>