<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>Very nice. I'll have to try this out this week when I'm back at work. Thanks for putting the effort in!<br><br>Sent from my iPhone</div><div><br>On May 25, 2015, at 10:42 AM, Sergio Mafra <<a href="mailto:sergiohmafra@gmail.com">sergiohmafra@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr"><div><div>Just forgot to add that you need to open port 80 in StarCluster security groups in order to allow to access Ganglia via web.<br></div>So, in config file you need to add:<br><br>[permission www]<br># open port 80 to the world<br>from_port = 80<br>to_port = 80<br><br></div>and in the [Cluster ....]<br>permissions = www<br><div><br><br></div><div>Good luck to all,<br><br></div><div>Sergio<br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, May 25, 2015 at 11:14 AM, Sergio Mafra <span dir="ltr"><<a href="mailto:sergiohmafra@gmail.com" target="_blank">sergiohmafra@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div>Hi SC_Folks,<br><br></div>I´ve made a little research and found interesting things that helped to build a plugin of configuring Ganglia to run with StarCluster.<br></div>Here comes what I did in case if you´re interested:<br><br></div>1. First, I used the Shared StarCluster AMI Ubuntu 14.04. It can be found on Public AMIs. (us-east-1 -> ami-38b99850).<br></div>2. This tutorial was great: <a href="https://www.digitalocean.com/community/tutorials/introduction-to-ganglia-on-ubuntu-14-04" target="_blank">https://www.digitalocean.com/community/tutorials/introduction-to-ganglia-on-ubuntu-14-04</a><br></div>3. I´ve installed these apps in base AMI using<code>: <span style="font-family:arial,helvetica,sans-serif">sudo apt-get install -y ganglia-monitor rrdtool gmetad ganglia-webfrontend</span><br></code></div><code><span style="font-family:arial,helvetica,sans-serif">4. Ganglia needs to be configured using two config files in /etc/config: gmetad.conf and gmond.conf<br></span></code></div><code><span style="font-family:arial,helvetica,sans-serif">5. The gmetad.conf needs to have the name of the cluster you intend to monitor. I used a fixed name (StarCluster) but you can chance in the future with the name tag of the cluster you´re provisioning, So this is what I´ve chnaged:<br></span></code><code><span style="font-family:arial,helvetica,sans-serif">data_souce "StarCluster" 60 localhost<br></span></code></div><div><code><span style="font-family:arial,helvetica,sans-serif">6. The gmond.conf needs to changed in 3 sections:<br></span></code></div><div><code><span style="font-family:arial,helvetica,sans-serif"> a) Cluster section: Note that I´ve change the unspecified to StarCluster (same name in the gmetad.conf)<br> cluster {<br> name = "StarCluster"<br> owner = "unspecified"<br> latlong = "unspecified"<br> url = "unspecified"<br> }<br><br></span></code></div><div><code><span style="font-family:arial,helvetica,sans-serif">b) udp_send_channel section: You have to comment the mcast_join line and add the ip of the cluster master. The plugin will handle this. So I add the line "host = master_ip". This will be the same if the node is master or slave.<br>udp_send_channel {<br># mcast_join = 239.2.11.71<br> host = master_ip<br> port = 8649<br> ttl = 1<br>}<br><br></span></code></div><div><code><span style="font-family:arial,helvetica,sans-serif">c) udp_recv_channel: You will have to have two kinds of configuration regarding the master and the slaves. In master, the section will be the following: The mcast_join line commented and bind line too.<br></span></code></div><div><code><span style="font-family:arial,helvetica,sans-serif"><br>> Master node:<br></span></code></div><div><code><span style="font-family:arial,helvetica,sans-serif">udp_recv_channel {<br># mcast_join = 239.2.11.71<br> port = 8649<br># bind = 239.2.11.71<br>}<br><br></span></code></div><div><code><span style="font-family:arial,helvetica,sans-serif">> Slave node: All section must be commented:<br></span></code><code><span style="font-family:arial,helvetica,sans-serif">#udp_recv_channel {<br># mcast_join = 239.2.11.71<br># port = 8649<br># bind = 239.2.11.71<br>#}</span></code><br><br></div><div>So I decided to create two gmond.conf in base AMI. The first one will be used to all Slaves and the second will be used to Master.<br></div><div>This will be changed in Plugin time.<br><br></div><div>I´ve prepared the gmond.conf with a) , b) and c) Slave node and gmond_master.conf with a), b) and c) Master node.<br></div><div><br></div><div>With the ganglia plugin:<br><br></div><div>1. Gets the private ip of the Master node (master_ip)<br></div><div>2. Checks if the node is master. <br>3. If true, replaces the gmond.conf file from gmond_server.conf, replaces the master_ip in this file and restart the services needed.<br></div><div>4. If false, replaces the master_ip in gmond.conf and restart the ganglia-monitor service.<br></div><div><br></div><div><br>from starcluster.clustersetup import ClusterSetup<br>from starcluster.logger import log<br><br>class ganglia(ClusterSetup):<br> def __init__(self):<br> log.debug('Installing Ganglia...')<br> def run(self, nodes, master, user, user_shell, volumes):<br> <a href="http://log.info" target="_blank">log.info</a>("Installing Ganglia in all nodes")<br> master_ip = None<br> for node in nodes:<br> if <a href="http://node.is">node.is</a>_master():<br> master_ip = node.private_ip_address<br> master.ssh.execute('mv /etc/ganglia/gmond_server.conf /etc/ganglia/gmond.conf')<br> master.ssh.execute("sed s/master_ip/%s/ /etc/ganglia/gmond.conf > ar&&mv ar /etc/ganglia/gmond.conf" % master_ip)<br> master.ssh.execute('service ganglia-monitor restart && sudo service gmetad restart && sudo service apache2 restart')<br> else: <br> node.ssh.execute("sed s/master_ip/%s/ /etc/ganglia/gmond.conf > ar&&mv ar /etc/ganglia/gmond.conf" % master_ip)<br> node.ssh.execute('service ganglia-monitor restart')<br><br><br></div><div>That´s it.. You now can check on http://<your_master_ip_address>/ganglia and monitor all cluster´s stuff.<br><br></div><div>Hope this helps you,<br><br></div><div>Sérgio Mafra<br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 22, 2015 at 8:47 AM, Sergio Mafra <span dir="ltr"><<a href="mailto:sergiohmafra@gmail.com" target="_blank">sergiohmafra@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi SC_Folks,<br><br></div>Is there any plugin around developed to install Ganglia on StarCluster?<br><br></div>All best,<br><br></div>Sergio<br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>StarCluster mailing list</span><br><span><a href="mailto:StarCluster@mit.edu">StarCluster@mit.edu</a></span><br><span><a href="http://mailman.mit.edu/mailman/listinfo/starcluster">http://mailman.mit.edu/mailman/listinfo/starcluster</a></span><br></div></blockquote></body></html>