[StarCluster] Ganglia Plugin ?

[Ryan G]

Very nice.  I'll have to try this out this week when I'm back at work.  Thanks for putting the effort in!

Sent from my iPhone

> On May 25, 2015, at 10:42 AM, Sergio Mafra <sergiohmafra at gmail.com> wrote:
> 
> Just forgot to add that you need to open port 80 in StarCluster security groups in order to allow to access Ganglia via web.
> So, in config file you need to add:
> 
> [permission www]
> # open port 80 to the world
> from_port = 80
> to_port = 80
> 
> and in the [Cluster ....]
> permissions = www
> 
> 
> Good luck to all,
> 
> Sergio
> 
> 
>> On Mon, May 25, 2015 at 11:14 AM, Sergio Mafra <sergiohmafra at gmail.com> wrote:
>> Hi SC_Folks,
>> 
>> I´ve made a little research and found interesting things that helped to build a plugin of configuring Ganglia to run with StarCluster.
>> Here comes what I did in case if you´re interested:
>> 
>> 1. First, I used the Shared StarCluster AMI Ubuntu 14.04. It can be found on Public AMIs. (us-east-1 -> ami-38b99850).
>> 2. This tutorial was great: https://www.digitalocean.com/community/tutorials/introduction-to-ganglia-on-ubuntu-14-04
>> 3. I´ve installed these apps in base AMI using: sudo apt-get install -y ganglia-monitor rrdtool gmetad ganglia-webfrontend
>> 4. Ganglia needs to be configured using two config files in /etc/config: gmetad.conf and gmond.conf
>> 5. The gmetad.conf needs to have the name of the cluster you intend to monitor. I used a fixed name (StarCluster) but you can chance in the future with the name tag of the cluster you´re provisioning, So this is what I´ve chnaged:
>> data_souce "StarCluster" 60 localhost
>> 6. The gmond.conf needs to changed in 3 sections:
>>  a) Cluster section: Note that I´ve change the unspecified to StarCluster (same name in the gmetad.conf)
>>  cluster {
>>    name = "StarCluster"
>>    owner = "unspecified"
>>    latlong = "unspecified"
>>    url = "unspecified"
>>  }
>> 
>> b) udp_send_channel section: You have to comment the mcast_join line and add the ip of the cluster master. The plugin will handle this. So I add the line "host = master_ip". This will be the same if the node is master or slave.
>> udp_send_channel {
>> #  mcast_join = 239.2.11.71
>>   host = master_ip
>>   port = 8649
>>   ttl = 1
>> }
>> 
>> c) udp_recv_channel: You will have to have two kinds of configuration regarding the master and the slaves. In master, the section will be the following: The mcast_join line commented and bind line too.
>> 
>> > Master node:
>> udp_recv_channel {
>> #  mcast_join = 239.2.11.71
>>   port = 8649
>> #  bind = 239.2.11.71
>> }
>> 
>> > Slave node: All section must be commented:
>> #udp_recv_channel {
>> #  mcast_join = 239.2.11.71
>> #  port = 8649
>> #  bind = 239.2.11.71
>> #}
>> 
>> So I decided to create two gmond.conf in base AMI. The first one will be used to all Slaves and the second will be used to Master.
>> This will be changed in Plugin time.
>> 
>> I´ve prepared the gmond.conf with a) , b) and c) Slave node and gmond_master.conf with a), b) and c) Master node.
>> 
>> With the ganglia plugin:
>> 
>> 1. Gets the private ip of the Master node (master_ip)
>> 2. Checks if the node is master. 
>> 3. If true, replaces the gmond.conf file from gmond_server.conf, replaces the master_ip in this file and restart the services needed.
>> 4. If false, replaces the master_ip in gmond.conf and restart the ganglia-monitor service.
>> 
>> 
>> from starcluster.clustersetup import ClusterSetup
>> from starcluster.logger import log
>> 
>> class ganglia(ClusterSetup):
>>      def __init__(self):
>>           log.debug('Installing Ganglia...')
>>      def run(self, nodes, master, user, user_shell, volumes):
>>             log.info("Installing Ganglia in all nodes")
>>             master_ip = None
>>             for node in nodes:
>>                 if node.is_master():
>>                    master_ip = node.private_ip_address
>>                    master.ssh.execute('mv /etc/ganglia/gmond_server.conf /etc/ganglia/gmond.conf')
>>                    master.ssh.execute("sed s/master_ip/%s/ /etc/ganglia/gmond.conf > ar&&mv ar /etc/ganglia/gmond.conf" % master_ip)
>>                    master.ssh.execute('service ganglia-monitor restart && sudo service gmetad restart && sudo service apache2 restart')
>>                 else: 
>>                    node.ssh.execute("sed s/master_ip/%s/ /etc/ganglia/gmond.conf > ar&&mv ar /etc/ganglia/gmond.conf" % master_ip)
>>                    node.ssh.execute('service ganglia-monitor restart')
>> 
>> 
>> That´s it.. You now can check on http://<your_master_ip_address>/ganglia and monitor all cluster´s stuff.
>> 
>> Hope this helps you,
>> 
>> Sérgio Mafra
>> 
>>> On Fri, May 22, 2015 at 8:47 AM, Sergio Mafra <sergiohmafra at gmail.com> wrote:
>>> Hi SC_Folks,
>>> 
>>> Is there any plugin around developed to install Ganglia on StarCluster?
>>> 
>>> All best,
>>> 
>>> Sergio
> 
> _______________________________________________
> StarCluster mailing list
> StarCluster at mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20150525/d9f3ce15/attachment.htm