[StarCluster] resource sg-* has a dependent object

[David Mrva]

Hello,

I've got a an issue similar to the one discussed at 
https://github.com/jtriley/StarCluster/issues/270. The full output from 
my terminate commandline is:

starcluster -c config terminate smc2.0

StarCluster - (http://star.mit.edu/cluster) (v. 0.95.5)
Software Tools for Academics and Researchers (STAR)
Please submit bug reports to starcluster at mit.edu

Terminate EBS cluster smc2.0 (y/n)? y
2014-10-29 14:32:50,042 >>> Running plugin speechmatics.CustomSetup
2014-10-29 14:32:50,127 >>> Running plugin 
starcluster.plugins.users.CreateUsers
2014-10-29 14:32:50,256 >>> Running plugin starcluster.plugins.sge.SGEPlugin
2014-10-29 14:32:50,324 >>> Running plugin 
starcluster.clustersetup.DefaultClusterSetup
2014-10-29 14:32:50,573 >>> Detaching volume vol-f75924f6 from master
2014-10-29 14:32:50,865 >>> Terminating node: master (i-5c350c57)
2014-10-29 14:32:51,116 >>> Waiting for cluster to terminate...
2014-10-29 14:32:52,345 >>> Removing placement group: @sc-smc2.0
2014-10-29 14:34:19,758 >>> Removing security group: @sc-smc2.0
2014-10-29 14:39:28,140 !!! ERROR - Failed to terminate cluster!
Traceback (most recent call last):
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/commands/terminate.py", 
line 87, in terminate
     self._terminate_cluster(cl)
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/commands/terminate.py", 
line 64, in _terminate_cluster
     cl.terminate_cluster()
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/cluster.py", 
line 1603, in terminate_cluster
     self.ec2.delete_group(sg)
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/awsutils.py", 
line 268, in delete_group
     ret_val = group.delete()
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/boto/ec2/securitygroup.py", 
line 90, in delete
     dry_run=dry_run
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/boto/ec2/connection.py", 
line 3004, in delete_security_group
     return self.get_status('DeleteSecurityGroup', params, verb='POST')
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/boto/connection.py", 
line 1196, in get_status
     raise self.ResponseError(response.status, response.reason, body)
EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>DependencyViolation</Code><Message>resource 
sg-f7079392 has a dependent 
object</Message></Error></Errors><RequestID>2c366b92-a46d-4113-90f2-ed62d1fa3f9d</RequestID></Response>
2014-10-29 14:39:28,141 !!! ERROR - Use -f to forcefully terminate the 
cluster
2014-10-29 14:39:28,141 !!! ERROR - DependencyViolation: resource 
sg-f7079392 has a dependent object
Traceback (most recent call last):
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/cli.py", 
line 274, in main
     sc.execute(args)
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/commands/terminate.py", 
line 101, in execute
     self.terminate(cluster_name, force=self.opts.force)
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/commands/terminate.py", 
line 87, in terminate
     self._terminate_cluster(cl)
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/commands/terminate.py", 
line 64, in _terminate_cluster
     cl.terminate_cluster()
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/cluster.py", 
line 1603, in terminate_cluster
     self.ec2.delete_group(sg)
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/starcluster/awsutils.py", 
line 268, in delete_group
     ret_val = group.delete()
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/boto/ec2/securitygroup.py", 
line 90, in delete
     dry_run=dry_run
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/boto/ec2/connection.py", 
line 3004, in delete_security_group
     return self.get_status('DeleteSecurityGroup', params, verb='POST')
   File 
"/speechmatics/venv/local/lib/python2.7/site-packages/boto/connection.py", 
line 1196, in get_status
     raise self.ResponseError(response.status, response.reason, body)
EC2ResponseError: EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>DependencyViolation</Code><Message>resource 
sg-f7079392 has a dependent 
object</Message></Error></Errors><RequestID>2c366b92-a46d-4113-90f2-ed62d1fa3f9d</RequestID></Response>


As the post at 
https://awssolutions.wordpress.com/2012/10/17/deleting-aws-security-group-error-has-a-dependant-job/ 
suggested I went to network interfaces tab in AWS EC2 web console and 
there were a number of network interfaces associated with security group 
@sc-smc2.0. These network interfaces were not associated with any 
instance. Only after removing the interfaces associated with this 
security group the terminate command  (with -f) removed the security 
group successfully. It does not seem that any amount was going to fix 
the issue. I had only master running for the last day and there were 
about five network interfaces that I had to remove by hand.

Hope this may help to identify the underlying problem. This issue has 
not happened before when terminating a cluster.

Best regards,
David