[StarCluster] ERROR - Invalid AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY combination
Justin Riley
jtriley at MIT.EDU
Wed Dec 22 10:12:22 EST 2010
Hi Stuart,
I was just about to point you here:
http://groups.google.com/group/boto-users/browse_thread/thread/c6700ddcfd55670e
and ask about your Python version :D
Glad you got it figured out and thanks for following up.
In the future, would you mind joining the mailing list and posting to
starcluster at mit.edu? This allows others to see your issues/questions and
will help folks that might run into similar issues.
Thanks!
~Justin
On 12/22/10 5:41 AM, Stuart Young wrote:
> Hi Justin,
>
> I figured out it was my version of Python (2.7) that handles unicode
> differently to python2.6 and was mangling my signatures so that I
> always got the 'SignatureDoesNotMatch' error. After I installed
> python2.6 it worked fine.
>
> starcluster -c /root/.starcluster/config listpublic
>
> >>> Listing all public StarCluster images...
>
> 32bit Images:
> -------------
> [0] ami-8cf913e5 us-east-1 starcluster-base-ubuntu-10.04-x86-rc3
> [1] ami-8f9e71e6 us-east-1 starcluster-base-ubuntu-9.04-x86
> [2] ami-17b15e7e us-east-1 starcluster-base-ubuntu-9.10-x86-rc7
> [3] ami-d1c42db8 us-east-1 starcluster-base-ubuntu-9.10-x86-rc8
>
> 64bit Images:
> --------------
> [0] ami-12b6477b us-east-1
> starcluster-base-centos-5.4-x86_64-ebs-hvm-gpu-rc2
> [1] ami-0af31963 us-east-1 starcluster-base-ubuntu-10.04-x86_64-rc1
> [2] ami-a19e71c8 us-east-1 starcluster-base-ubuntu-9.04-x86_64
> [3] ami-2941ad40 us-east-1 starcluster-base-ubuntu-9.10-x86_64-rc3
> [4] ami-a5c42dcc us-east-1 starcluster-base-ubuntu-9.10-x86_64-rc4
>
> total images: 9
>
>
> Thanks for your help!
>
> Stuart
>
>
>
>
> On 12/21/2010 3:46 AM, Stuart Young wrote:
>> Hi Justin,
>>
>> I'm using boto version 1.9b, as it was downloaded automatically when
>> I installed StarCluster 0.91.2.
>>
>> python -c 'import boto; print boto.Version'
>> 1.9b
>>
>> Btw, I looked into how boto 1.9b is sending out the request to
>> pinpoint exactly where the error occurs (Note the "reason:
>> 'Forbidden'" part of the response):
>>
>>
>> connection.make_request connection.make_request(self, action,
>> params, path, verb)
>> connection.get_signature connection.get_signature(self,
>> params, verb, path)
>> connection.calc_signature2 connection.calc_signature2(self,
>> params, verb, path)
>> connection.calc_signature2 Returning qs:
>> AWSAccessKeyId=AKIXXXXXXXXXXXXTHQ&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2010-12-21T08%3A27%3A08&Version=2009-11-30
>> and b64: QNqIbL9pTcv0RXXXXXXXXXXXXXXXXXXGsfZeb1afk=
>> connection.make_request Returning
>> AWSAuthConnection.make_request(...)
>> connection.AWSAuthConnection.make_request
>> connection.AWSAuthConnection.make_request(self, method, path,
>> headers, data, host, auth_path, sender)
>> connection.AWSAuthConnection.make_request Before return self._mexe
>> connection._mexe connection._mexe(method, path, data, headers,
>> host, sender)
>> Method: GET
>> Path:
>> /?AWSAccessKeyId=AKIXXXXXXXXXXXTHQ&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2010-12-21T08%3A27%3A08&Version=2009-11-30&Signature=QNqIbL9pTcvXXXXXXXXXXXXXXXXXXXGsfZeb1afk%3D
>> Data:
>> Headers: {'Date': 'Tue, 21 Dec 2010 08:27:08 GMT',
>> 'Content-Length': '0', 'Authorization': 'AWS
>> AKIAIXXXXXXXXXXTHQ:6lzoMU3zIhy9909bXDkKpB3MqVQ=', 'User-Agent':
>> 'Boto/1.9b (linux2)'}
>> Host: None
>> connection._mexe try 0
>> connection._mexe callable(sender) NOT DEFINED
>> connection._mexe response: <httplib.HTTPResponse instance at
>> 0x140f2d8>
>> connection._mexe BEFORE if response.status == 500
>> connection._mexe response.status < 300 or >=400
>> <HTTPResponse at 140f2d8>
>> _method: 'GET'
>> chunk_left: None
>> chunked: 1
>> debuglevel: 0
>> fp: <_fileobject at 0x1404050>: <socket._fileobject object at
>> 0x1404050>
>> length: None
>> msg: <HTTPMessage at 140f320: Transfer-Encoding: chunked
>> Date: Tue, 21 Dec 2010 08:27:08 GMT
>> Server: AmazonEC2
>> >
>> dict: <dictionary at 0x1496a60>: {'transfer-encoding':
>> 'chunked', 'date': 'Tue, 21 Dec 2010 08:27:08 GMT', 'server':
>> 'AmazonEC2'}
>> encodingheader: None
>> fp: None
>> headers: <list at 0x13f2368>: ['Transfer-Encoding:
>> chunked\r\n', 'Date: Tue, 21 Dec 2010 08:27:08 GMT\r\n', 'Server:
>> AmazonEC2\r\n']
>> maintype: 'text'
>> plist: <list at 0x13f23f8>: []
>> plisttext: ''
>> seekable: 0
>> startofbody: None
>> startofheaders: None
>> status: ''
>> subtype: 'plain'
>> type: 'text/plain'
>> typeheader: None
>> unixfrom: ''
>> reason: 'Forbidden'
>> status: 403
>> strict: 0
>> version: 11
>> will_close: <bool at 0x787990>: False
>> boto.exception.EC2ResponseError
>> boto.exception.EC2ResponseError(BotoServerError)
>> boto.exception.EC2ResponseError BotoServerError: <class
>> 'boto.exception.BotoServerError'>
>> <class 'boto.exception.BotoServerError'>
>> awsutils.is_valid_conn boto.exception.EC2REsponseError RAISED!!!
>> awsutils.is_valid_conn e: EC2ResponseError: 403 Forbidden
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Response><Errors><Error><Code>SignatureDoesNotMatch</Code><Message>The
>> request signature we calculated does not match the signature you
>> provided. Check your AWS Secret Access Key and signing method.
>> Consult the service documentation for
>> details.</Message></Error></Errors><RequestID>cdb73666-37a6-4dfc-9f66-2add7b04a0bf</RequestID></Response>
>> awsutils.is_valid_conn DUMP boto.exception.EC2ResponseError:
>> <class 'boto.exception.EC2ResponseError'>
>>
>> cluster._validate_credentials self.ec2.is_valid_conn NOT TRUE
>> cluster.is_valid ERROR
>> cluster.py:782 - ERROR - Invalid
>> AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY combination.
>> cli.py:243 - ERROR - settings for cluster template "smallcluster"
>> are not valid
>>
>>
>> I emailed Amazon to ask them to check the Request ID. Looking at the
>> request, is there anything that jumps out at you?
>>
>> Cheers,
>>
>> Stuart
>>
>>
>>
>> On 12/20/2010 10:05 PM, Justin Riley wrote:
>>> What version of boto are you using? Also You can determine this using:
>>>
>>> $ python -c 'import boto; print boto.Version'
>>>
>>> Also this is using StarCluster 0.91.2 correct?
>>>
>>> ~Justin
>>>
>>> On 12/20/10 8:36 PM, Stuart Young wrote:
>>>> Hi Justin,
>>>>
>>>> ElasticFox works fine with all three sets of credentials. I'm still
>>>> looking into boto's error. Will update you if I find anything.
>>>>
>>>> Cheers,
>>>>
>>>> Stuart
>>>>
>>>>
>>>> On 12/20/2010 4:21 PM, Stuart Young wrote:
>>>>> Hi Justin,
>>>>>
>>>>> Thanks for the debugging methods information - I actually got it
>>>>> working by trial and error using a permuation of method 2 but
>>>>> method 1 is so much more straightforward! (As you'll guess, I'm a
>>>>> noob to python.) I tracked the error down to boto's connect.py and
>>>>> ec2/connect.py modules:
>>>>>
>>>>> awsutils.conn self.conn()
>>>>> awsutils.conn self.aws_access_key:
>>>>> **AAAAA<no-weird-symbols>HHHHQ**
>>>>> awsutils.conn self.aws_secret_access_key:
>>>>> **4+0Ma<no-weird-symbols-except'+'>DrschU6**
>>>>> awsutils.py:57 - DEBUG - creating self._conn w/
>>>>> connection_authenticator kwargs = {'path': '/', 'region':
>>>>> None, 'port': None, 'is_secure': True}
>>>>> awsutils.conn Doing self._conn =
>>>>> self.connection_authenticator()
>>>>> awsutils.conn self.connection_authenticator: <function
>>>>> connect_ec2 at 0x9c5cf8>
>>>>> boto.__init__.connect_ec2
>>>>> boto.__init__.connect_ec2(aws_access_key_id,
>>>>> aws_secret_access_key, kwargs)
>>>>> boto.__init__.connect_ec2 aws_access_key_id:
>>>>> AAAAA<no-weird-symbols>HHHHQ
>>>>> boto.__init__.connect_ec2
>>>>> aws_secret_boto.__init__.connect_ec2access_key:
>>>>> 4+0Ma<no-weird-symbols-except'+'>DrschU6
>>>>> boto.connection.AWSQueryConnection.__init__
>>>>> boto.connection.AWSQueryConnection.__init__(self, host,
>>>>> aws_access_key_id, aws_secret_access_key, etc)
>>>>> boto.connection.AWSQueryConnection.__init__
>>>>> aws_access_key_id: AAAAA<no-weird-symbols>HHHHQ
>>>>> boto.connection.AWSQueryConnection.__init__
>>>>> aws_secret_access_key: 4+0Ma<no-weird-symbols-except'+'>DrschU6
>>>>> awsutils.conn Returning...
>>>>> awsutils.is_valid_conn boto.exception.EC2REsponseError
>>>>> RAISED!!!
>>>>> awsutils.is_valid_conn e: EC2ResponseError: 403 Forbidden
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <Response><Errors><Error><Code>SignatureDoesNotMatch</Code><Message>The
>>>>> request signature we calculated does not match the signature
>>>>> you provided. Check your AWS Secret Access Key and signing
>>>>> method. Consult the service documentation for
>>>>> details.</Message></Error></Errors><RequestID>deb3819a-d8d1-457a-a432-b6f35675be14</RequestID></Response>
>>>>> cluster._validate_credentials ERROR OCCURRED in
>>>>> self.ec2.is_valid_conn
>>>>> cluster.py:770 - ERROR - Invalid
>>>>> AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY combination.
>>>>> cli.py:253 - ERROR - settings for cluster template
>>>>> "smallcluster" are not valid
>>>>>
>>>>>
>>>>> I was able to connect and launch instances with my existing
>>>>> AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY via the perl module
>>>>> Net::Amazon::EC2 so I'm perplexed as to why I'm getting the error
>>>>> using boto. I will try elasticfox as you suggested and update you
>>>>> later.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Stuart.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 12/20/2010 4:04 PM, Justin Riley wrote:
>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>> Hash: SHA1
>>>>>>
>>>>>> On 12/20/2010 11:19 AM, Stuart Young wrote:
>>>>>>> cli.py:1083 - ERROR - SignatureDoesNotMatch: The request signature
>>>>>>> we calculated does not match the signature you provided. Check your
>>>>>>> AWS Secret Access Key and signing method. Consult the service
>>>>>>> documentation for details.
>>>>>> Hmmm this is an error reported directly from Amazon; this shouldn't be
>>>>>> caused
>>>>>> by boto and definitely not paramiko. I can reproduce this message if I
>>>>>> remove a character from my AWS_SECRET_ACCESS_KEY so it seems either
>>>>>> something's wrong with the keys in your config or wrong on Amazon's side.
>>>>>> In the past when I've seen this message it's always due to using quotes
>>>>>> or a stray
>>>>>> character some where...
>>>>>>
>>>>>> Are these recently created accounts?
>>>>>>
>>>>>>> I got the same error when I tried with two other AWS accounts. Is this
>>>>>>> maybe something to do with boto or paramiko?
>>>>>> Would you mind installing elasticfox (firefox plugin) and see if it has
>>>>>> issues with your access keys as well?
>>>>>>
>>>>>>> I was also wondering if it's possible to to run a local, unzipped copy of the StarCluster egg for
>>>>>>> debugging purposes?
>>>>>> Yes it is and there are two ways:
>>>>>>
>>>>>> 1. First uninstall starcluster (remove the egg from your site-packages).
>>>>>> Download the 0.91.2 release, unpack it somewhere, and run python
>>>>>> setup.py develop. This will 'link' the unzipped source to your python
>>>>>> install. This means if you make changes to the source code, you will be
>>>>>> able to import and see/use those changes without having to reinstall.
>>>>>> This is currently how I'm developing the code.
>>>>>>
>>>>>> OR
>>>>>>
>>>>>> 2. Simply unzip the egg in your Python's site-packages directory and
>>>>>> edit the source within your site-packages directory. You'll likely want
>>>>>> to move the egg outside of your site-packages directory after you unzip
>>>>>> it to make sure you import the unzipped code and not the egg.
>>>>>>
>>>>>> Does that make sense?
>>>>>>
>>>>>> ~Justin
>>>>>> -----BEGIN PGP SIGNATURE-----
>>>>>> Version: GnuPG v2.0.16 (GNU/Linux)
>>>>>> Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/
>>>>>>
>>>>>> iEYEARECAAYFAk0PxMwACgkQ4llAkMfDcrkylwCfQpHxxNaZsbXX3xBNe9hbRfgE
>>>>>> QAkAnR2YlcUQTH4cQTEg4M8V8Cg8MQIc
>>>>>> =1Rcm
>>>>>> -----END PGP SIGNATURE-----
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20101222/200bf146/attachment.htm
More information about the StarCluster
mailing list