R53 403 Forbidden error

[jimmyt857]

Reopening because I still seem to be getting errors. Accidentally posted to the old issue, so copying my post:

I'm still getting and error, although it's a bit different now. Here's the log:

[INFO] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Initializing CurlHandleContainer with size 25
[TRACE] 2016-08-02 19:35:15 AWSClient [140409173084160] Found body, but content-length has not been set, attempting to compute content-length
[DEBUG] 2016-08-02 19:35:15 AWSAuthV4Signer [140409173084160] Calculated sha256 d3e27de837776916e82c6a5738a5561ad5577be05364ec77ab2c8f16d2caf2e3 for payload.
[DEBUG] 2016-08-02 19:35:15 AWSAuthV4Signer [140409173084160] Canonical Header String: content-length:730
content-type:application/xml
host:route53.amazonaws.com
user-agent:aws-sdk-cpp/0.13.11 Linux/3.13.0-74-generic x86_64
x-amz-content-sha256:d3e27de837776916e82c6a5738a5561ad5577be05364ec77ab2c8f16d2caf2e3
x-amz-date:20160802T193515Z

[DEBUG] 2016-08-02 19:35:15 AWSAuthV4Signer [140409173084160] Signed Headers value:content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date
[DEBUG] 2016-08-02 19:35:15 AWSAuthV4Signer [140409173084160] Canonical Request String: POST
/2013-04-01/hostedzone/hostedzone/Z2CLVO1AT5KHVK/rrset/

content-length:730
content-type:application/xml
host:route53.amazonaws.com
user-agent:aws-sdk-cpp/0.13.11 Linux/3.13.0-74-generic x86_64
x-amz-content-sha256:d3e27de837776916e82c6a5738a5561ad5577be05364ec77ab2c8f16d2caf2e3
x-amz-date:20160802T193515Z

content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date
d3e27de837776916e82c6a5738a5561ad5577be05364ec77ab2c8f16d2caf2e3
[DEBUG] 2016-08-02 19:35:15 AWSAuthV4Signer [140409173084160] Final String to sign: AWS4-HMAC-SHA256
20160802T193515Z
20160802/us-east-1/route53/aws4_request
7973676c2b756368f34192ecb1f7f031b950ba07d38b19e78aad9ac183e8a7de
[DEBUG] 2016-08-02 19:35:15 AWSAuthV4Signer [140409173084160] Final computed signing hash: 0e54806fedd0d32e6af7cb386ba8d183406af38dcc3441397de3496e79b30b7f
[DEBUG] 2016-08-02 19:35:15 AWSAuthV4Signer [140409173084160] Signing request with: AWS4-HMAC-SHA256 Credential=AKIAIMTDWTMSLPNECNJA/20160802/us-east-1/route53/aws4_request, SignedHeaders=content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=0e54806fedd0d32e6af7cb386ba8d183406af38dcc3441397de3496e79b30b7f
[DEBUG] 2016-08-02 19:35:15 AWSClient [140409173084160] Request Successfully signed
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Making request to https://route53.amazonaws.com/2013-04-01/hostedzone/hostedzone/Z2CLVO1AT5KHVK/rrset/
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Including headers:
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] authorization: AWS4-HMAC-SHA256 Credential=AKIAIMTDWTMSLPNECNJA/20160802/us-east-1/route53/aws4_request, SignedHeaders=content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=0e54806fedd0d32e6af7cb386ba8d183406af38dcc3441397de3496e79b30b7f
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] content-length: 730
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] content-type: application/xml
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] host: route53.amazonaws.com
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] user-agent: aws-sdk-cpp/0.13.11 Linux/3.13.0-74-generic x86_64
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] x-amz-content-sha256: d3e27de837776916e82c6a5738a5561ad5577be05364ec77ab2c8f16d2caf2e3
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] x-amz-date: 20160802T193515Z
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Attempting to acquire curl connection.
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] No current connections available in pool. Attempting to create new connections.
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] attempting to grow pool size by 2
[INFO] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Pool successfully grown by 2
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Returning connection handle 0x3e7ca80
[DEBUG] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Obtained connection handle 0x3e7ca80
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] HTTP/1.1 100 Continue^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] ^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] HTTP/1.1 403 Forbidden^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] x-amzn-RequestId: 027fde3c-58e8-11e6-b65a-3dbd29267b32^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Content-Length: 152^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Date: Tue, 02 Aug 2016 19:33:37 GMT^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Connection: close^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] ^M

[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] 152 bytes written to response.
[DEBUG] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Returned http response code 403
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Response content-length header: 152
[TRACE] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Response body length: 152
[DEBUG] 2016-08-02 19:35:15 CurlHttpClient [140409173084160] Releasing curl handle 0x3e7ca80
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Releasing curl handle 0x3e7ca80
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Notifying waiting threads.
[DEBUG] 2016-08-02 19:35:15 AWSClient [140409173084160] Request returned error. Attempting to generate appropriate error codes from response
[TRACE] 2016-08-02 19:35:15 AWSClient [140409173084160] Error response is

Unable to determine service/operation name to be authorized

[WARN] 2016-08-02 19:35:15 AWSErrorMarshaller [140409173084160] Encountered Unknown AWSError

Unable to generate a proper httpResponse from the response stream. Response code: 403:
[TRACE] 2016-08-02 19:35:15 AWSClient [140409173084160] Request was either successful, or we are now out of retries.
[INFO] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Cleaning up CurlHandleContainer.
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Cleaning up 0x3e7ca80.
[DEBUG] 2016-08-02 19:35:15 CurlHandleContainer [140409173084160] Cleaning up 0x3fde8d0.

[JonathanHenson]

are ANY of your requests to r53 successful?

[jimmyt857]

No, unfortunately I seem to get this same error every time

[JonathanHenson]

It looks like you are passing extra data to HostedZoneId on your ChangeResourceRecordSetsRequest object. It should look something like this:

POST
/2013-04-01/hostedzone/Z2CLVO1AT5KHVK/rrset/

but you are passing

POST
/2013-04-01/hostedzone/hostedzone/Z2CLVO1AT5KHVK/rrset/

[jimmyt857]

Oh, wow, thanks for spotting that. I'm getting the hosted zone id from a HostedZone object. It seems that HostedZone.GetId() returns "/hostedzone/Z2CLVO1AT5KHVK" for me. It that how it's supposed to work?

[jimmyt857]

The synchronous version works for me now. Once again, I am getting a segfault when calling the async version:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffee7bd700 (LWP 20478)]
0x00000000021b8ca0 in typeinfo for Aws::Client::AWSJsonClient ()
(gdb) bt
#0 0x00000000021b8ca0 in typeinfo for Aws::Client::AWSJsonClient ()
#1 0x00000000019e3767 in Aws::Route53::Route53Client::ChangeResourceRecordSetsAsyncHelper(Aws::Route53::Model::ChangeResourceRecordSetsRequest const&, std::function<void (Aws::Route53::Route53Client const*, Aws::Route53::Model::ChangeResourceRecordSetsRequest const&, Aws::Utils::Outcome<Aws::Route53::Model::ChangeResourceRecordSetsResult, Aws::Client::AWSErrorAws::Route53::Route53Errors > const&, std::shared_ptr<Aws::Client::AsyncCallerContext const> const&)> const&, std::shared_ptr<Aws::Client::AsyncCallerContext const> const&) const ()
#2 0x00007ffff5b65a60 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff724f182 in start_thread (arg=0x7fffee7bd700) at pthread_create.c:312
#4 0x00007ffff52cd47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

This may be related to my other segfault issues, though.

[JonathanHenson]

what is your stack size? Have you attempted increasing it?

[jimmyt857]

Stack size is 8MB, so I don't think that could be it

[jimmyt857]

Just tried increasing to 16MB, didnt help

[jimmyt857]

Tried this again with debugging symbols. For some reason I'm getting a different stack trace, but here it is:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffee7ae700 (LWP 29429)]
0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x000000000209a1f0 in Aws::Route53::Route53Client::ChangeResourceRecordSetsAsyncHelper(Aws::Route53::Model::ChangeResourceRecordSetsRequest const&, std::function<void (Aws::Route53::Route53Client const*, Aws::Route53::Model::ChangeResourceRecordSetsRequest const&, Aws::Utils::Outcome<Aws::Route53::Model::ChangeResourceRecordSetsResult, Aws::Client::AWSErrorAws::Route53::Route53Errors > const&, std::shared_ptr<Aws::Client::AsyncCallerContext const> const&)> const&, std::shared_ptr<Aws::Client::AsyncCallerContext const> const&) const (this=0x7fffffffdca0, request=..., handler=..., context=std::shared_ptr (empty) 0x0)
at /home/ubuntu/colossi-servers/cpp/aws-sdk-cpp/aws-cpp-sdk-route53/source/Route53Client.cpp:204
#2 0x000000000209a07b in Aws::Route53::Route53Client::<lambda()>::operator()(void) const (__closure=0x3105690)
at /home/ubuntu/colossi-servers/cpp/aws-sdk-cpp/aws-cpp-sdk-route53/source/Route53Client.cpp:199
#3 0x00000000020cfe04 in std::_Bind<Aws::Route53::Route53Client::ChangeResourceRecordSetsAsync(const Aws::Route53::Model::ChangeResourceRecordSetsRequest&, const ChangeResourceRecordSetsResponseReceivedHandler&, const std::shared_ptr&) const::<lambda()>()>::__call(<unknown type in /home/ubuntu/colossi-servers/cpp/worker/Worker.exe, CU 0x50cd275, DIE 0x52c8dfa>, std::_Index_tuple<>) (this=0x3105690,
__args=<unknown type in /home/ubuntu/colossi-servers/cpp/worker/Worker.exe, CU 0x50cd275, DIE 0x52c8dfa>) at /usr/include/c++/4.9/functional:1264
#4 0x00000000020c602e in std::_Bind<Aws::Route53::Route53Client::ChangeResourceRecordSetsAsync(const Aws::Route53::Model::ChangeResourceRecordSetsRequest&, const ChangeResourceRecordSetsResponseReceivedHandler&, const std::shared_ptr&) const::<lambda()>()>::operator()<, void>(void) (this=0x3105690)
at /usr/include/c++/4.9/functional:1323
#5 0x00000000020bc8cf in std::_Function_handler<void(), std::_Bind<Aws::Route53::Route53Client::ChangeResourceRecordSetsAsync(const Aws::Route53::Model::ChangeResourceRecordSetsRequest&, const ChangeResourceRecordSetsResponseReceivedHandler&, const std::shared_ptr&) const::<lambda()>()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/4.9/functional:2039
#6 0x0000000002319a96 in std::_Bind_simple<std::function<void ()> ()>::_M_invoke<>(std::_Index_tuple<>) (this=0x326af98) at /usr/include/c++/4.9/functional:1700
#7 0x00000000023199db in std::_Bind_simple<std::function<void ()> ()>::operator()() (this=0x326af98) at /usr/include/c++/4.9/functional:1688
#8 0x0000000002319958 in std::thread::_Impl<std::_Bind_simple<std::function<void ()> ()> >::_M_run() (this=0x326af80) at /usr/include/c++/4.9/thread:115
#9 0x00007ffff5b36350 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007ffff724f182 in start_thread (arg=0x7fffee7ae700) at pthread_create.c:312
#11 0x00007ffff52be47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

The lowest level visible source line is:

handler(this, request, ChangeResourceRecordSets(request), context);

context is an empty shared pointer, but that's just the default value so that can't be it, right?

[JonathanHenson]

Closing issue, since this is not about route 53 anymore and we have another issue debugging this #264