home / 2017.03.30 16:00 /aws /api gateway /iam /sigv4
This document explores the process of accessing services exposed through Amazon’s API Gateway that are secured with IAM credentials instead of Cognito tokens. The mechanism for getting access to those services is based on signing the request using our secret key instead of actually sending the key over. When accessing Amazon services through the SDK, the SDK takes care of this signing process for you, but if you want to access your own custom services exposed through API Gateway you will have to write your own code to sign that request, and this is quite an involved process.
First we need a service we want to access. I’ve created a simple javascript Lambda that returns some JSON data (I named it getUserForms):
exports.handler = (event, context, callback) => {
var result = {
'fields': [
{ 'name': 'family name', 'value': ''},
{ 'name': 'first name', 'value': ''},
{ 'name': 'address', 'value': ''},
{ 'name': 'postcode', 'value': ''}
]
}
callback(null, result);
};
Next, I make this service accessible through API Gateway by creating a new resource and a GET method invoking the Lambda. Once the API is published to staging, we can invoke it to obtain our data with some very simple javascript code:
var https = require('https');
var bl = require('bl');
function collecter(callback) {
return function(result) {
function onceCollected(err, data) {
var string = data.toString();
callback(string);
}
result.setEncoding('utf8');
result.pipe(bl(onceCollected));
result.on('error', console.error);
}
}
function loadForms(callback) {
var options = {
method: 'GET',
host: 'staging url',
path: '/cognitotest/forms'
};
var request = https.request(options, collecter(callback));
request.end()
}
loadForms(function(result) {
console.log(result);
});
I’ve added this code to a file test.js. Running it on node (with the appropriate dependencies available), we see the data getting printed on screen. Now it’s time to secure this endpoint. Go back to the API Gateway console, on your forms resource and the GET method and click the method request section. Under authorization settings select AWS_IAM for the authorization field and click the small check sign to confirm (small side note: the role corresponding to the credentials you will use to connect to the API needs to have permissions to execute API Gateway endpoints, and you can take care of that in the IAM console). Deploy to staging and run the test.js script again and you should see the following message:
{"message":"Missing Authentication Token"}
Signing a request is a complicated process. I have added relevant links about this at the end of the document, but I will go over the process and its implementation in the following sections, step by step, providing a library that can be used for signing almost any requests (there are still some parts I have not addressed).
The process for singing a request are:
These steps would work on the options
object used to make the request. Besides the options
object, other inputs will be
requestBody
accessKeyID
- will be part of the signature headersecretKey
- this will be used to sign the requestregion
- needed for the signature headerservice
- needed for the signature headersessionToken
- this is
required when we are working with temporary credentials obtained from
STS, which is the case when we are using federated Cognito identitiesThe following signRequest
is the main method which will handle the whole process:
function signRequest(options, requestBody, accessKeyID, secretKey, region, service, sessionToken) {
formatHeaders(options, sessionToken);
var canonicalRequestAndHash = toCanonicalRequestAndHash(options, requestBody)
var stringToSign = createStringToSign(options, canonicalRequestAndHash.rehashedCanonicalRequest, region, service)
var signingKey = createSigningKey(options, secretKey, region, service);
var signature = calculateSignature(signingKey, stringToSign.stringToSign)
var authorization = createAuthorizationHeader(accessKeyID, stringToSign.credentialsScope, canonicalRequestAndHash.signedHeaders, signature);
options.headers['Authorization'] = authorization;
}
A signed request to API Gateway will require an amazon date header and the host header. The session token will also need to be included in a header if we have one.
function formatHeaders(options, sessionToken) {
if (options.headers == undefined) {
options.headers = {}
}
if (options.headers[AMAZON_DATE_HEADER] == undefined) {
options.headers[AMAZON_DATE_HEADER] = getISO8601UTC();
}
if (options.headers[HOST_HEADER] == undefined) {
options.headers[HOST_HEADER] = options.host;
}
if (sessionToken) {
options.headers[AMAZON_SESSION_TOKEN_HEADER] = sessionToken;
}
}
The date itself needs to be the UTC date in ISO8601 format, and I compute it with the following (very unrefined) code:
function getISO8601UTC() {
var today = new Date();
var string = ''
string += today.getUTCFullYear()
string += withLeadingZero((today.getUTCMonth() + 1))
string += withLeadingZero(today.getUTCDate())
string += 'T'
string += withLeadingZero(today.getUTCHours())
string += withLeadingZero(today.getUTCMinutes())
string += withLeadingZero(today.getUTCSeconds())
string += 'Z'
return string;
}
function withLeadingZero(value) {
if (value < 10) {
return '0' + value;
} else {
return value;
}
}
The Amazon documentation tells us the canonical request is just a very text representation of the request in the following form:
CanonicalRequest =
HTTPRequestMethod + '\n' +
CanonicalURI + '\n' +
CanonicalQueryString + '\n' +
CanonicalHeaders + '\n' +
SignedHeaders + '\n' +
HexEncode(Hash(RequestPayload))
Hash
is a function that produces a message digest, typically SHA-256, HexEncode
returns a base-16 encoding of the digest in lowercase characters. My implementation of this follows:
function toCanonicalRequestAndHash(options, requestBody) {
var canonicalRequest = '';
canonicalRequest += options.method + '\n';
canonicalRequest += canonicalURI(options);
canonicalRequest += canonicalQueryString(options);
var canonicalHeaders = canonicalHeadersAndSignedHeaders(options);
canonicalRequest += canonicalHeaders.combined
var canonicalRequestHash = hashAndHexEncode(requestBody)
canonicalRequest += canonicalRequestHash;
var rehashedCanonicalRequest = hashAndHexEncode(canonicalRequest);
return {
'canonicalRequest': canonicalRequest,
'canonicalRequestHash': canonicalRequestHash,
'signedHeaders': canonicalHeaders.signedHeaders,
'rehashedCanonicalRequest': rehashedCanonicalRequest
};
}
canonicalRequest
to an empty stringoptions.method
to this string, followed by newline (note: maybe that options.method
should be capitalized)Next, let’s visit the individual methods, starting with canonicalURI
:
function canonicalURI(options) {
var path = options.path;
var parametersStart = path.indexOf("?");
if (parametersStart > -1) {
path = path.substring(0, parametersStart)
}
if (path && path.length > 0) {
return encodeURI(path) + '\n';
} else {
return encodeURI('/') + '\n';
}
}
canonicalURI
just takes the path, excluding the host and excluding the parameters, and returns it as a string on a single line/
as the canonical URIfunction canonicalQueryString(options) {
// todo: implement
return '\n';
}
function canonicalHeadersAndSignedHeaders(options) {
var sList = sortedList(options.headers);
var cHeaders = '';
var sHeaders = '';
for (var i = 0; i < sList.length; i++) {
cHeaders += sList[i].name + ':' + sList[i].value + '\n'
sHeaders += ';' + sList[i].name;
}
if (cHeaders.length == 0) {
// probably will never occur since the host header always needs to be included
cHeaders += '\n';
}
sHeaders = sHeaders.substring(1)
return {
'canonicalHeaders': cHeaders,
'signedHeaders': sHeaders,
'combined': (cHeaders + '\n' + sHeaders + '\n')
}
}
function sortedList(map) {
var list = []
for (name in map) {
list.push({'name': lowercase(name), 'value': trimall(map[name])});
}
list.sort(function(a, b) {
return a.name.localeCompare(b.name);
});
return list;
}
function lowercase(value) {
return value.toLowerCase();
}
function trimall(value) {
return value.trim().replace(new RegExp(/\s+/gi), " ");
}
sortedList
method fortrimall
method for thatfunction hashAndHexEncode(requestBody) {
var sha256 = crypto.createHash("sha256");
var hashObject = sha256.update(requestBody || '', 'utf8');
var hex = hashObject.digest('hex').toLowerCase();
return hex;
}
crypto
library to create a hash using the sha256
algorithmhex
digest and converting all characters to lowercaseThe algorithm (in pseudocode) for creating a string to sign is:
StringToSign =
Algorithm + \n +
RequestDateTime + \n +
CredentialScope + \n +
HashedCanonicalRequest
The createStringToSign
method follows this algorithm very closely, but the return object of this method contains two fields, one the string to sign and the other the credential scope, because we will need both fields later.
function createStringToSign(options, canonicalRequestHash, region, service) {
var stringToSign = '';
stringToSign += 'AWS4-HMAC-SHA256' + '\n' // corresponds to SHA256
stringToSign += options.headers[AMAZON_DATE_HEADER] + '\n'
var credentialsScope = createCredentialsScope(options, region, service)
stringToSign += credentialsScope + '\n'
stringToSign += canonicalRequestHash;
return {
'stringToSign': stringToSign,
'credentialsScope': credentialsScope
}
}
function createCredentialsScope(options, region, service) {
var date = options.headers[AMAZON_DATE_HEADER].substring(0, 8);
return date + '/' + region + '/' + service + '/aws4_request'
}
aws4_request
string (we are implementing teh version 4 signature, there is a version 2 signing process as well).The signing key is created by following this algorithm:
kSecret = secret access key
kDate = HMAC("AWS4" + kSecret, Date)
kRegion = HMAC(kDate, Region)
kService = HMAC(kRegion, Service)
kSigning = HMAC(kService, "aws4_request")
aws4_request
string with the service keyfunction createSigningKey(options, secret, region, service) {
var date = options.headers[AMAZON_DATE_HEADER].substring(0, 8);
var kDate = HMAC("AWS4" + secret, date)
var kRegion = HMAC(kDate, region)
var kService = HMAC(kRegion, service)
var kSigning = HMAC(kService, "aws4_request")
return kSigning
}
function HMAC(original, update) {
return crypto.createHmac('sha256', original).update(update).digest();
}
createSigningKey
method follows every step of this algoritmHMAC
method takes the original
, the key, and the update
, the value that is getting encodedHMAC
method returns a binary digest of the key (this is a buffer).digest('binary')
, which does not work because binary
is not an accepted input for that method; this is why I ended up implementing the whole process from scratchWe now have all we need to compute the signature, with the calculateSignature
method. We are signing the string to sign with the signing key we obtained before:
function calculateSignature(signingKey, stringToSign) {
return crypto.createHmac('sha256', signingKey).update(stringToSign).digest('hex').toLowerCase();
}
function createAuthorizationHeader(accessKeyID, credentialsScope, signedHeaders, signature) {
var authorization = '';
authorization += 'AWS4-HMAC-SHA256' + ' ';
authorization += 'Credential=' + accessKeyID + '/' + credentialsScope + ', ';
authorization += 'SignedHeaders=' + signedHeaders + ', '
authorization += 'Signature=' + signature
return authorization;
}
AWS4-HMAC-SHA256
accessKeyID
and the credentials scope we obtained in the createStringToSign
methodtoCanonicalRequestAndHash
methodcalculateSignature
methodsignRequest
method is also adding the value computed by createAuthorizationHeader
to the request options, under the Authorization
header nameLast thing we need to do is export the signRequest
method and we can use our library to make signed calls to API Gateway:
exports.signRequest = signRequest
In the test.js I created another method, loadFormsWithAuth
that will use our library to sign the request before sending it to API Gateway:
var signature = require('./signature.js');
function loadFormsWithAuth(callback) {
var options = {
method: 'GET',
host: 'api gateway url',
path: '/cognitotest/forms'
};
// build the signature
var accessKeyId = 'access key id'
var secretKey = 'secret key'
var sessionToken = 'session token'
signature.signRequest(options, '', accessKeyId, secretKey, 'region', 'execute-api', sessionToken);
var request = https.request(options, collecter(callback));
request.end()
}
loadFormsWithAuth(console.log);
accessKeyID
, secretKey
and sessionToken
values are hardcoded here, I explain how to obtain them in the Cognito federated identities documentexecute-api
Running the code, you should now have access to the secured API and see the following output on the console:
{ fields:
[ { name: 'family name', value: '' },
{ name: 'first name', value: '' },
{ name: 'address', value: '' },
{ name: 'postcode', value: '' } ] }