Posting images to AWS S3 with QNetworkAccessManager PUT (Android)

[marinadel]

I am trying to get my android app (qt5 c++) to upload taken pictures to an amazon bucket using PUT method but keep getting errors about calculated signature not matching the one provided.

here's my code:

QFile * fileData = new QFile(path, this);
  if (fileData->open(QIODevice::ReadOnly))
  {
    QFileInfo fileInfo(fileData->fileName());
 
    QByteArray AccessKey = "SOMEACCESKEY";
    QByteArray SecretKey = "SOMESECRETKEY";
    QByteArray DayOfWeek = QDateTime::currentDateTime().date().toString("ddd").toUtf8();
    QByteArray RFCDate = DayOfWeek + ", " + QDateTime::currentDateTime().toString(Qt::RFC2822Date).toUtf8();
    QByteArray S3Bucket = "somes3bucket";
    QByteArray S3region = "us-east-1";
    QByteArray ContentType = "image/jpeg";
    QByteArray StringToSign = "PUT\\n\\n" + ContentType + "\\n" + RFCDate + "\\n" + "/" + S3Bucket + "/" + fileInfo.fileName().toUtf8();
 
    QByteArray SignString = QMessageAuthenticationCode::hash(StringToSign, SecretKey, QCryptographicHash::Sha1).toBase64();
 
    QNetworkAccessManager * networkAccessManager = new QNetworkAccessManager(this);
 
    QNetworkRequest request(QUrl("https://" + S3Bucket + ".s3." + S3region + ".amazonaws.com/" + fileInfo.fileName()));
    request.setHeader(QNetworkRequest::ContentTypeHeader, ContentType);
    request.setHeader(QNetworkRequest::ContentLengthHeader, fileInfo.size());
    request.setRawHeader("Host", S3Bucket + ".s3." + S3region + ".amazonaws.com");
    request.setRawHeader("Date", RFCDate);
    request.setRawHeader("Authorization", "AWS " + AccessKey + ":" + SignString);
 
    m_reply = networkAccessManager->put(request, fileData);
    connect(m_reply, SIGNAL(finished()), this, SLOT(mySlot()));
}

QFile * fileData = new QFile(path, this);
  if (fileData->open(QIODevice::ReadOnly))
  {
    QFileInfo fileInfo(fileData->fileName());

    QByteArray AccessKey = "SOMEACCESKEY";
    QByteArray SecretKey = "SOMESECRETKEY";
    QByteArray DayOfWeek = QDateTime::currentDateTime().date().toString("ddd").toUtf8();
    QByteArray RFCDate = DayOfWeek + ", " + QDateTime::currentDateTime().toString(Qt::RFC2822Date).toUtf8();
    QByteArray S3Bucket = "somes3bucket";
    QByteArray S3region = "us-east-1";
    QByteArray ContentType = "image/jpeg";
    QByteArray StringToSign = "PUT\\n\\n" + ContentType + "\\n" + RFCDate + "\\n" + "/" + S3Bucket + "/" + fileInfo.fileName().toUtf8();

    QByteArray SignString = QMessageAuthenticationCode::hash(StringToSign, SecretKey, QCryptographicHash::Sha1).toBase64();

    QNetworkAccessManager * networkAccessManager = new QNetworkAccessManager(this);

    QNetworkRequest request(QUrl("https://" + S3Bucket + ".s3." + S3region + ".amazonaws.com/" + fileInfo.fileName()));
    request.setHeader(QNetworkRequest::ContentTypeHeader, ContentType);
    request.setHeader(QNetworkRequest::ContentLengthHeader, fileInfo.size());
    request.setRawHeader("Host", S3Bucket + ".s3." + S3region + ".amazonaws.com");
    request.setRawHeader("Date", RFCDate);
    request.setRawHeader("Authorization", "AWS " + AccessKey + ":" + SignString);

    m_reply = networkAccessManager->put(request, fileData);
    connect(m_reply, SIGNAL(finished()), this, SLOT(mySlot()));
}

To copy to clipboard, switch view to plain text mode 

here's the error msg I am getting when reading m_reply->readAll():

<Code>SignatureDoesNotMatch</?Code>
<Message>The request signature we calculated does not match the signature you provided.? Check your key and signing method.?</?Message>


any help is much appreciated!

[d_stranz]

Do you have access to the signature stored with AWS? Is it in fact different from the one you are sending? Have you carefully checked your access code and secret keys to make sure there aren't any typos?

Are there byte order (endian) differences between android architecture and the order expected by AWS (presumably internet byte order, big endian)? Take a look at QtEndian if so.

[marinadel]

the error message provides <StringToSign> and <SignatureProvided> fields, I believe <StringToSign> is auto calculated based on the header parameters I send and it matches the one I build (StringToSign in the code), however when I use online hmac calculators (like https://dinochiesa.github.io/hmachash/index.html for example) and enter that string, my secret key and select sha1 function the base64 output doesn't match my signed string for some reason. does it mean that QMessageAuthenticationCode::hash(StringToSign, SecretKey, QCryptographicHash::Sha1).toBase64() doesn't work as expected?

[d_stranz]

Network byte order is big-endian. Android is little-endian, so it is possible that your hash() call is generating a little-endian QByteArray. So try this:

#include <QtEndian>
 
QByteArray AndroidTemp = QMessageAuthenticationCode::hash(StringToSign, SecretKey, QCryptographicHash::Sha1);
QByteArray AndroidString = AndroidTemp.toBase64();  // for comparison purposes
 
QByteArray SignTemp( AndroidString );  // copy so the two arrays are the same size
qToBigEndian( &AndroidTemp, AndroidTemp.size(), &SignTemp );
 
QByteArray SignString = SignTemp.toBase64();

#include <QtEndian>

QByteArray AndroidTemp = QMessageAuthenticationCode::hash(StringToSign, SecretKey, QCryptographicHash::Sha1);
QByteArray AndroidString = AndroidTemp.toBase64();  // for comparison purposes

QByteArray SignTemp( AndroidString );  // copy so the two arrays are the same size
qToBigEndian( &AndroidTemp, AndroidTemp.size(), &SignTemp );

QByteArray SignString = SignTemp.toBase64();

To copy to clipboard, switch view to plain text mode 

Are SignString and AndroidString the same? If so, then hash() is generating a big-endian array. If not, then it is probably little-endian.
Try sending this version of SignString to AWS

[marinadel]

Network byte order is big-endian. Android is little-endian, so it is possible that your hash() call is generating a little-endian QByteArray. So try this:

#include <QtEndian>
 
QByteArray AndroidTemp = QMessageAuthenticationCode::hash(StringToSign, SecretKey, QCryptographicHash::Sha1);
QByteArray AndroidString = AndroidTemp.toBase64();  // for comparison purposes
 
QByteArray SignTemp( AndroidString );  // copy so the two arrays are the same size
qToBigEndian( &AndroidTemp, AndroidTemp.size(), &SignTemp );
 
QByteArray SignString = SignTemp.toBase64();

#include <QtEndian>

QByteArray AndroidTemp = QMessageAuthenticationCode::hash(StringToSign, SecretKey, QCryptographicHash::Sha1);
QByteArray AndroidString = AndroidTemp.toBase64();  // for comparison purposes

QByteArray SignTemp( AndroidString );  // copy so the two arrays are the same size
qToBigEndian( &AndroidTemp, AndroidTemp.size(), &SignTemp );

QByteArray SignString = SignTemp.toBase64();

To copy to clipboard, switch view to plain text mode 

Are SignString and AndroidString the same? If so, then hash() is generating a big-endian array. If not, then it is probably little-endian.
Try sending this version of SignString to AWS

thank you for your reply - turned out my issue with signature was caused by the double slashes in StringToSign so replacing \\n with \n solved it - I am not getting any more errors but m_reply->readAll() now returns an empty string, no errors are reported by errorOccurred signal and nothing gets posted to s3. so not sure what is wrong now

[d_stranz]

I am not getting any more errors but m_reply->readAll() now returns an empty string

From what I read in the docs about QNetworkReply, the finished() signal is emitted after the put() request is completely done. So by the time you see that, all of the reply has been sent and there is nothing more to read.

I think what you need to do is the handle the QIODevice::readyRead() signal and in your slot retrieve all of the pending data and put it onto the end of a buffer. Keep doing that until you receive the finished() signal, at which point you know you have received everything the server is going to send.