Issue with download and writing of file using ready read

[ejoshva]

I am trying to download a file from server and write onto a file using readyRead() of QNetworkReply .
Since I am facing problem in keeping the whole file say 600Mb in buffer and writing in one go as Windows doesn't allow buffer data more than 150Mb.

bookdownloader::bookdownloader(QUrl url, QObject *parent,QString zipFilePath) : QObject(parent)
{
    qb.setWindowTitle("Download in Progress");
    progressDialog = new QProgressDialog("Preparing to Download", "Cancel", 0, 100);
    progressDialog->setWindowTitle("Downloading");
    progressDialog->show();
    QNetworkAccessManager *m_WebCtrl = new QNetworkAccessManager(this);
 
    setFilePath(zipFilePath);
    file.setFileName(this->getFilePath());
 
    downloadedDataSize = 0;
    QByteArray postData;
 
    QNetworkRequest request(url);
    request.setHeader(QNetworkRequest::ContentTypeHeader,  "application/x-www-form-urlencoded");
 
    reply = m_WebCtrl->get(request);
 
    connect(reply,SIGNAL(readyRead()),this,SLOT(slotReadData()));
    connect(reply,SIGNAL(finished()), this,SLOT(fileDownloaded()));
    connect(reply, SIGNAL(downloadProgress(qint64,qint64)),this, SLOT(downloadprogress(qint64,qint64)));
}
 
void bookdownloader::fileDownloaded()
{    
    progressDialog->close();
    emit downloaded();
}
 
void bookdownloader::downloadprogress(qint64 recieved, qint64 total)
{
    if(total < 1)
        return;
 
    progressDialog->setValue((recieved*100/total));
 
    if (progressDialog->wasCanceled())
    {
        progressDialog->close();
        m_WebCtrl->blockSignals(true);
    }
 
    progressDialog->setLabelText(QString::number((recieved*100/total))+"%");
 
}
 
void bookdownloader::slotReadData()
{
    file.setFileName(this->getFilePath());
    if(!file.open(QIODevice::WriteOnly| QIODevice::Append))
    {
        QMessageBox::critical(NULL,"Hello!","Error saving file!");
        return;
    }
    QDataStream out(&file);
    out << reply->readAll();
    file.close();
 
    downloadedDataSize += file.size();
}

bookdownloader::bookdownloader(QUrl url, QObject *parent,QString zipFilePath) : QObject(parent)
{
    qb.setWindowTitle("Download in Progress");
    progressDialog = new QProgressDialog("Preparing to Download", "Cancel", 0, 100);
    progressDialog->setWindowTitle("Downloading");
    progressDialog->show();
    QNetworkAccessManager *m_WebCtrl = new QNetworkAccessManager(this);

    setFilePath(zipFilePath);
    file.setFileName(this->getFilePath());

    downloadedDataSize = 0;
    QByteArray postData;

    QNetworkRequest request(url);
    request.setHeader(QNetworkRequest::ContentTypeHeader,  "application/x-www-form-urlencoded");

    reply = m_WebCtrl->get(request);

    connect(reply,SIGNAL(readyRead()),this,SLOT(slotReadData()));
    connect(reply,SIGNAL(finished()), this,SLOT(fileDownloaded()));
    connect(reply, SIGNAL(downloadProgress(qint64,qint64)),this, SLOT(downloadprogress(qint64,qint64)));
}

void bookdownloader::fileDownloaded()
{    
    progressDialog->close();
    emit downloaded();
}

void bookdownloader::downloadprogress(qint64 recieved, qint64 total)
{
    if(total < 1)
        return;

    progressDialog->setValue((recieved*100/total));

    if (progressDialog->wasCanceled())
    {
        progressDialog->close();
        m_WebCtrl->blockSignals(true);
    }

    progressDialog->setLabelText(QString::number((recieved*100/total))+"%");

}

void bookdownloader::slotReadData()
{
    file.setFileName(this->getFilePath());
    if(!file.open(QIODevice::WriteOnly| QIODevice::Append))
    {
        QMessageBox::critical(NULL,"Hello!","Error saving file!");
        return;
    }
    QDataStream out(&file);
    out << reply->readAll();
    file.close();

    downloadedDataSize += file.size();
}

To copy to clipboard, switch view to plain text mode 

Once the download and writing onto file is complete, when I try to manually unzip the file error thrown as "file is corrupt"

In the extracter code when I try to find the list in the zip file, it's 0.

Kindly let me know what's going wrong in the download and writing onto file part by part.

---

Added after 40 minutes:

Got it working with the code below

void bookdownloader::slotReadData()
{
    file.setFileName(this->getFilePath());
    file.open(QIODevice::Append);
    file.write(reply->read(reply->bytesAvailable()));
    file.close();
}

void bookdownloader::slotReadData()
{
    file.setFileName(this->getFilePath());
    file.open(QIODevice::Append);
    file.write(reply->read(reply->bytesAvailable()));
    file.close();
}

To copy to clipboard, switch view to plain text mode 

[ChrisW67]

You do not want to be using QDataStream to write the raw binary information into your file. You are giving the QDataStream a series of QByteArrays, and it serialises those as a size value (4 bytes IIRC) before the raw data it contains. Your resulting file will be larger than the source file with "random" bytes inserted between the blocks of bytes you actually wanted.

Simply call file->write() with the QByteArray you get from readAll().

As an aside:

• You are appending to a file that this code does not ensure is empty.
• Repeatedly opening and closing the file takes time and does not achieve a great deal. Open it once when you start the transfer and close it when the transfer finishes.
• The transfer can finish because of an error you do not check for. What should happen to the target file in that circumstance?