Results 1 to 3 of 3

Thread: "Too many open files" error

  1. #1
    Join Date
    Aug 2007
    Posts
    244
    Thanks
    42
    Thanked 8 Times in 8 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default "Too many open files" error

    Hi, I got the error in the title running a class derived from QNetworkAccessManager to download and save files. It works fine except when requesting a large number of file (600~). Here the implementation (quite common):

    Qt Code:
    1. FileDownloader::FileDownloader(QObject *parent) :
    2. QNetworkAccessManager(parent)
    3. {
    4. }
    5.  
    6. FileDownloader::~FileDownloader()
    7. {
    8. deleteLater();
    9. }
    10.  
    11. void FileDownloader::setPath(const QString &path)
    12. {
    13. mPath = path;
    14. }
    15.  
    16. void FileDownloader::setFileName(const QString &fileName)
    17. {
    18. mFileName = fileName;
    19. }
    20.  
    21. void FileDownloader::setUrl(const QString &url)
    22. {
    23. mUrl = url;
    24. }
    25.  
    26. void FileDownloader::download()
    27. {
    28. file = new QFile;
    29. file->setFileName(mPath + QDir::separator() + mFileName);
    30.  
    31. if(file->exists())
    32. if(!file->remove()) {
    33. qDebug() << file->errorString();
    34. return;
    35. }
    36.  
    37. file->open(QIODevice::WriteOnly);
    38.  
    39. QNetworkRequest request;
    40. request.setUrl(QUrl(mUrl));
    41. reply = get(request);
    42.  
    43. connect(reply, SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(onDownloadProgress(qint64,qint64)));
    44. connect(this, SIGNAL(finished(QNetworkReply*)), this, SLOT(onFinished(QNetworkReply*)));
    45. connect(reply, SIGNAL(readyRead()), this, SLOT(onReadyRead()));
    46. connect(reply, SIGNAL(finished()), this, SLOT(onReplyFinished()));
    47. }
    48.  
    49.  
    50. void FileDownloader::onFinished(QNetworkReply *reply)
    51. {
    52. switch(reply->error()) {
    53. case QNetworkReply::NoError:
    54. qDebug() << "Downloaded:" << file->fileName();
    55. break;
    56.  
    57. default:
    58. qDebug(reply->errorString().toLatin1());
    59. }
    60.  
    61. if(file->isOpen()) {
    62. file->close();
    63. file->deleteLater();
    64. }
    65. }
    66.  
    67. void FileDownloader::onReadyRead()
    68. {
    69. file->write(reply->readAll());
    70. }
    71.  
    72. void FileDownloader::onReplyFinished()
    73. {
    74. if(file->isOpen()) {
    75. file->close();
    76. file->deleteLater();
    77. }
    78. }
    79.  
    80. void FileDownloader::onDownloadProgress(qint64, qint64)
    81. {
    82. qDebug(QString::number(bytesRead).toLatin1() + " - " + QString::number(bytesTotal).toLatin1());
    83. }
    To copy to clipboard, switch view to plain text mode 

    I'm calling it this way:

    Qt Code:
    1. foreach(QString file, files) {
    2. FileDownloader * downloader = new FileDownloader(this);
    3. downloader->setUrl(/*...*/);
    4. downloader->setPath(/*...*/);
    5. downloader->setFileName(/*...*/);
    6. downloader->download();
    7. }
    To copy to clipboard, switch view to plain text mode 

    Searching the web I read that a solution should be QRunnable, but I'm using it in the wrong way:
    * had subclassed QRunnable:
    Qt Code:
    1. class FileDownloader : public QNetworkAccessManager, public QRunnable
    To copy to clipboard, switch view to plain text mode 
    * renamed FileDownloader::download() in FileDownloader::run()
    * modified calling routine:
    Qt Code:
    1. foreach(QString file, files) {
    2. FileDownloader * downloader = new FileDownloader(this);
    3. downloader->setUrl(/*...*/);
    4. downloader->setPath(/*...*/);
    5. downloader->setFileName(/*...*/);
    6. QThreadPool::globalInstance()->start(downloader); // <- new version
    To copy to clipboard, switch view to plain text mode 

    In console I got repetition of the following error:

    QObject: Cannot create children for a parent that is in a different thread.
    (Parent is FileDownloader(0x13297e0), parent's thread is QThread(0x118c420), current thread is QThread(0x13525d0)
    Any help will be very appreciated. Thanks.
    Giuseppe CalÃ

  2. #2
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: "Too many open files" error

    The "Too many open files" error, which you completely fail to mention in the , is because all operating system have a practical limit to the number of simultaneously open files/sockets that you are exceeding. This is partly a result of resource limits in internal tables etc. and partly a defence mechanism against DOS attacks. You are attempting 600+ simultaneous downloads, which strikes me as a very odd thing. You should serialise the downloads into a few streams.
    Last edited by ChrisW67; 29th October 2012 at 00:48.

  3. #3
    Join Date
    Aug 2007
    Posts
    244
    Thanks
    42
    Thanked 8 Times in 8 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: "Too many open files" error

    Solved queuing all downloads in a QQueue and cotrolling the number of simultaneous downloads.
    Giuseppe CalÃ

Similar Threads

  1. Replies: 5
    Last Post: 15th August 2014, 04:04
  2. Replies: 9
    Last Post: 20th May 2010, 10:55
  3. Implementing an "open recently opened files" menu?
    By joelthelion in forum Qt Programming
    Replies: 3
    Last Post: 6th March 2009, 11:49
  4. Translation QFileDialog standart buttons ("Open"/"Save"/"Cancel")
    By victor.yacovlev in forum Qt Programming
    Replies: 4
    Last Post: 24th January 2008, 20:05
  5. QFile Problem~ "Unknow error" in "open(QIODevice::ReadWrite)"
    By fengtian.we in forum Qt Programming
    Replies: 3
    Last Post: 23rd May 2007, 16:58

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.