Event Queue Stop during download

[bmn]

Hello,

I have a file synchronisation class which is responsible to sync files between an local directory and a respository. Within this class I am downloading files.
The problem I am facing is, that the application seems to stop/hang in the event queue while the file is downloading (after that it is proceeding and works as usual).

I used the default setting for connect, Qt::AutoConnection which seems to use a QueuedConnection when used with threads to add it to the threads event loop.
The network is managed with QNetworkAccessManager (_nam).

Signals/slots for all the operations (within the downloader):

_reply = _nam->get(QNetworkRequest(url));
connect(_reply, SIGNAL(finished()), this, SLOT(httpFinished()));
connect(_reply, SIGNAL(readyRead()), this, SLOT(httpReadyRead()));
connect(_reply, SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(httpDownloadProgress(qint64,qint64)));

_reply = _nam->get(QNetworkRequest(url));
connect(_reply, SIGNAL(finished()), this, SLOT(httpFinished()));
connect(_reply, SIGNAL(readyRead()), this, SLOT(httpReadyRead()));
connect(_reply, SIGNAL(downloadProgress(qint64,qint64)), this, SLOT(httpDownloadProgress(qint64,qint64)));

To copy to clipboard, switch view to plain text mode 

I tried to shift the the class (fSync contains the downloader class) in another thread:

fSync = new FSyncClass();
fSync->moveToThread(_thread);
connect(_thread, SIGNAL(finished()), _thread, SLOT(deleteLater()));
connect(_thread, SIGNAL(finished()), fSync, SLOT(deleteLater()));
_thread->start();

fSync = new FSyncClass();
fSync->moveToThread(_thread);
connect(_thread, SIGNAL(finished()), _thread, SLOT(deleteLater()));
connect(_thread, SIGNAL(finished()), fSync, SLOT(deleteLater()));
_thread->start();

To copy to clipboard, switch view to plain text mode 

GDB shows:

Main Thread-ID: 0xb6781000           //I call currentThreadId() to get this debug output within the main event loop
[New Thread 0xb674b450 (LWP 1254)]        //gdb output
[New Thread 0xb5dff450 (LWP 1255)]         //gdb output
[New Thread 0xb53ff450 (LWP 1256)]         //gdb output
fSync Thread-ID: 0xb5dff450                     //I call currentThreadId() to get this debug output within the class moved to the thread

Main Thread-ID: 0xb6781000           //I call currentThreadId() to get this debug output within the main event loop
[New Thread 0xb674b450 (LWP 1254)]        //gdb output
[New Thread 0xb5dff450 (LWP 1255)]         //gdb output
[New Thread 0xb53ff450 (LWP 1256)]         //gdb output
fSync Thread-ID: 0xb5dff450                     //I call currentThreadId() to get this debug output within the class moved to the thread

To copy to clipboard, switch view to plain text mode 

So it seems like the class is really running in a separate thread.


I am running out of ideas and would appreciate any help.

Thank you in advance.

[anda_skoa]

There is not enough information to go on.

Is the first code snippet part of the FSyncClass?
Is _nam a QObject child of "this"?
Is _thread a plain QThread or a custom subclass? If the latter, does it call exec()?

What does the main thread do after it called _thread->start()?

Cheers,
_

[bmn]

Sorry for my poor explanation:

The structure of the class is:

-> FSyncClass()
    //This is within the FSyncClass
    -> _nam = new QNetworkAccessManager(this)
    -> ... = DownloaderClass(_nam, this);                            //contains the first code snippet

-> FSyncClass()
    //This is within the FSyncClass
    -> _nam = new QNetworkAccessManager(this)
    -> ... = DownloaderClass(_nam, this);                            //contains the first code snippet

To copy to clipboard, switch view to plain text mode 

The call of the FSyncClass (second code snippet) happens within the main thread.

_thread->start() is not reimplemented, if you mean that.
From the documentation ( start() calls run() calls exec() ) it should start an event queue.


Thank you.

[anda_skoa]

Sorry for my poor explanation:

The structure of the class is:

-> FSyncClass(... *parent)
    //This is within the FSyncClass
    -> _nam = new QNetworkAccessManager(parent)
    -> ... = DownloaderClass(_nam, parent);                            //contains the first code snippet

-> FSyncClass(... *parent)
    //This is within the FSyncClass
    -> _nam = new QNetworkAccessManager(parent)
    -> ... = DownloaderClass(_nam, parent);                            //contains the first code snippet

To copy to clipboard, switch view to plain text mode 

You mean "this" here instead of "parent"?
Or are you passing the parent you get in the FSyncClass constructor?
Because that looks like to be the object that remains in the main thread, which would mean your QNAM and downloader are also on the main thread.

_thread->start() is not reimplemented, if you mean that.
From the documentation ( start() calls run() calls exec() ) it should start an event queue.

Right, if it is a plain QThread then its run() will call exec().

Does your main thread return to the event loop after it has created and started the second thread?

Cheers,
_

[bmn]

You mean "this" here instead of "parent"?
Or are you passing the parent you get in the FSyncClass constructor?
Because that looks like to be the object that remains in the main thread, which would mean your QNAM and downloader are also on the main thread.

This was my fault. I copied the snippet from the implementation and not from the actual call. There is no pointer in the FSyncClass constructor call:

fSync = new so_FileSync();

fSync = new so_FileSync();

To copy to clipboard, switch view to plain text mode 

I also edited my posts from above.

Does your main thread return to the event loop after it has created and started the second thread?

Yes I think so. Otherwise the main thread would stop execution after the call (which is not the case), right?

[anda_skoa]

This was my fault. I copied the snippet from the implementation and not from the actual call. There is no pointer in the FSyncClass constructor call:

fSync = new so_FileSync();

fSync = new so_FileSync();

To copy to clipboard, switch view to plain text mode 

I also edited my posts from above.

Ah, yes, that looks ok.

Yes I think so. Otherwise the main thread would stop execution after the call (which is not the case), right?

Well you originally said it does not?

the application seems to stop/hang in the event queue while the file is downloading

Have you tried running a timer on the main thread with a simple slot that just writes some qDebug() and see if that still happens?

Cheers,
_

[bmn]

Well you originally said it does not?

The main event loop runs fine till i start a download.
I create the thread -> shift the FSyncClass to the thread -> the main event loop runs fine, till the call to start downloading occurs. Thats where the execution seems to stop.

Have you tried running a timer on the main thread with a simple slot that just writes some qDebug() and see if that still happens?

I just tried, the qDebug output runs fine till the download starts. Then it stops.

[anda_skoa]

This is weird, even without a thread this should not block.

How do you trigger the download?
Is the QNAM::get() call in the Downloader constructor?
Do you do any nested event loop in Downloader?
What if you run it with a single thread?

Cheers,
_

[bmn]

How do you trigger the download?

_reply = _nam->get(QNetworkRequest(url));

_reply = _nam->get(QNetworkRequest(url));

To copy to clipboard, switch view to plain text mode 

Is the QNAM::get() call in the Downloader constructor?

This call is within a separate method within the Downloader class, not in the constructor.

Do you do any nested event loop in Downloader?

No, I used the QNAM together with the QNetworkRequest to start a download request.
The whole process should be asynchronous since i use signals/slots. The connections are the one from code snippet 1 of the original post.

What if you run it with a single thread?

Thats what I did in the beginning, since I thought (still think) signals/slots prevent this behaviour. The output is exactly the same as if I use a second thread.


I am beginning to think that this may have something to do with limited resources I am facing...
The core application is running on an custom embedded system running linux.
Anyway the Cortex A9 dual core processor is bored most of the time.

Here is the output of top while I am downloading:

Mem: 486304K used, 546704K free, 13256K shrd, 4632K buff, 414664K cached
CPU:   6% usr   2% sys   0% nic  47% idle  43% io   0% irq   0% sirq
Load average: 0.40 0.11 0.08 2/66 2471
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
 2254     1 root     S      98m  10%   8% /opt/so/bin/myApplication

Mem: 486304K used, 546704K free, 13256K shrd, 4632K buff, 414664K cached
CPU:   6% usr   2% sys   0% nic  47% idle  43% io   0% irq   0% sirq
Load average: 0.40 0.11 0.08 2/66 2471
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
 2254     1 root     S      98m  10%   8% /opt/so/bin/myApplication

To copy to clipboard, switch view to plain text mode 

I do not really see an issue here...

I appreciate your help, thank you.

[anda_skoa]

_reply = _nam->get(QNetworkRequest(url));

_reply = _nam->get(QNetworkRequest(url));

To copy to clipboard, switch view to plain text mode 

That's how the download is started, but what triggers that?

This call is within a separate method within the Downloader class, not in the constructor.

How and from where is that method being called

No, I used the QNAM together with the QNetworkRequest to start a download request.
The whole process should be asynchronous since i use signals/slots.

Ok, good!

Thats what I did in the beginning, since I thought (still think) signals/slots prevent this behaviour.

I would definitely go for single threaded unless you really, really, need a thread.
Removes a whole lot of potential problem sources.

I am beginning to think that this may have something to do with limited resources I am facing...
The core application is running on an custom embedded system running linux.
Anyway the Cortex A9 dual core processor is bored most of the time.

Sounds unlikely, especially in the single threaded case.

Cheers,
_

[bmn]

That's how the download is started, but what triggers that?
How and from where is that method being called

I am using signals/slots to communicate across thread boundaries (eg. from the main thread to the FSyncClass thread).
The classes within the FSyncClass thread then also use normal method calls too.


I am a little lost here...
Hope you have some more ideas.