Advise sought on QEventLoop behavior

[TorAn]

Per Wysota's example of the QConcurrent-based solution I am now looking at QFuture usage. I don't understand how to implement following scenario using this framework.

My application processes various data files and submits it to the website for storage. Each data file comes from a separate provider. So, in respect to QConcurrent, I can describe operation for data processing using functor that I can pass to QtConcurrent::map. But this instance has to make multiple requests to the website via webservice, wait for the response and then decide what to do next - either stop or continue to send data until it finishes. If more data is available it makes another request to send data and so on.

So, within this function object I have to implement waiting mechanism. Is my scenario a good candidate for QConcurrent or I should stick with direct threads?

[wysota]

Why do you want threads there at all? You can do all that in a single thread.

[TorAn]

There is more to the use case that make threading a right choice. What I don't understand is if QtConcurrent provieds support beyond the sequential tasks.

[wysota]

There is more to the use case that make threading a right choice.

Name one thing, please.

What I don't understand is if QtConcurrent provieds support beyond the sequential tasks.

QtConcurrent doesn't care about the nature of tasks it executes.

[TorAn]

For example, data acquisition processes for individual datasources can take from 3 min to 2 hours, depending on the datasource. There around 10 datasources that have to be processed, starting at the same time.

I understand that QtConcurrent can process any task. But when the task that is processed is event or signal driven I have to organize it in the task instance and that brings back eventloop into the task.

[wysota]

For example, data acquisition processes for individual datasources can take from 3 min to 2 hours, depending on the datasource. There around 10 datasources that have to be processed, starting at the same time.

So why is it useful to do it in threads? All your threads will be doing most of the time is waiting for data to flow in. One thread is enough to handle all that without any effort.

I understand that QtConcurrent can process any task. But when the task that is processed is event or signal driven I have to organize it in the task instance and that brings back eventloop into the task.

Obviously, if you need events, you need an event loop. But you don't need an event loop for signals.

If I understand correctly what you are trying to do, then you are way overcomplicating things. The best strategy is to acquire the data from network (or whatever outside source you're using) in a single thread for all the sources and then process the data using QtConcurrent. This gives you the simplest and the most efficient code.

[TorAn]

Wysota,

I very much appreciate your review. I agree with the approach you suggested, part of it is actually implemented in the production code. Network communication is shared between threads, only data processing happends individually in a separate threads. All the reasons to keep it that way are relatively weak and I should redo the implementation using one thread for network and another for data processing.

Per the results of my use case sample I already made the transition from QRunnable/QEventLoop to QThread. It required a miniscule change and everything works now as expected. However, in the next round of development I will make the next transition, from separate QThreads to one QThread and QConcurrent maps.

Your advise and support is very helpful, thanks again.

[wysota]

However, in the next round of development I will make the next transition, from separate QThreads to one QThread and QConcurrent maps.

No, no QThread. The main thread will do just fine, you don't need an extra thread for networking.