QtConcurrent, i need advice

[SABROG]

I have a cycle what iterate 4294967295 times ((quint32)-1). And i want use QtConcurrent for use both cores. But i need use QProgressDialog who change progress each 1% (42949672,95). One thread calculate range 0-2147483647, secons thread 2147483648-4294967295. So i need create signal like step() what i can emit from each thread on each percernt operation. I need also have feature uses Cancel button on QProgressDialog what will be stop both threads. Can you get me some advice how i can do this? I think i must create QObject's, declare bool flag like isCanceled what check inside for(){} and using QtConcurrent::run(). I can't use QtConcurrent::map becouse 4294967295*4= ~17Gb RAM excluding container overhead. And i think i can't use QEventLoop inside thread like in QThread::run(){exec()} for respond on signals from main thread, becouse i use QtConcurrent instead QThread.

[wysota]

Have a look at QFutureWatcher api. I would use QtConcurrent::mapped() not ::run(). You can break your problem into subproblems to avoid problems with memory consumption.

[SABROG]

You can break your problem into subproblems to avoid problems with memory consumption.

If i reserve ~20Mb this about 5242880 elements in each container and 819 calling QtConcurrent::mapped(). This is not overhead?
---
I create class based on QObject and emit signal step() inside thread

for (int i = 0, j = count/100*1; i < count; ++i, ++start) {
        if (!isCanceled()) {
            if (i % j) {
                emit step();
            }
        } else {
            break;
        }
    }
    deleteLater();
    return 0;

for (int i = 0, j = count/100*1; i < count; ++i, ++start) {
        if (!isCanceled()) {
            if (i % j) {
                emit step();
            }
        } else {
            break;
        }
    }
    deleteLater();
    return 0;

To copy to clipboard, switch view to plain text mode 

If connection QueuedConnection my QProgressDialog is freezed, if DirectConnection - all alright. Why?

[wysota]

Compared to what?

[SABROG]

Compared to what?

Compared to solid cycle with signals.

[wysota]

There will be little overhead with memory footprint but you should get better (or comparable) overall performance.

[SABROG]

Anyway i'll do it with custom class based on QObject and my signals from thread working.

for (int i = 0; i < idealThreadCount; ++i) {
                MyClass *myclass = new MyClass;
                connect(progress.data(), SIGNAL(canceled()), myclass, SLOT(cancel()));
                connect(myclass, SIGNAL(step()), this, SLOT(stepChanged()));
                QFuture <void> future = QtConcurrent::run(myclass, &MyClass::run, offset, blockSize, (quint32)-1); 
//there is need place QFuture to outside container

for (int i = 0; i < idealThreadCount; ++i) {
                MyClass *myclass = new MyClass;
                connect(progress.data(), SIGNAL(canceled()), myclass, SLOT(cancel()));
                connect(myclass, SIGNAL(step()), this, SLOT(stepChanged()));
                QFuture <void> future = QtConcurrent::run(myclass, &MyClass::run, offset, blockSize, (quint32)-1); 
//there is need place QFuture to outside container

To copy to clipboard, switch view to plain text mode 

qint32 MyClass::run(quint32 start, qint32 count, quint32 total)
{
    for (int i = 0, j = (total/100)*1; i < count; ++i, ++start) {
        if (!isCanceled()) {
            if (i % j == 0) {
                emit step();
                qDebug() << j << i << start << count << QThread::currentThreadId() << thread();
            }
        } else {
            break;
        }
    }
    deleteLater();
    return 0;
}

qint32 MyClass::run(quint32 start, qint32 count, quint32 total)
{
    for (int i = 0, j = (total/100)*1; i < count; ++i, ++start) {
        if (!isCanceled()) {
            if (i % j == 0) {
                emit step();
                qDebug() << j << i << start << count << QThread::currentThreadId() << thread();
            }
        } else {
            break;
        }
    }
    deleteLater();
    return 0;
}

To copy to clipboard, switch view to plain text mode 

There is another problem, how to distribute count of operation on cores if devide operation on cores don't get integer without remainder. So i need somehow distribute this remainder on operations.

[wysota]

It's better to have a queue filled with input data which the threads will read. Just don't place all the data into the queue at once, it won't fit there. You can implement it with an extra thread acting as a producer with wait conditions acting as guards over the data buffer.

[SABROG]

I don't need only one thread, becouse i need do calculation on all cores synchronously. How i say before, one thread calculate 0-10 range, second thread 11-20 range. So i need two producers? For each thread?

[wysota]

I meant two calculating threads plus an extra thread acting as a producer for the queue feeding both calculators. The producer sleeps until the queue runs out of data and is signaled (using a wait condition) to produce more data before going to sleep again.

A rough code for the producer:

QQueue<int> buffer;
QWaitCondition condition;
QMutex mutex;
 
class Producer : public QThread {
public:
  Producer() : QThread() { m_last = 0; }
  void run() {
    while(m_last<MAX_INT){
      mutex.lock();
      produce();
      condition.wait(&mutex);
    }
  }
  void produce() {
    for(int i=0;i<65535;i++) queue.enqueue(++m_last);
  }
private:
  int m_last;
};

QQueue<int> buffer;
QWaitCondition condition;
QMutex mutex;

class Producer : public QThread {
public:
  Producer() : QThread() { m_last = 0; }
  void run() {
    while(m_last<MAX_INT){
      mutex.lock();
      produce();
      condition.wait(&mutex);
    }
  }
  void produce() {
    for(int i=0;i<65535;i++) queue.enqueue(++m_last);
  }
private:
  int m_last;
};

To copy to clipboard, switch view to plain text mode 

[SABROG]

I solved my task himself, with signals and slots and using QtConcurrent::run().