design pattern for long time operation

[Talei]

Hello.
I don't know if this is right place to write this, so pardon me if it's not.
Problem: perform many operation (like 12k++) one after another (to shorten this time I want to execute each operation as soon as previouse is finished). So in nutshell the same set of functions are performed many times on different set of data.

My implementation:
Currently I do it "wrong way" and not so wrong (maybe a right?) way.

Wrong way using signal/slot mechanism (wrong in terms of logical implementation, not in code itself) - pseudo code:

void myProgrma::myProgrma(){
 connect( this, SIGNAL( jobDone() ), this, SLOT(controlSLOT()));
 myProgrma();
}
void myProgrma::controlSLOT()
{
 if(jobIsDone)
  return;
 else 
  doJobSLOT()
}
 
void myProgrma::doJobSLOT()
{
 //do job here
  emit jobDone();
}

void myProgrma::myProgrma(){
 connect( this, SIGNAL( jobDone() ), this, SLOT(controlSLOT()));
 myProgrma();
}
void myProgrma::controlSLOT()
{
 if(jobIsDone)
  return;
 else 
  doJobSLOT()
}

void myProgrma::doJobSLOT()
{
 //do job here
  emit jobDone();
}

To copy to clipboard, switch view to plain text mode 

As you can see there is one flaw with this implementation with is that for many, like 64k++, jobs (or less depending on what is in doJobSLOT() ) this will fail (AFAIK - please elaborate about this, if my thinking is wrong) due to the exceeding job queue stack size - due to not returning from slots that's leads to queue being full and thus error.

I found out, on my trails and errors path, that using only signal/slot mechanism is a loot more faster then using QTimer solution (QTimer is about 6+ times slower). I guess that's due to the all those checks and overhead of the QTimer class. Basically I'm looking for fastest i9mplementation of this problem.

I know that I can implement solution for this problem using QTimer and timeout() signal, but my question is:
Is there any other way to implement "while/for" loop behaviour (basically above example does those loops) (or AFAIK so called Observer Pattern) in Qt?

Thank you for any advice on this mater.
Best regards.

PS. in pseudo code "jobIsDone" is bool to check if there is any pending job to be done.

[Lykurg]

Hi,

in your pseudo-code the signal and slot connection behave like a normal while loop. So for what reason do you need that slot?

It all sounds to me, that you should use a thread which performs the 12k++ tasks in a simple for-loop that just checks every time for a stop-condition. For communicating with your main program use signals and slots.

[Talei]

Thank you for reply. Your solution is great, and to be honest I thought about it myself but, I didn't see "the whole picture".
Basically my idea was to do some kind of Controller to manage 12k+ task and divide it for X thread (x = QThread::idealThreadCount() ) and my problem was in this X.
But Who the hell change CPU during program run??? I mean I have thread num so I divide 12k+/idealThreadCount() and pas this "job list" into each thread. Problem solved.

Again thanks for a help.

PS. Why is above (modified below) code crash for 6k+ jobs?
Modified code:

void myProgrma::myProgrma(){
 connect( this, SIGNAL( jobDone() ), this, SLOT(controlSLOT()));
 myProgrma();
 int count = 12000;
}
void myProgrma::controlSLOT()
{
 if(count  == 12000)
  return;
 else 
  doJobSLOT()
}
 
void myProgrma::doJobSLOT()
{
 //do job here
  emit jobDone();
}

void myProgrma::myProgrma(){
 connect( this, SIGNAL( jobDone() ), this, SLOT(controlSLOT()));
 myProgrma();
 int count = 12000;
}
void myProgrma::controlSLOT()
{
 if(count  == 12000)
  return;
 else 
  doJobSLOT()
}

void myProgrma::doJobSLOT()
{
 //do job here
  emit jobDone();
}

To copy to clipboard, switch view to plain text mode 

[Santosh Reddy]

You can as well implement this without thread (it you wish to), just trigger the jobs on a QTimer with 0 ms timeout.

void myProgram::myProgram(){
 QTimer * timer = new QTimer(this);
 connect(timer, SIGNAL(timeout()), this, SLOT(doJobSLOT()));
 timer->start();
}
 
void myProgram::doJobSLOT()
{
 //do job here
 if(!isJobsPending())
  timer->stop;
}

void myProgram::myProgram(){
 QTimer * timer = new QTimer(this);
 connect(timer, SIGNAL(timeout()), this, SLOT(doJobSLOT()));
 timer->start();
}

void myProgram::doJobSLOT()
{
 //do job here
 if(!isJobsPending())
  timer->stop;
}

To copy to clipboard, switch view to plain text mode 

[Lykurg]

One can't tell much because of the pseudo code...
Is count global? In your example it isn't thus you have an infinite loop. Further do you increase count? And why do you call the c-tor inside the c-tor??? (line 3)

[Talei]

@Santosh Reddy Thanks for reply. I know, and already implemented, solution with QTimer and timeout() (works like a charm, except it's 6 time slower then signal/slot "solution", probably due all those timer checks), see my first post, but I'm looking, because of educational reason, for more possibilities to solve this problem, preferred without QThread.

@Lykurg
The code is irrelevant and I wanted only show the idea and the problem with signal/slot queue stacking.
Yes "count" is global, I wanted just show type of it, and Line 3 is an error, should be controlSLOT();

corrected Example:

void myProgrma::myProgrma(){
 connect( this, SIGNAL( jobDone() ), this, SLOT(controlSLOT()));
 controlSLOT();
 count = 12000; //int count global
}
void myProgrma::controlSLOT()
{
 if(count  == 12000)
  return;
 else 
  doJobSLOT()
}
 
void myProgrma::doJobSLOT()
{
 //do job here
  emit jobDone();
}

void myProgrma::myProgrma(){
 connect( this, SIGNAL( jobDone() ), this, SLOT(controlSLOT()));
 controlSLOT();
 count = 12000; //int count global
}
void myProgrma::controlSLOT()
{
 if(count  == 12000)
  return;
 else 
  doJobSLOT()
}

void myProgrma::doJobSLOT()
{
 //do job here
  emit jobDone();
}

To copy to clipboard, switch view to plain text mode 

I attached an example that shows the error of signal/slot and QTimer solution. QTimer is slower then signal/slot.

[Santosh Reddy]

Either use QTimer or use plain recursive calls.

QTimer : all you jobs are guaranteed to work, job completion time may be large (here time is not limited)
Recursion: only a limited number of jobs can be completed, as stack space is limited (here memory is limited)

Using signals slots in recursive mode is very bad idea, you are basically are not using sigals / slots at all. Rather just call the slot directly like this.

void MainWindow::doJob()
{
    //do job
    jobCount++;
//        emit jobDone(); //with this jobs completed before stack is finished = 6166
    jobControler(); //with this jobs completed before stack is finished = 21584
    return;
}

void MainWindow::doJob()
{
    //do job
    jobCount++;
//        emit jobDone(); //with this jobs completed before stack is finished = 6166
    jobControler(); //with this jobs completed before stack is finished = 21584
    return;
}

To copy to clipboard, switch view to plain text mode 

notice how much of the stack is being wasted by using signal / slot in recurssive mode, more over you can never easily detect a stack overflow condition, so your program may crash at any point.

Recursion is good technique, but has to be used wisely for a controlled number of iterations.

So for educational purpose, even if you move things to a separate thread or multiple threads, you will still run into same problems, as thread by themselves use a significant amount of memory compared to recursive call stack