Main thread - worker thread communication.

[kikapu]

Hi to all and great for me to be here!

I was previously used to C# and delegates to communicate from Main Form's thread with a worker thread that i spawn from that form and now i use Python and Qt and trying to do the same thing.
I have the very simple example of counting from "start" to "end" and just display the progress in a progress bar.
The routine that does that is running on a secondary thread. I have made the Gui with Qt Designer and i have the following code that i wrote:

import sys
import threading
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep
 
class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
 
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        t = threading.Thread(self.doCount())
        t.start()
 
    def doCount(self):
        start = self.ui.spFrom.value()
        end = self.ui.spTo.value() + 1
 
        for x in range(start, end):
            self.ui.progres.setValue(x)
            sleep(1)
            app.processEvents()
 
if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    count = CountNumbersForm();
    count.show()
    sys.exit(app.exec_())

import sys
import threading
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
       
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        t = threading.Thread(self.doCount())
        t.start()
    
    def doCount(self):
        start = self.ui.spFrom.value()
        end = self.ui.spTo.value() + 1
        
        for x in range(start, end):
            self.ui.progres.setValue(x)
            sleep(1)
            app.processEvents()
    
if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    count = CountNumbersForm();
    count.show()
    sys.exit(app.exec_())

To copy to clipboard, switch view to plain text mode 

(spFrom and spTo are spin boxes, "start" is a button that to press to do the job)


Apart from the fact that progress bar does not updated correctly (it stops updating at some percent and not at 100%), my main problem is that main thread (ie. the main Form) is not so responsive as i thought it would be.
I read somewhere that Qt's Signals and Slots mechanism is working correctly in the cross-thread situation but apparently i have made something wrong and thus program hase that behavior.

Can anyone please help me with this ? How can i reliably communicate from Form's main thread to it's working thread in Qt ?

Thanks a lot for any help!

[wysota]

You're not using signals and slots here. First of all you should use Qt threads instead of Python threads, otherwise there is a good chance it won't work properly.

In run() method of the thread create an object that inherits QObject and there declare a signal which you then have to emit each time you want to change the progress bar value. Then connect the signal to the setValue slot in the progress bar and start the thread. And forget about processEvents() - you don't need it.

[kikapu]

You're not using signals and slots here. First of all you should use Qt threads instead of Python threads, otherwise there is a good chance it won't work properly.

But i thought that i used Signals and Slots...Isn't the declaration "def on_start_clicked(self):" just an alternative way to the usual "QtCore.QObject.connect" call ?
I mean, the Signals and Slots mechanism is underneath of both ways, isn't ??

In run() method of the thread create an object that inherits QObject and there declare a signal which you then have to emit each time you want to change the progress bar value. Then connect the signal to the setValue slot in the progress bar and start the thread.

Ok, i will try right know this and i let you know. Should i implement a QThread instead of a Python one ??

And forget about processEvents() - you don't need it.

Ok, i just did (forgot)

[wysota]

Yes, provided that start is a button. Yes, you should use QThreads.

[kikapu]

You're not using signals and slots here. First of all you should use Qt threads instead of Python threads, otherwise there is a good chance it won't work properly.

In run() method of the thread create an object that inherits QObject and there declare a signal which you then have to emit each time you want to change the progress bar value. Then connect the signal to the setValue slot in the progress bar and start the thread. And forget about processEvents() - you don't need it.

At first, a million thanks for helping me out...

So, i am not sure i am following here..
Do you mean to do something like this:

class ProgressNotifier(QObject):
    def __init__(self, start, end):
        self.start = start
        self.end = end
 
    def notify(self):
        # declare signal here
 
class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
 
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        pn = ProgressNotifier(self.ui.spFrom.value(), self.ui.spTo.value() + 1);
        # connect the signal here (the method of pn) to the progress bar "setValue" slot
        t = threading.Thread(self.doCount())
        t.start()
 
    def doCount(self):
        for x in range(start, end):
            #emit signal so "setValue
            sleep(1)

class ProgressNotifier(QObject):
    def __init__(self, start, end):
        self.start = start
        self.end = end
    
    def notify(self):
        # declare signal here
        
class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
       
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        pn = ProgressNotifier(self.ui.spFrom.value(), self.ui.spTo.value() + 1);
        # connect the signal here (the method of pn) to the progress bar "setValue" slot
        t = threading.Thread(self.doCount())
        t.start()
    
    def doCount(self):
        for x in range(start, end):
            #emit signal so "setValue
            sleep(1)

To copy to clipboard, switch view to plain text mode 

If not, then how can i code it correctly ??
If yes, why i need a second class to do this ??

Again, a million thanks...

[jpn]

But i thought that i used Signals and Slots...Isn't the declaration "def on_start_clicked(self):" just an alternative way to the usual "QtCore.QObject.connect" call ?
I mean, the Signals and Slots mechanism is underneath of both ways, isn't ??

Connecting Signals and Slots

[wysota]

You're still not using QThread.

[kikapu]

You're still not using QThread.

I think the main "problem" is not if i use or not QThread, i will certainly use it if you say that i have to. The main thing is if my skeleton code that i wrote is correct!
Is it ??


@jpn: Thanks, i already have this guide, i'll check it again.

[kikapu]

Ok, i read some docs and came up with something like the below. I like the postEvent/customEvent thing although i surely try to make an example with directly use the signals/slots mechanism.

But i have the error "TypeError: 'int' object is not callable" at "t.start()" line...
Can anyone help with this ?

The code is:

import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep
 
class WorkerThread(QtCore.QThread):
    def __init__(self, start, end, receiver):
        QtCore.QThread.__init__(self)
        self.receiver = receiver
        self.start = start
        self.end = end
 
    def run(self):
        for x in range(start, end):
            #sleep(1)
            event = QCustomEvent(200)
            event.setData(x)
            QThread.postEvent(self.receiver, event)
 
class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
 
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        t = WorkerThread(self.ui.spFrom.value(), self.ui.spTo.value() + 1, self);
        t.start()
 
    def customEvent(self, event):
        if event.type() == 200:
            i = event.data()
            self.ui.progres.setValue(i)
 
if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    count = CountNumbersForm();
    count.show()
    sys.exit(app.exec_())

import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class WorkerThread(QtCore.QThread):
    def __init__(self, start, end, receiver):
        QtCore.QThread.__init__(self)
        self.receiver = receiver
        self.start = start
        self.end = end
    
    def run(self):
        for x in range(start, end):
            #sleep(1)
            event = QCustomEvent(200)
            event.setData(x)
            QThread.postEvent(self.receiver, event)
        
class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
       
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        t = WorkerThread(self.ui.spFrom.value(), self.ui.spTo.value() + 1, self);
        t.start()
    
    def customEvent(self, event):
        if event.type() == 200:
            i = event.data()
            self.ui.progres.setValue(i)
    
if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    count = CountNumbersForm();
    count.show()
    sys.exit(app.exec_())

To copy to clipboard, switch view to plain text mode 

Thanks a lot for any help...

[wysota]

We won't know unless you implement the thread. The thread needs to emit a signal that will update the progress bar. I don't see that in your code.

[kikapu]

We won't know unless you implement the thread. The thread needs to emit a signal that will update the progress bar. I don't see that in your code.

Xmm,...ok, i do this now. But, isn't the postEvent/customEvent thing a viable mechanism to do such things ??

[wysota]

You can use custom events but using signals is simpler. They use the event mechanism behind the scenes, so the result is the same, just less coding.

[kikapu]

You can use custom events but using signals is simpler. They use the event mechanism behind the scenes, so the result is the same, just less coding.

Ok the...I try to make it your way and i post the revised code below. I get the same error message as before: "TypeError: 'int' object is not callable" at t.start() line.

here is the code:

import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep
 
class WorkerThread(QtCore.QThread):
    def __init__(self, form, start, end):
        QtCore.QThread.__init__(self)
        self.start = start
        self.end = end
        self.form = form
 
    def run(self):
        c = C()
        QtCore.QObject.connect(form.ui.progressBar, c.update, form, QtCore.SLOT("setValue(int)"))
 
        for x in range(start, end):
            c.emit(c.update, x)
 
class C(QtCore.QObject):
    update = QtCore.SIGNAL("update(int)")
 
 
class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
 
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        t = WorkerThread(self, self.ui.spFrom.value(), self.ui.spTo.value() + 1);
        t.start()

import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class WorkerThread(QtCore.QThread):
    def __init__(self, form, start, end):
        QtCore.QThread.__init__(self)
        self.start = start
        self.end = end
        self.form = form
    
    def run(self):
        c = C()
        QtCore.QObject.connect(form.ui.progressBar, c.update, form, QtCore.SLOT("setValue(int)"))
        
        for x in range(start, end):
            c.emit(c.update, x)
            
class C(QtCore.QObject):
    update = QtCore.SIGNAL("update(int)")
        

class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
       
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        t = WorkerThread(self, self.ui.spFrom.value(), self.ui.spTo.value() + 1);
        t.start()

To copy to clipboard, switch view to plain text mode 

I am very much frustrated here...

[kikapu]

After a lot of study, i now coded it more correctly (i think) but strangely enough i get the same error message as before. ('int' object is not callable" at t.start() )

Code is:

import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep
 
class WorkerThread(QtCore.QThread):
    def __init__(self, start, end):
        QtCore.QThread.__init__(self)
        self.start = start
        self.end = end
 
    def run(self):
        for x in range(start, end):
            self.emit(QtCore.SIGNAL("updateProgress"), x)
 
 
class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
 
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        worker = WorkerThread(self.ui.spFrom.value(), self.ui.spTo.value() + 1);
        QtCore.QObject.connect(worker, QtCore.SIGNAL("updateProgress"), self.updateProgress, QtCore.Qt.QueuedConnection)
        worker.start()
 
    def updateProgress(self, val):
        self.ui.progres.setValue(val)
 
 
if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    count = CountNumbersForm();
    count.show()
    sys.exit(app.exec_())

import sys
from PyQt4 import QtCore, QtGui
from calculatorform_ui import Ui_CalculatorForm
from ui_countNumbers import Ui_MainWindow
from time import sleep

class WorkerThread(QtCore.QThread):
    def __init__(self, start, end):
        QtCore.QThread.__init__(self)
        self.start = start
        self.end = end
    
    def run(self):
        for x in range(start, end):
            self.emit(QtCore.SIGNAL("updateProgress"), x)
            

class CountNumbersForm(QtGui.QMainWindow):
    def __init__(self, parent=None):
        QtGui.QMainWindow.__init__(self, parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
       
    @QtCore.pyqtSignature("")
    def on_start_clicked(self):
        worker = WorkerThread(self.ui.spFrom.value(), self.ui.spTo.value() + 1);
        QtCore.QObject.connect(worker, QtCore.SIGNAL("updateProgress"), self.updateProgress, QtCore.Qt.QueuedConnection)
        worker.start()
 
    def updateProgress(self, val):
        self.ui.progres.setValue(val)
    
    
if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    count = CountNumbersForm();
    count.show()
    sys.exit(app.exec_())

To copy to clipboard, switch view to plain text mode 

[wysota]

What is class C?

I can give you a draft of C++ code that does what you want. Then you can translate it into python, as my python skills are a bit stale...

class Emiter : public QObject {
  Q_OBJECT
public:
  Emiter(QObject *parent=0) : QObject(parent){}
  void emitSignal(int v){ emit sig(v); }
signals:
  void sig(int);
};
class MyThread : public QThread {
  Q_OBJECT
public:
  MyThread(QObject *parent =0) : QThread(parent){}
  void run() {
    Emiter e;
    connect(&e, SIGNAL(sig(int)), this, SIGNAL(propagate(int)));
    i=0;
    while(i<=100){
      sleep(1);
      e.emitSignal(i++);
    }
  }
signals:
  void propagate(int);
};
 
//...
MyThread thread;
QProgressBar pbar;
connect(&thread, SIGNAL(propagate(int)), &pbar, SLOT(setValue(int)));
thread.start();

class Emiter : public QObject {
  Q_OBJECT
public:
  Emiter(QObject *parent=0) : QObject(parent){}
  void emitSignal(int v){ emit sig(v); }
signals:
  void sig(int);
};
class MyThread : public QThread {
  Q_OBJECT
public:
  MyThread(QObject *parent =0) : QThread(parent){}
  void run() {
    Emiter e;
    connect(&e, SIGNAL(sig(int)), this, SIGNAL(propagate(int)));
    i=0;
    while(i<=100){
      sleep(1);
      e.emitSignal(i++);
    }
  }
signals:
  void propagate(int);
};

//...
MyThread thread;
QProgressBar pbar;
connect(&thread, SIGNAL(propagate(int)), &pbar, SLOT(setValue(int)));
thread.start();

To copy to clipboard, switch view to plain text mode 

Of course there are other ways to do the same thing, for example the propagate() signal is not needed, you can connect the original signal directly to the progress bar if you have access to it.

[kikapu]

What is class C?

There is no class C anymore (see my just above post for a more correct solution). Ok, i try to translate your C code although i think that the last Python version i posted is very much alike...

[wysota]

The code you posted is dangerous because you emit a signal from a worker thread on account of an object that is in the gui thread and I have no idea in context of which thread the signal will be emitted and will it be queued or executed directly. My Emiter class makes sure the context of the worker thread is used and the signal is queued and propagated in the context of the GUI thread.

[kikapu]

The code you posted is dangerous because you emit a signal from a worker thread on account of an object that is in the gui thread

But isn't the whole idea ?? In C# we use delegates to do that, in Qt i hoped that with Signals and Slots i will be able to do it. I think that Qt automatically find the correct thread to "use".

Your Emiter class is "translated" to Python like this ?

class Emiter(QtCore.QObject):
    def __init__(self):
        QObject.__init__(self)
 
    def emitSignal(self, val):
        self.emit(QtCore.SIGNAL("updateProgress"), x)

class Emiter(QtCore.QObject):
    def __init__(self):
        QObject.__init__(self)
    
    def emitSignal(self, val):
        self.emit(QtCore.SIGNAL("updateProgress"), x)

To copy to clipboard, switch view to plain text mode 

And what is the purpose (and implementation as it is not shown how it does it's work) of MyThread:: propagate ? Just to forward the signal ?

I think i am stuck...

[wysota]

But isn't the whole idea ?? In C# we use delegates to do that, in Qt i hoped that with Signals and Slots i will be able to do it. I think that Qt automatically find the correct thread to "use".

Yes, but I'm not sure what it will decide when you call a function from an object that lives in another thread. It might work, but I'm not sure, therefore I used this kind of "proxy".

Your Emiter class is "translated" to Python like this ?

class Emiter(QtCore.QObject):
    def __init__(self):
        QObject.__init__(self)
 
    def emitSignal(self, val):
        self.emit(QtCore.SIGNAL("updateProgress"), x)

class Emiter(QtCore.QObject):
    def __init__(self):
        QObject.__init__(self)
    
    def emitSignal(self, val):
        self.emit(QtCore.SIGNAL("updateProgress"), x)

To copy to clipboard, switch view to plain text mode 

Looks correct.

And what is the purpose (and implementation as it is not shown how it does it's work) of MyThread:: propagate ? Just to forward the signal ?

Yes, just to forward the signal. As I mentioned, you could do it without it, but then you'd have to have access to the progress bar from within the thread object. That's your choice.

[kikapu]

Yes, just to forward the signal. As I mentioned, you could do it without it, but then you'd have to have access to the progress bar from within the thread object. That's your choice.

Yes, i now what you mean, we sometimes use it in .net too. But i am afraid i can not go too far, every solution that i implement, i see this error "TypeError: 'int' object is not callable"

I really feel like an idiot, i have written countless thousands lines of code in my life and i am stucked in 10 lines of this. What can i say...