Need Qt SerialPort class example using signals and slots WITHIN A THREAD

[kuzulis]

@RolandHughes

Mr. b-s-a showed for you how to use QtSerialPort object in non-gui threads, this is the basic concept when
need start Qt-event looping by call exec() method. Without Qt-event loop signals will not work.

Your statement that "lost signals" are totally groundless, you did not provide any simple code to reproduce this "problem".
All that you said - it's just empty words. No one will offer a turnkey solution for you, you must implement it yourself.

QExtSerialPort IS developed. Here is QextSerialPort 1.2 RC

QExtSerialPort does not provide fully asynchronous I/O on Windows, so you can lost productivity.
But in any case, if you consider yourself an expert, you can implement everything in WinAPI / POSIX themselves.


I repeat:
QtSerialPort - an Open Source project, if you do not like - you can fix it yourself or suggest a some solution,
or on end, do not use it, find another project.

[sarcasm]
After all, you're an "expert", but all around - "students"...bla-bla-bla
[/sarcasm]

[RolandHughes]

kuzulis,

Your statement that "lost signals" are totally groundless, you did not provide any simple code to reproduce this "problem".
All that you said - it's just empty words. No one will offer a turnkey solution for you, you must implement it yourself.

Please learn just a tiny bit about a library before inserting your foot so far into your mouth. Signals are lost in most Qt applications and this is be design. When you use an object which emits signals do you assign a slot to catch each and every signal? No. You assign a slot to catch only the signal you are interested in. ALL THE OTHER SIGNALS EMITTED BY THE OBJECT ARE LOST. They have no entry in the dynamically created table matching signals to slots within and across threads. They are emitted and lost.

When new "projects" get added, sometimes they don't do things correctly when creating entries for non-GUI threads. When this happens, the signals emitted which do not have routing entries in the table are LOST. Rather than tossing errors and making developers catch each and every signal from each and every object, the Qt architecture allows developers to catch only those signals and it quietly LOSES the rest.

QExtSerialPort does not provide fully asynchronous I/O on Windows, so you can lost productivity.
But in any case, if you consider yourself an expert, you can implement everything in WinAPI / POSIX themselves.

Please re-read first message. This shop doesn't speak Windows. Development is occuring on Ubuntu Linux.

Students must learn to pay attention when gathering requirements so they don't develop something completely useless.

Thank you for your time.

[wysota]

everyone who suggested the examples and unit tests were good "suggested" this since it is the only thing they used when not running in the GUI thread.

I don't know who is "everyone" and what they "suggested", I'm not telling you to look at examples but to use QIODevice API like it is supposed to be used. You can search the Internet for similar discussions on doing networking with threads. "Everyone" "suggests" this is supposed to be done in threads because that's how they have been doing it using other technologies that didn't provide non-blocking API. It is not supposed to be done in threads.

Aparently you have never done data acquisition. Your assumption is completely invalid. We do have multiple cores, but you do not need one per thread, especially for serial. I've run a 16 serial ports flat out (115200 baud at each port) on a sub Ghz processor without ever losing anything.

Which is exactly my point. You don't need "non-blocking threads" (whatever you mean by that) to handle serial ports simply because the operating system will buffer data for you. If you are running 115200 baud transmission, it doesn't mean you need to read 115200 bits per second. It means you need to read 115200 per second on average. This is only 14kB per port, I'm sure single thread can handle reading 16 of these per second and still be able to do some fancy stuff in your UI.

Someone kicking off the GUI "report" to plot and graph outliers, or some other such "report" will bind that processor for minutes, if not hours.

Did it come to your mind to do this "minutes if not hours" lasting processing in a thread (or even in a separate process)? Are you really blocking your UI for "minutes if not hours" and still think that's ok? Are you sure it is me who never did any data aquisition and processing?

Don't compare this report to something you've done in school

We never did that in school because by the time I was finishing school we were running a single 56k modem shared by around 20 machines.

Think SETI or some other serious research application where data points from a few minutes to a few days can completely fill a 1TB drive when stored in binary format.

Oh jeez, and you want to process it in the GUI thread? That's so professional...

Signals are dynamically mapped. Signals can easily get lost any time the entry in the signal map is missing or pointing to a null or pointing to an incorrect thread, hence why I was looking at the signal spy stuff as the next step.

You have to be kidding. If you are worried about it and are unable to check if that is really the case then do proper synchronization in your code.

Stop trolling and claiming to have all the answers and start reading on proper usage of components at hand. Your post on disabled copy constructor in a QObject subclass clearly shows how well you understand Qt code.

---

Added after 6 minutes:

Please learn just a tiny bit about a library before inserting your foot so far into your mouth.

I think Kuzulis knows best of us how the QtSerialPort library works.

Signals are lost in most Qt applications and this is be design. When you use an object which emits signals do you assign a slot to catch each and every signal? No. You assign a slot to catch only the signal you are interested in. ALL THE OTHER SIGNALS EMITTED BY THE OBJECT ARE LOST.

So that's what you mean by a "lost" signal... I would rather call that "ignored".

They have no entry in the dynamically created table matching signals to slots within and across threads. They are emitted and lost.

Some of them might even not be emitted at all

When new "projects" get added, sometimes they don't do things correctly when creating entries for non-GUI threads. When this happens, the signals emitted which do not have routing entries in the table are LOST.

Are you talking about your own projects or projects in general?

[prof.ebral]

I'm still not 100% certain what you are looking for. From the initial post you want the QtSerialPort to process data while in a thread, and then the main program to handle that data while in a thread? Is that correct? If that is what you are looking for you might consider an MVC model where you model is your QtSerialPort classes and QThread classes, your View handles UI updates, and your Controller handles the signal connections. Your controller would accept emitted signals and start new threads to handle data processing.

As far as I understand data sent by signals is sent to the main thread. You are asking for this, "Looking actual compiling example of SerialPort class using signals and slots within a thread which is NOT the GUI thread."

[wysota]

As far as I understand data sent by signals is sent to the main thread.

There are two things here. "Data" (as in arguments to signals) are not sent to any thread, in the meaning that thread affinity of them does not change in any way. If you pass a pointer to a QObject that lives in thread A to thread B, then accessing that object there might cause data corruption or worse. If you pass some non-QObject by pointer then it doesn't have any thread affinity and it is its thread-safety that decides whether this is safe or not. if you pass a non-QObject by value then it is copied and in regular cases will have no notion of its other clone being accessed from the original thread. However another topic is if we interpret what you said as "signals are sent to the main thread" in which case this is not true -- signals are sent in the thread they are emitted in (it's just a plain function call) but connecting to them may cause an event to be posted to an object living in a different thread (it doesn't have to be the "main" thread though).

[prof.ebral]

However another topic is if we interpret what you said as "signals are sent to the main thread" in which case this is not true -- signals are sent in the thread they are emitted in (it's just a plain function call) but connecting to them may cause an event to be posted to an object living in a different thread (it doesn't have to be the "main" thread though).

Hmmm, I thought the main thread received the signal as a handler and then executed the method defined by the slot.

[wysota]

Hmmm, I thought the main thread received the signal as a handler and then executed the method defined by the slot.

No, it depends on the thread affinity of the object that the slot belongs to. Here is a simple example:

#include <QtCore>
 
class Object : public QObject {
  Q_OBJECT
public:
  Object(QObject *parent = 0) : QObject(parent) {
 
  }
public slots:
  void reportThread() {
    qDebug() << Q_FUNC_INFO << QThread::currentThread();
  }
};
 
#include "main.moc"
 
int main(int argc, char **argv) {
  QCoreApplication app(argc, argv);
 
  QVector<QThread*> threads;
  QVector<Object*> objects;
  for(int i=0;i<4;++i) {
    QThread *thr = new QThread;
    threads << thr;
 
    Object *o = new Object;
    o->moveToThread(thr);
    objects << o;
 
    thr->start();
 
  }
  // last object in main thread
  objects << new Object;
  qDebug() << "GUI thread:" << objects.last()->thread();  
  QTimer t;
  t.start(2000);
  foreach(Object *o, objects)
    QObject::connect(&t, SIGNAL(timeout()), o, SLOT(reportThread()));  
  return app.exec();
}

#include <QtCore>

class Object : public QObject {
  Q_OBJECT
public:
  Object(QObject *parent = 0) : QObject(parent) {
  
  }
public slots:
  void reportThread() {
    qDebug() << Q_FUNC_INFO << QThread::currentThread();
  }
};

#include "main.moc"

int main(int argc, char **argv) {
  QCoreApplication app(argc, argv);
  
  QVector<QThread*> threads;
  QVector<Object*> objects;
  for(int i=0;i<4;++i) {
    QThread *thr = new QThread;
    threads << thr;
    
    Object *o = new Object;
    o->moveToThread(thr);
    objects << o;
    
    thr->start();
    
  }
  // last object in main thread
  objects << new Object;
  qDebug() << "GUI thread:" << objects.last()->thread();  
  QTimer t;
  t.start(2000);
  foreach(Object *o, objects)
    QObject::connect(&t, SIGNAL(timeout()), o, SLOT(reportThread()));  
  return app.exec();
}

To copy to clipboard, switch view to plain text mode 

You can also move the timer to one of the threads and see if it changes anything.

Oh, one more thing -- you can run this example under a debugger, set a breakpoint in the slot and look at the backtrace. You'll see how it differs for the last added object (the one that lives in the main thread) compared to other objects.

[prof.ebral]

Ok, thanks wysota. I use PyQt and when I look at the current thread inside a method that has been called by a signal it is the main thread. I've yet to see an instance when it is isn't in PyQt.

[wysota]

Ok, thanks wysota. I use PyQt and when I look at the current thread inside a method that has been called by a signal it is the main thread. I've yet to see an instance when it is isn't in PyQt.

Maybe you are using Python threads and not QThreads. I have no idea how Qt will behave then.

I ran this test program:

from PySide import QtCore
import sys
 
class Object(QtCore.QObject):
  def reportThread(self):
    print(QtCore.QThread.currentThread())
 
app = QtCore.QCoreApplication(sys.argv)
thr = QtCore.QThread()
obj = Object()
 
timer = QtCore.QTimer()
print("MainThread"+str(QtCore.QThread.currentThread()))
timer.timeout.connect(obj.reportThread)
timer.start(2000)
app.exec_()

from PySide import QtCore
import sys

class Object(QtCore.QObject):
  def reportThread(self):
    print(QtCore.QThread.currentThread())

app = QtCore.QCoreApplication(sys.argv)
thr = QtCore.QThread()
obj = Object()

timer = QtCore.QTimer()
print("MainThread"+str(QtCore.QThread.currentThread()))
timer.timeout.connect(obj.reportThread)
timer.start(2000)
app.exec_()

To copy to clipboard, switch view to plain text mode 

and it behaves the same way as I described:

MainThread<PySide.QtCore.QThread object at 0xf6a050>
<PySide.QtCore.QThread object at 0xf6a0e0>
<PySide.QtCore.QThread object at 0xf6a0e0>

[prof.ebral]

This is the output I get

MainThread<PyQt4.QtCore.QThread object at 0x7f16ef83c408>
<PyQt4.QtCore.QThread object at 0x7f16ef83c408>

MainThread<PyQt4.QtCore.QThread object at 0x7f16ef83c408>
<PyQt4.QtCore.QThread object at 0x7f16ef83c408>

To copy to clipboard, switch view to plain text mode 

The main thread is located at the same memory allocation as all of the other threads.

edit: By the way, here is the changed code for PyQt

from PyQt4 import QtCore
import sys
 
class Object(QtCore.QObject):
  def reportThread(self):
    print(QtCore.QThread.currentThread())
 
app = QtCore.QCoreApplication(sys.argv)
thr = QtCore.QThread()
obj = Object()
 
timer = QtCore.QTimer()
print("MainThread"+str(QtCore.QThread.currentThread()))
timer.timeout.connect(obj.reportThread)
timer.start(2000)
app.exec_()

from PyQt4 import QtCore
import sys

class Object(QtCore.QObject):
  def reportThread(self):
    print(QtCore.QThread.currentThread())

app = QtCore.QCoreApplication(sys.argv)
thr = QtCore.QThread()
obj = Object()

timer = QtCore.QTimer()
print("MainThread"+str(QtCore.QThread.currentThread()))
timer.timeout.connect(obj.reportThread)
timer.start(2000)
app.exec_()

To copy to clipboard, switch view to plain text mode 

Not much different.

[wysota]

Maybe that's an issue with PyQt4 (I would rather suspect that it returns an incorrect thread pointer rather than runs the slot in a wrong thread). As you can see I was using PySide.


Edit: LOL... I forgot to move the object to the thread and start the thread

Here is a working version:

from PyQt4 import QtCore
import sys
 
class Object(QtCore.QObject):
  def __init__(self):
    QtCore.QThread.__init__(self)
 
  def reportThread(self):
    print(QtCore.QThread.currentThreadId())
 
app = QtCore.QCoreApplication(sys.argv)
thr = QtCore.QThread()
obj = Object()
obj.moveToThread(thr)
thr.start()
 
timer = QtCore.QTimer()
print("MainThread"+str(QtCore.QThread.currentThreadId()))
timer.timeout.connect(obj.reportThread)
timer.start(2000)
 
t2 = QtCore.QTimer()
t2.timeout.connect(app.quit)
t2.start(10000)
app.exec_()

from PyQt4 import QtCore
import sys

class Object(QtCore.QObject):
  def __init__(self):
    QtCore.QThread.__init__(self)

  def reportThread(self):
    print(QtCore.QThread.currentThreadId())

app = QtCore.QCoreApplication(sys.argv)
thr = QtCore.QThread()
obj = Object()
obj.moveToThread(thr)
thr.start()

timer = QtCore.QTimer()
print("MainThread"+str(QtCore.QThread.currentThreadId()))
timer.timeout.connect(obj.reportThread)
timer.start(2000)

t2 = QtCore.QTimer()
t2.timeout.connect(app.quit)
t2.start(10000)
app.exec_()

To copy to clipboard, switch view to plain text mode 

[prof.ebral]

Maybe that's an issue with PyQt4 (I would rather suspect that it returns an incorrect thread pointer rather than runs the slot in a wrong thread). As you can see I was using PySide.

I think it is a feature of PyQt. I'll have to check over the documentation again, but I think PyQt signals and slots operate 'through' the main thread. Like how I described before. Sorry for adding confusion to the post.

[RolandHughes]

I don't know who is "everyone" and what they "suggested", I'm not telling you to look at examples but to use QIODevice API like it is supposed to be used. You can search the Internet for similar discussions on doing networking with threads. "Everyone" "suggests" this is supposed to be done in threads because that's how they have been doing it using other technologies that didn't provide non-blocking API. It is not supposed to be done in threads.

Yes it is. Please do not try to view this application in terms of something running on a machine where the user can also surf the Web and check email.

Which is exactly my point. You don't need "non-blocking threads" (whatever you mean by that) to handle serial ports simply because the operating system will buffer data for you. If you are running 115200 baud transmission, it doesn't mean you need to read 115200 bits per second. It means you need to read 115200 per second on average. This is only 14kB per port, I'm sure single thread can handle reading 16 of these per second and still be able to do some fancy stuff in your UI.

That is very poor design and not allowed in a scientific environment were loss of a single data point scraps the entire test run.

Did it come to your mind to do this "minutes if not hours" lasting processing in a thread (or even in a separate process)? Are you really blocking your UI for "minutes if not hours" and still think that's ok? Are you sure it is me who never did any data aquisition and processing?

Oh jeez, and you want to process it in the GUI thread? That's so professional...

It is a design requirement. Do not confuse this with some application on a machine which has any other purpose or ability in life. The really large data sets can and do get off-loaded to serious computers for detailed analysis. The initial analysis to weed out non-interesting data sets occurs on the collection machine. What you consider a waste saves weeks and in some cases months of serious computing time.

You have to be kidding. If you are worried about it and are unable to check if that is really the case then do proper synchronization in your code.

Stop trolling and claiming to have all the answers and start reading on proper usage of components at hand. Your post on disabled copy constructor in a QObject subclass clearly shows how well you understand Qt code.

I'm not trolling or claiming to have all of the answers. I came in asking if there was unit test code proving this works. I got a bunch of responses which sounded like a Bing commercial.

My next debugging course was to use the signal tool, but any class which does not have a copy constructor cannot be used with it...a point you have repeatedly failed to recognize.

Please don't view this as a user application. This is a single purpose system running in an environment were no data point can be lost. Every device must be service by a non-blocking thread and those threads must be distributed across processors. It is not an end user application nor a hokey little smart phone app. It gathers all data points and the GUI only has two purposes. 1) Configure everything for a test run 2) run the reports/graphs used to identify interesting data sets. That's it.

Some of them might even not be emitted at all

Hence the wish to test with the signal tool.

Are you talking about your own projects or projects in general?

Projects which get released for use with Qt. I find they only provide unit test code/examples for GUI use, and nothing for use within non-GUI threads.

At any rate. This message thread can now end. I'm on my own fixing it or I have to switch to qextserial.

---

Added after 7 minutes:

As far as I understand data sent by signals is sent to the main thread. You are asking for this, "Looking actual compiling example of SerialPort class using signals and slots within a thread which is NOT the GUI thread."

http://doc.qt.digia.com/qt/threads-q...across-threads

Queued Connection The slot is invoked when control returns to the event loop of the receiver's thread. The slot is executed in the receiver's thread.