Implementation considerations for data flow pattern

[redBeard]

I created a signal processing (data flow) engine in Java and now need to convert it to C++ and Qt. It's been a few years since I've done significant C++ programming and this is my first program using Qt.

The Java engine has the following characteristics:

1) consists of a number of data processing elements (some data transform and some display elements)

2) elements each run in their own thread. Connection between engine and elements is using the Observable and Observer Java classes.

3) elements communicate with each other using BlockingQueues.

4) The engine can 'interrupt' each element as they're waiting on the blocking queue for data. Elements (threads) basically wind down in a controlled manner when an interrupt is received.

5) the data 'sent down the pipe' consists of a large number of small messages. Messages are generated by devices every 15 - 30 ms and consist of a dozen numbers.

This all works very well in Java. The problem is my graphics elements (implemented in OpenGL) are too inefficient and the engine/elements are consuming a goodly portion of the CPU.

I'm re-writing the engine and elements in C++ and Qt. I have some basic questions about two main areas: engine<->element connection and element->element communication.

For the engine<->element connection, I'm thinking of just implementing a basic multithreaded structure with the engine starting each element and then doing something to terminate them (and then 'wait()' for them to complete).

For the element->element communication mechanism, I'll use either a basic mutex on an STL queue or a QSemaphore, not sure which.

However, can the engine 'interrupt' a QSemaphore::acquire() method (if I go that route)? Elements will mostly just be waiting for data from their upstream partners. I would like the engine to just interrupt that call and allow them to gracefully wind down.

If I use a mutex on an STL queue, I'll probably use a QWaitCondition and 'wait()' on that object for the upstream partner to add data to the queue. Again, can the engine 'interrupt' threads if they're in the middle of a QWaitCondition::wait() call?

That's about it. I can't seem to find any doc on interrupting Qt-based threads if they're waiting on QSemaphore or QWaitCondition.... thanks for the help.

r

[wysota]

A short remark. I would be against using threads here unless any of the operations is really time consuming. And even if it is, then I would only delegate this particular task (when it is actually being performed) to a worker thread and I'd keep all other things in one thread. Why? To avoid wasting CPU time on context switching and thread synchronization. Don't use threads just because you can. Use them where doing that gives you an actual, real and significant advantage. In other cases use Qt's event system and signal/slot mechanism to communicate objects together. They will provide appropriate synchronization for you.