Threads and cpu utilization

[babu198649]

Hi,
I have 9 threads in my application. All the threads are started at the start of the application and continues to run till the termination of the program.

When the application is started the cpu utilization shows 99%. If i close the application the cpu utiliztion is only 3%.This causes other application to respond slowly. I am using ubuntu-8.10.

Whatever the cpu-time that is allocated for the application by the operating system, only that time is split b/w threads. How does one application affect another applications response time.

How to control the cpu utilizition of the threading programs.

[tbscope]

For example:

yourThread->start(QThread::LowestPriority);

[wysota]

How to control the cpu utilizition of the threading programs.

You can make sure your threads utilize the cpu properly - for instance remove all busy loops and introduce proper synchronization of the threads using mutexes/semaphores/wait conditions.

[squidge]

When the application is started the cpu utilization shows 99%. If i close the application the cpu utiliztion is only 3%.This causes other application to respond slowly. I am using ubuntu-8.10.

Is there a reason your threads require 99% processor usage? Are they doing some large calculations?

[babu198649]

Thanks for replys,

Is there a reason your threads require 99% processor usage? Are they doing some large calculations?

All the threads have a local socket(uses event-loop). The data received from the socket is passed to another thread using signals and slots mechanism. The connections(signals and slots) b/w threads are made in the main thread.

I have implemented the same application using single thread.(i.e) All the sockets are created in a single thread and all the sockets use that single threads event-loop. This implementation did not take much cpu utilization, but its response to sockets was much slower compared to the multithreaded implementation.

Any better design ideas?

The operating system provies equal cpu-time for processes, in that case how would one application is capable of consuming another applications cpu-time?

[wysota]

Any better design ideas?

I would definitely not spawn so many threads. One-two threads would probably suffice.

The operating system provies equal cpu-time for processes, in that case how would one application is capable of consuming another applications cpu-time?

There is no such thing as equal cpu time for each process. If it was like that, if one process would block waiting for some i/o, then no other process could operate which would be silly. In reality it means each process has an equal chance of getting a slice of cpu time but if one process doesn't want it, another process will own the cpu. So if you app constantly cries for cpu power, it will effectively steal it from other processes that do some i/o. In other words one badly written application can slow the whole system (almost) to a halt.

[squidge]

For something as simple as that, you certainly shouldn't be using up 99% of processor usage. It indicates that some thread is not releasing it's allocation of time slice. If you could post some code, maybe we can determine where the fault lies.

Neither Windows nor Linux gives all processes equal cpu-time. They are pre-emptive priorty based task schedulers. If you wish to ensure minimal effect on the system when running at near 100% CPU usage, you should reduce the priority of the task to lower than that of the rest of the system. On Linux, you can do this using the 'nice' command. However, it's better if your threads release all ununsed time in each time slice so the system can sleep the processor when the system is idle.

[wysota]

Neither Windows nor Linux gives all processes equal cpu-time. They are pre-emptive priorty based task schedulers.

Hold on There is more than one allocator available for Linux. And if you give some processes a real-time priority, they effectively get equal slices of the cpu in a round-robin fashion. Of course what I said earlier still applies.

Still, if the program we are talking about mainly reads from sockets, it should never come anywhere near 100% of cpu usage unless there is another process that constantly feeds the socket(s) with large amount of new data in quick bursts.

[babu198649]

If you could post some code, maybe we can determine where the fault lies.

There is one ServerThread which listens on a port, While all the other ClientThreads try to connect to different servers. Once a ClientThread connects to a server it emits the signal to the mainthread so that its status is displayed on the gui.
The client that connects to the ServerThread may send xml documents, which is parsed and different binary frames(QByteArrays) are created which are sent to client threads by emitting byteArrays inside signals. Which when received by the ClientThreads sends it to the Servers it is connected to.

Some of the ClientThreads may receive information periodically from the server to which they are connected and this information is send to the ServerThread and also displayed on the gui.

The ClientThread implementation is shown below.

void ClientThread1::run() {
 QTcpSocket socket;
    do {
        if(quit_thread) {
            return;
        }
        socket.connectToHost(ip, port);
        if(time.elapsed() > emitTimeOut) {
            //emit updateStatus("  Connecting to ..tsg..");
            time.restart();
        }
    }while (!socket.waitForConnected(timeOut));
    emit dataReady("<b><font color=\"green\">Connected to Server1</font></b>");//display in gui
    emit socketConnected();
 
    connect(&socket, SIGNAL(disconnected()), this, SLOT(reStartThread()));
    //the events for the sockets is handled in SocketHandler
    SocketHandler socketHandler(&socket);
    //forward the data to SocketHandler which is received from other thread
   connect(this, SIGNAL(writeFramesReady(QByteArray)), &socketHandler, SLOT(writeData(QByteArray)), Qt::QueuedConnection);
   //send the data to other threads
   connect(&socketHandler, SIGNAL(readFramesReady(QByteArray)), this, SIGNAL(readFramesReady(QByteArray)));
   //some more connections.
   exec();
}
SocketHandler::SocketHandler(QTcpSocket *socket) : socket(socket)
{
    connect(socket, SIGNAL(readyRead()), this, SLOT(readData()));
}
 
void SocketHandler::readData()
{
    receivedBytes += socket->readAll();
    //parse the data
    if(/*some logic of the received data*/) {
      emit dataForThread2(byteArray);
    } else {
      emit dataForThread3(byteArray);
    }
}
 
void SocketHandler::writeData(QByteArray byteArray)
{
     socket->write(byteArray);
}

void ClientThread1::run() {
 QTcpSocket socket;
    do {
        if(quit_thread) {
            return;
        }
        socket.connectToHost(ip, port);
        if(time.elapsed() > emitTimeOut) {
            //emit updateStatus("  Connecting to ..tsg..");
            time.restart();
        }
    }while (!socket.waitForConnected(timeOut));
    emit dataReady("<b><font color=\"green\">Connected to Server1</font></b>");//display in gui
    emit socketConnected();

    connect(&socket, SIGNAL(disconnected()), this, SLOT(reStartThread()));
    //the events for the sockets is handled in SocketHandler
    SocketHandler socketHandler(&socket);
    //forward the data to SocketHandler which is received from other thread
   connect(this, SIGNAL(writeFramesReady(QByteArray)), &socketHandler, SLOT(writeData(QByteArray)), Qt::QueuedConnection);
   //send the data to other threads
   connect(&socketHandler, SIGNAL(readFramesReady(QByteArray)), this, SIGNAL(readFramesReady(QByteArray)));
   //some more connections.
   exec();
}
SocketHandler::SocketHandler(QTcpSocket *socket) : socket(socket)
{
    connect(socket, SIGNAL(readyRead()), this, SLOT(readData()));
}

void SocketHandler::readData()
{
    receivedBytes += socket->readAll();
    //parse the data
    if(/*some logic of the received data*/) {
      emit dataForThread2(byteArray);
    } else {
      emit dataForThread3(byteArray);
    }
}

void SocketHandler::writeData(QByteArray byteArray)
{
     socket->write(byteArray);
}

To copy to clipboard, switch view to plain text mode 

The ServerThread is also implemented in the same fashion except that the server listens on a port instead of connecting to client.
The signals b/w the threads connected in the mainThread,

[squidge]

I think we have found the problem. Instead of using a do/while loop, I think instead you should have a state machine based on the stateChanged signal, and only attempt a connect when in a particular state (perhaps with a qtimer to ensure you don't try and connect 1,000 times per second)

Then, for example, the state machine would move you from the UnconnectedState to the HostLookupState/ConnectingState in which you would disable the timer, then if you get the ConnectedState you could then send the main form the "I'm connected" signal. If it reverts back to UnconnectedState, then you know to restart the timer, etc.

[wysota]

There is waitForConnected() so he is not connecting all the time, only when he doesn't manage to connect during the value of timeOut.

[babu198649]

The value of timeOut is 3000(3 seconds).

[wysota]

I would probably set it to -1 or at least to 10 seconds but regardless of the value of this variable, this shouldn't cause so high cpu usage (unless of course you can't connect to the server all the time).

[squidge]

For some reason, I took the condition of that while loop to return immediately if there wasn't a connection. In which case, you need need to plant some qDebug() in there to see exactly what lines are being constantly run to give you that 99% cpu usage. You could run it through the debugger too, but sometimes qDebug() can be easier to find faults like this.

[wysota]

If the cpu usage is at 99% then there has to be a busy loop somewhere. Or some logical problem...