QTcpServer limit for incoming connections

[mdecandia]

Hi all,
I'm developing a client-server environment using QTcpServer class as fortuneserver in Qt examples.
I need that server may accept a huge number of connections. After almost 1000 connections I have this messages:

socketDescriptor :1187
QSocketNotifier: Socket descriptor too large for select()
QSocketNotifier: Internal error

What does this error means? Is There an upper limit for incoming connections? May this be solved?

Thanks,

Michele De CAndia

[marcel]

Have you tried void QTcpServer::setMaxPendingConnections ( int numConnections )?

If the number of connections grows past numConnections the server will stop accepting new connections but the OS may queue them.

Regards

[mdecandia]

Have you tried void QTcpServer::setMaxPendingConnections ( int numConnections )?

If the number of connections grows past numConnections the server will stop accepting new connections but the OS may queue them.

Regards

Yes, I've done it already and nothing has changed.

[Bitto]

Qt uses BSD sockets, and these have a maximum socket connection count, depending on the kernel. On Windows, you can redefine FD_SETSIZE before including any headers to increase this count. On Unixes, you must reconfigure your kernel.

A typical value of FD_SETSIZE is 1024, which is just about as many sockets as you were able to create.

If you google for FD_SETSIZE, you'll find more info.

[mdecandia]

Qt uses BSD sockets, and these have a maximum socket connection count, depending on the kernel. On Windows, you can redefine FD_SETSIZE before including any headers to increase this count. On Unixes, you must reconfigure your kernel.

A typical value of FD_SETSIZE is 1024, which is just about as many sockets as you were able to create.

If you google for FD_SETSIZE, you'll find more info.

After resizing FD_SETSIZE I don't have that error but I have this one:

QThread::start: Thread creation error:

I found that QTcpServer fails when the number of opened threads is about 280 and I see that threads closing is slower than opening. How may I make faster closing of threads?

[wysota]

Why do you need more than 280 threads or 1000 connections opened at the same time? You'll get a huge overhead unless your machine has dozens of processors.

[mdecandia]

Why do you need more than 280 threads or 1000 connections opened at the same time? You'll get a huge overhead unless your machine has dozens of processors.

I don't need 280 simultaneous thread: the problem is that QTCPserver doesn't close that as faster as incoming request rate.
The threads are deleted from deleteLater() slot, called by

connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()))

but it's not done immediately and finished threads (but not closed) remain opened.

Is there someway to delete immediately QThread object when it finishes his run() method? I can't call delete() inside the object and I can't connect finished() signal to external slot that runs delete(thread) because I can't pass object pointer to him. There's a way to know the pointer of the object that emits the signal from the called slot?


Thanks

[marcel]

Yes.
From the deleteLater() slot emit another signal, for ex threadHandle( QThread* ), which is connected to the object that created the thread.
You can delete it from there, safely, because deleteLater will have exited when the slot in the creator object is executed( queued connection ).

Regards

[wysota]

I don't need 280 simultaneous thread: the problem is that QTCPserver doesn't close that as faster as incoming request rate.

Maybe you're starving the event loop?

Is there someway to delete immediately QThread object when it finishes his run() method? I can't call delete() inside the object and I can't connect finished() signal to external slot that runs delete(thread) because I can't pass object pointer to him. There's a way to know the pointer of the object that emits the signal from the called slot?

How about reusing the threads instead of creating new ones? Create a pool of threads assign them new tasks when they finish the previous ones. If the number of requests exceed the number of idle threads, queue requests. If the request queue is empty, put a thread to sleep using a wait condition.

[Bitto]

Like Wysota says, it sounds like you're replacing one performance problem by another. Firstly, even with native socket code no single machine can efficiently handle more than a few hundred connections. Secondly, threads don't make your code faster unless you have as many processors. The only reason to add as many threads as you're trying to here, is it you want each of them to not block another. Or maybe you really think threads make things faster ;-).

For networking code, it rarely makes any sense to have more than a couple of persistent extra threads next to your main threads. Never restart threads, that's never smart from a performance standpoint. Keep them running. And for thousands of connections, hey, hardware is cheap. Every system that handles that many connections does it by adding hardware, not by adding threads.