How to limit TCP writes to particular size and then block untlil the data is read

[ustulation]

ok. I had this problem that client asks for 9-10 MB's of data per message and it sends around 15 messages. It doesn't process data equally fast as Server writes it. So the memory consumption at the server end was peaking to around 200 MB's per client connection. This was seen as a concern and i was asked to reduce it. Merely on using waitForBytesWritten( -1 ) immediately after QTcpSocket::write(..) the memory consumption reduced to 30-35 MB at peak. This behaviour is however evidently only in Windows 7. In Win XP the peak memory consumption reduced from 200 MB's to 100 MB's so this is still viewed as a concern. Do you think it is necessary to control it further as tester is concerned that such memory usage will restrict the number of clients the server can connect to before running out of memory?

What function exists that can actually tell me that the client has done reading the sent data from it's read buffer? Then i'll wait for this and only when it's over will i process the other bufferred messages to be replied to/sent?

[wysota]

The best thing you can do is queue requests. If the client requests 15 messages, send him just one and queue 14 remaining ones until sending the first one is complete. Only then process the next one. This way you'll be using memory for only one message per client. Blocking until sending is complete is not a good idea because your program can't do other things (like serving other clients) at the same time.

[ustulation]

right this is what i think i exactly need. You wrote: "send him just one and queue 14 remaining ones until sending the first one is complete" -- How do i know that the first one is complete? Client does not do it at application level. So is there any Qt function to detect that client has read the message from it's receiving window?

[wysota]

How do i know that the first one is complete?

Connect to bytesWritten() and/or monitor what write() returns.

So is there any Qt function to detect that client has read the message from it's receiving window?

No at there would be no sense in doing that. The way TCP works is that it requires acknowledging every segment it sends and if it doesn't receive proper ACKs from the other side, it retransmits data starting from the last ack'ed octet. Thus all unacknowledged data sits in the write buffer of the sender. If the write buffer is full the socket stops accepting data from the application (so write() will return a value smaller than the size of data you fed it with). Thus implicitly all that means that if the socket stops accepting data, it means the other side can't process it fast enough. Of course you have to bear in mind that the receiver has its own read buffer that is filled with incoming data. If the other app is written with QTcpSocket and you allow the app to process its events, then QTcpSocket will read pending data into its own buffers. By default the buffer is unlimited so as long as the application is responsive, only network congestion and not application processing speed will influence the capacity of the write buffer on the sending side.

[ustulation]

thank you so much. I'll try a few things based on what you said. One last query: is there anything similar to QAbstractSocket::setReadBufferSize( qint64 size ) for WriteBufferSize(..) ? Then i can precisely control how much memory consumption i want befor Socket blocks on further writes.

[wysota]

One last query: is there anything similar to QAbstractSocket::setReadBufferSize( qint64 size ) for WriteBufferSize(..) ?

No, I can't see an equivalent but you can implement it by subclassing QTcpSocket and reimplementing write related virtual methods.

Then i can precisely control how much memory consumption i want befor Socket blocks on further writes.

Not entirely. Remember that the operating system has buffers of its own.