Problems with maximum data sent over a QTcpSocket

[jordip]

Hi, I am trying to create a basic videoconference program for an university assignment.
I am using Qt 4.5.2 on Linux.

When I send full images (something like 230k) over the socket it waits in the socket.disconnect() on the sender side and on the receiver side it only goes to 170k or so of bytesAvailable().

I am sending images through QTcpSocket like this:

Sender's side:

int NetworkSender::sendData (char *image, int dataSize, unsigned short port)
{
	QTcpSocket socket;
	int return_value;
 
	socket.connectToHost(remoteHost, port);
 
	QByteArray block;
	QDataStream out(&block, QIODevice::WriteOnly);
	out.setVersion(QDataStream::Qt_4_0);
	out << (quint32)0;
	out.writeBytes((const char*)image, dataSize); 
	out.device()->seek(0);
	out << (quint32)(block.size() - sizeof(quint32));
 
	return_value = socket.write(block);
	//socket.flush();
	//socket.waitForBytesWritten(20);
 
	socket.disconnectFromHost();
	socket.waitForDisconnected();
 
	if (return_value != -1) {
 
		return 0;
	}else {
		return -1;
	}
}

int NetworkSender::sendData (char *image, int dataSize, unsigned short port)
{
	QTcpSocket socket;
	int return_value;

	socket.connectToHost(remoteHost, port);

	QByteArray block;
	QDataStream out(&block, QIODevice::WriteOnly);
	out.setVersion(QDataStream::Qt_4_0);
	out << (quint32)0;
	out.writeBytes((const char*)image, dataSize); 
	out.device()->seek(0);
	out << (quint32)(block.size() - sizeof(quint32));

	return_value = socket.write(block);
	//socket.flush();
	//socket.waitForBytesWritten(20);
	
	socket.disconnectFromHost();
	socket.waitForDisconnected();

	if (return_value != -1) {

		return 0;
	}else {
		return -1;
	}
}

To copy to clipboard, switch view to plain text mode 

On the receiver side I have a thread, like this:

void NetworkReceiverThread::run()
 {
	QTcpSocket socket;
//	socket.setReadBufferSize(500000);
 
	int timeout = 2 * 1000;
 
	if (!socket.setSocketDescriptor(socketDescriptor)) {
		emit error(socket.error(), socket.errorString());
		return;
	}
 
	while (socket.bytesAvailable() < (int)sizeof(quint32)) {
		if (!socket.waitForReadyRead(timeout)) {
			emit error(socket.error(), socket.errorString());
			return;
		}
	}
 
	quint32 blockSize;
	QDataStream in(&socket);
	in.setVersion(QDataStream::Qt_4_0);
	in >> blockSize;
	while (socket.bytesAvailable() < blockSize) {
		if (!socket.waitForReadyRead(timeout)) {
			qDebug("4");
			emit error(socket.error(), socket.errorString());
			return;
		}
	}
       qDebug("5");
 
	unsigned int data_len;
	in.readBytes(data, data_len);
 
	qDebug("received data");
	qDebug(QString::number(data_len).toStdString().data());
 
	emit(done());
}

void NetworkReceiverThread::run()
 {
	QTcpSocket socket;
//	socket.setReadBufferSize(500000);

	int timeout = 2 * 1000;

	if (!socket.setSocketDescriptor(socketDescriptor)) {
		emit error(socket.error(), socket.errorString());
		return;
	}

	while (socket.bytesAvailable() < (int)sizeof(quint32)) {
		if (!socket.waitForReadyRead(timeout)) {
			emit error(socket.error(), socket.errorString());
			return;
		}
	}

	quint32 blockSize;
	QDataStream in(&socket);
	in.setVersion(QDataStream::Qt_4_0);
	in >> blockSize;
	while (socket.bytesAvailable() < blockSize) {
		if (!socket.waitForReadyRead(timeout)) {
			qDebug("4");
			emit error(socket.error(), socket.errorString());
			return;
		}
	}
       qDebug("5");

	unsigned int data_len;
	in.readBytes(data, data_len);

	qDebug("received data");
	qDebug(QString::number(data_len).toStdString().data());

	emit(done());
}

To copy to clipboard, switch view to plain text mode 

As I said if I try to put a debug on the reader inside " while (socket.bytesAvailable() < blockSize) {" printing bytes available I got something like 80k , 120k 160k and then it stops.
I have tried to set manually the buffer of the receiver as can be seen as a commented line, same result.
I have tried also calling flush() on the sender just after the write(), I got this:

QNativeSocketEngine::write() was not called in QAbstractSocket::ConnectedState
QAbstractSocket::waitForDisconnected() is not allowed in UnconnectedState

I have tried waitForBytesWritten() but same result

I am testing on loopback (127.0.0.1)

[^NyAw^]

Hi,

If you want to use Threads into reciver application with a socket call "exec()" instead and connect the SIGNAL "readyRead" from the socket to your slot.

[high_flyer]

Don't wait until you have enough bytes available to read, rather, read all the bytes that are available, and count them, and stop reading when you got all the bytes you need.
The problem is, that it might be that the bytes you are expecting are more then the internal socket buffer size, in which case, you will never get enough bytes on bytesAvailable().

[faldzip]

The problem you faced is a normal socket behavior. Notice that QIODevice::write() also returna how many byte were actually written. When receiving you need to read as many byte as it arrives, store it in some buffer and repeat it until you have all required data in your buffer.

[jordip]

Thanks for you input!
I have changed the code, now I read as there are bytes available instead of waiting for the whole thing to come, anyway it is still not working ;(
Also, I am sure that the data + position pointer mangling can be done much better ...

char *data;
	uint data_len = 0;
 
	quint32 blockSize;
	QDataStream in(&socket);
	unsigned int position = 0;
 
	in.setVersion(QDataStream::Qt_4_5);	
	in >> blockSize;
 
	data = new char[blockSize];
 
	while (position < blockSize) {
		in.readRawData (data + position, data_len);
		blockSize -= data_len;
		position += data_len;
		qDebug(QString ("pos: " + QString::number(position) + " " + QString::number(data_len)).toStdString().data());
		if (blockSize > 0) {
			if (!socket.waitForReadyRead(timeout)) {
				qDebug("4");
				emit error(socket.error(), socket.errorString());
				return;
			}
		}
	}

char *data;
	uint data_len = 0;
	
	quint32 blockSize;
	QDataStream in(&socket);
	unsigned int position = 0;
	
	in.setVersion(QDataStream::Qt_4_5);	
	in >> blockSize;
	
	data = new char[blockSize];
	
	while (position < blockSize) {
		in.readRawData (data + position, data_len);
		blockSize -= data_len;
		position += data_len;
		qDebug(QString ("pos: " + QString::number(position) + " " + QString::number(data_len)).toStdString().data());
		if (blockSize > 0) {
			if (!socket.waitForReadyRead(timeout)) {
				qDebug("4");
				emit error(socket.error(), socket.errorString());
				return;
			}
		}
	}

To copy to clipboard, switch view to plain text mode 

The very clumsy qDebug up there prints position 0 in each iteration of the loop, so, something should be wrong with the sender, I guess.

This is my sender:

//socket.connectToHost(remoteHost, port);
	socket.connectToHost("127.0.0.1", port);
/*
        if ( socket.state() != QAbstractSocket::ConnectedState ) 
	{
		qDebug(socket.errorString().toStdString().data());
		return -1;
	}
*/	
	QByteArray block;
	QDataStream out(&block, QIODevice::WriteOnly);
	out.setVersion(QDataStream::Qt_4_5);
 
	out << (quint32)0;
	out.writeBytes((const char*)image, dataSize); 
	out.device()->seek(0);
	out << (quint32)(block.size() - sizeof(quint32));
 
	int bytes_written = 0;
	int bytes_to_write = block.size();
 
	while (bytes_written < bytes_to_write)
	{	
		written = socket.write(block);
		socket.flush();
		bytes_written += written;
 
		qDebug(QString ("write: " + QString::number(written) + " " + QString::number(bytes_to_write)).toStdString().data());	
 
		socket.waitForBytesWritten(20);
	}

//socket.connectToHost(remoteHost, port);
	socket.connectToHost("127.0.0.1", port);
/*
        if ( socket.state() != QAbstractSocket::ConnectedState ) 
	{
		qDebug(socket.errorString().toStdString().data());
		return -1;
	}
*/	
	QByteArray block;
	QDataStream out(&block, QIODevice::WriteOnly);
	out.setVersion(QDataStream::Qt_4_5);
	
	out << (quint32)0;
	out.writeBytes((const char*)image, dataSize); 
	out.device()->seek(0);
	out << (quint32)(block.size() - sizeof(quint32));

	int bytes_written = 0;
	int bytes_to_write = block.size();
	
	while (bytes_written < bytes_to_write)
	{	
		written = socket.write(block);
		socket.flush();
		bytes_written += written;
		
		qDebug(QString ("write: " + QString::number(written) + " " + QString::number(bytes_to_write)).toStdString().data());	
	
		socket.waitForBytesWritten(20);
	}

To copy to clipboard, switch view to plain text mode 

The qDebug shows that I am writing the datastream in only one call and I get stuck waiting.

If I uncomment the check for the connection status, I can see I am not connected and I errorString is unknown error If I run netstat then, I can see a lot of TIME_WAIT connections on that port.
In the case I send a minimun amount of data seems to connect thought

I am quite clueless at this point

[faldzip]

you have set data_len to 0 and never update it so you always let your stream to read 0 bytes. Use QAbstractSocket::bytesAvailable() to get available bytes count.

[mgoetz]

I'd not use the waitFor functions.
Connect slots to the readyRead() signal of the socket and do the reading there.

[avril]

hi, may be you need to wait for connected before calling socket.write()

try add
socket.waitForConnected(); between the line socket.connectToHost() and socket.write(block)

I encountered this problem also, and the unknown error was fixed after adding it.

[gorka_sm]

Hello , one question , when the following code is executed , what is the value of socketDescriptor or how do you obtain the value of socketDescriptor


if (!socket.setSocketDescriptor(socketDescriptor)) {
emit error(socket.error(), socket.errorString());
return;
}

Hi, I am trying to create a basic videoconference program for an university assignment.
I am using Qt 4.5.2 on Linux.

When I send full images (something like 230k) over the socket it waits in the socket.disconnect() on the sender side and on the receiver side it only goes to 170k or so of bytesAvailable().

I am sending images through QTcpSocket like this:

Sender's side:

int NetworkSender::sendData (char *image, int dataSize, unsigned short port)
{
	QTcpSocket socket;
	int return_value;
 
	socket.connectToHost(remoteHost, port);
 
	QByteArray block;
	QDataStream out(&block, QIODevice::WriteOnly);
	out.setVersion(QDataStream::Qt_4_0);
	out << (quint32)0;
	out.writeBytes((const char*)image, dataSize); 
	out.device()->seek(0);
	out << (quint32)(block.size() - sizeof(quint32));
 
	return_value = socket.write(block);
	//socket.flush();
	//socket.waitForBytesWritten(20);
 
	socket.disconnectFromHost();
	socket.waitForDisconnected();
 
	if (return_value != -1) {
 
		return 0;
	}else {
		return -1;
	}
}

int NetworkSender::sendData (char *image, int dataSize, unsigned short port)
{
	QTcpSocket socket;
	int return_value;

	socket.connectToHost(remoteHost, port);

	QByteArray block;
	QDataStream out(&block, QIODevice::WriteOnly);
	out.setVersion(QDataStream::Qt_4_0);
	out << (quint32)0;
	out.writeBytes((const char*)image, dataSize); 
	out.device()->seek(0);
	out << (quint32)(block.size() - sizeof(quint32));

	return_value = socket.write(block);
	//socket.flush();
	//socket.waitForBytesWritten(20);
	
	socket.disconnectFromHost();
	socket.waitForDisconnected();

	if (return_value != -1) {

		return 0;
	}else {
		return -1;
	}
}

To copy to clipboard, switch view to plain text mode 

On the receiver side I have a thread, like this:

void NetworkReceiverThread::run()
 {
	QTcpSocket socket;
//	socket.setReadBufferSize(500000);
 
	int timeout = 2 * 1000;
 
	if (!socket.setSocketDescriptor(socketDescriptor)) {
		emit error(socket.error(), socket.errorString());
		return;
	}
 
	while (socket.bytesAvailable() < (int)sizeof(quint32)) {
		if (!socket.waitForReadyRead(timeout)) {
			emit error(socket.error(), socket.errorString());
			return;
		}
	}
 
	quint32 blockSize;
	QDataStream in(&socket);
	in.setVersion(QDataStream::Qt_4_0);
	in >> blockSize;
	while (socket.bytesAvailable() < blockSize) {
		if (!socket.waitForReadyRead(timeout)) {
			qDebug("4");
			emit error(socket.error(), socket.errorString());
			return;
		}
	}
       qDebug("5");
 
	unsigned int data_len;
	in.readBytes(data, data_len);
 
	qDebug("received data");
	qDebug(QString::number(data_len).toStdString().data());
 
	emit(done());
}

void NetworkReceiverThread::run()
 {
	QTcpSocket socket;
//	socket.setReadBufferSize(500000);

	int timeout = 2 * 1000;

	if (!socket.setSocketDescriptor(socketDescriptor)) {
		emit error(socket.error(), socket.errorString());
		return;
	}

	while (socket.bytesAvailable() < (int)sizeof(quint32)) {
		if (!socket.waitForReadyRead(timeout)) {
			emit error(socket.error(), socket.errorString());
			return;
		}
	}

	quint32 blockSize;
	QDataStream in(&socket);
	in.setVersion(QDataStream::Qt_4_0);
	in >> blockSize;
	while (socket.bytesAvailable() < blockSize) {
		if (!socket.waitForReadyRead(timeout)) {
			qDebug("4");
			emit error(socket.error(), socket.errorString());
			return;
		}
	}
       qDebug("5");

	unsigned int data_len;
	in.readBytes(data, data_len);

	qDebug("received data");
	qDebug(QString::number(data_len).toStdString().data());

	emit(done());
}

To copy to clipboard, switch view to plain text mode 

As I said if I try to put a debug on the reader inside " while (socket.bytesAvailable() < blockSize) {" printing bytes available I got something like 80k , 120k 160k and then it stops.
I have tried to set manually the buffer of the receiver as can be seen as a commented line, same result.
I have tried also calling flush() on the sender just after the write(), I got this:


I have tried waitForBytesWritten() but same result

I am testing on loopback (127.0.0.1)