Serial loosing data and QAbstractSerial

[Momergil]

Hello!

I'm almost having a headache because of a problem related to reading from a serial device (through the library QAbstractSerial).

I'm using a software that, through a QAbstracSerial port, communicates with a microprocessor ColdFire by sending commands to it (simple chars) and reading and interpreting its answers.

I was doing a series of advances in my software and everything was running fine. But I made some improvements in the software so it would have more commands in it (i.e. nothing that would change its logic of sending and receiving + interpreting the data, only adding some more QPushButtons and more if/else in the readyRead() function for it to understand the ColdFire answers) and since than the software simply began to fail in reading the data received by the serial port. I commented all new changes and even run the old versions of it, but the same problem persistet. I decided to use one of those freewares avaliable in the internet to "talk" to serial ports and it demonstrates that it wasn't a problem of the ColdFire programming or in the serial cables (I even used a very nice oscilloscope that can read and "translate" the data that passes through a serial port, so I could verify that the data being received by the computer was OK). But despite all changes, the new problem simply didn't dissapeared, and I'm beginning to worry if the problem is with the QAbstractSerial library.

To be more precise, the problem is that the readyRead() function is not "capturing" all data. Sometimes this means an entire line, but some times i means the lost of the first byte in all lines received. Its very important to notice that I didn't change any part of the QAbstractSerial code, neither I changed something special in the computer or else something algorithmically relevant in the software (the go-backs to previous versions with the persistent error appearing dispatches this possibility). So at the end I lost 2h of my daying trying lots of things and calling experts in computer programming and nothing worked. Could somebody give a tip of what maybe is happening?

Here is the important parts of my readyRead() function:

QByteArray bytesReceived;
    Sem.acquire(1);
    while (port1->bytesAvailable() > 0)
        bytesReceived.append(port1->readLine());  //I don't know why, but QAbstractSerial's readlAll() simply doesn't work, so I had to do this...
    Sem.release(1);
 
    if (!bytesReceived.isEmpty())
    {
        emit sendMessage(bytesReceived,Qt::blue); //This sends what was received to a QPlainTextEditor in the QMainWindow
 
        if (testesendorealizado == true)
        {
            qDebug() << "Dados/Struct recebida: Tamanho: " << bytesReceived.size() << "Dados: " <<
            bytesReceived;
 
            if (ultimodadofoiincompleto == false)
            {
                if (bytesReceived[0] == BT_RQ_SEARCH || bytesReceived[0] == '&') //This part tries to see the first byte in what was received, and always the first                                                                                                             //byte is simply lost!
                {
                    ...
                    ...
                }
                else qDebug() << "Not a known structure";
            }
        }
    }

QByteArray bytesReceived;
    Sem.acquire(1);
    while (port1->bytesAvailable() > 0)
        bytesReceived.append(port1->readLine());  //I don't know why, but QAbstractSerial's readlAll() simply doesn't work, so I had to do this...
    Sem.release(1);

    if (!bytesReceived.isEmpty())
    {
        emit sendMessage(bytesReceived,Qt::blue); //This sends what was received to a QPlainTextEditor in the QMainWindow

        if (testesendorealizado == true)
        {
            qDebug() << "Dados/Struct recebida: Tamanho: " << bytesReceived.size() << "Dados: " <<
            bytesReceived;

            if (ultimodadofoiincompleto == false)
            {
                if (bytesReceived[0] == BT_RQ_SEARCH || bytesReceived[0] == '&') //This part tries to see the first byte in what was received, and always the first                                                                                                             //byte is simply lost!
                {
                    ...
                    ...
                }
                else qDebug() << "Not a known structure";
            }
        }
    }

To copy to clipboard, switch view to plain text mode 

One more very interesting thing: apart from the times when an entire line is lost by the readyRead(), the lost of the first byte allways happens in the second relevant reading of the serial interface - what makes this problem even more strange.


Could somebody help, please?

Thanks!


Momergil

[ChrisW67]

readyRead() is emitted by QIODevice when new data arrives. This does not mean a complete, for whatever definition of complete you might use, set of data has been received/is available. readAll() gives you all the data that is available, which may include zero or more complete responses and an incomplete fragment. You need to collect data as it is received until you have a complete response before attempting to process a complete response, and then leave any remaining fragment in the buffer for later processing.

This logic has been explained many times in this forum in regard to network-connected QIODevices.

[marcvanriet]

Are the bytes showing in your QPlainTextEdit in the mainwindow ?

You could put a qDebug() << "data received";after the emit SendMessage(). This way you can easily see if a message is received in different chuncks (what ChrisW67 is talking about).

Regards,
Marc

P.S. Only 2 hours of debugging serial communication ? I usually count in days for that

[kuzulis]

2 Momergil

I advise you to look at the library QSerialDevice version 2.0.
The reasons for this, see here.

[Momergil]

readyRead() is emitted by QIODevice when new data arrives. This does not mean a complete, for whatever definition of complete you might use, set of data has been received/is available. readAll() gives you all the data that is available, which may include zero or more complete responses and an incomplete fragment. You need to collect data as it is received until you have a complete response before attempting to process a complete response, and then leave any remaining fragment in the buffer for later processing.

This logic has been explained many times in this forum in regard to network-connected QIODevices.

Hello, Chris!

First, thanks for the reply.

Now I'm familiar with the way QIODevice::readyRead() treats data from the serial, since I had to work with that in my software in previous versions and in all the other softwares I used serial communication. But in no case this problem appeared. The problem is that this doesn't seems to be the problem: remember that I sad that the software was working fine, than I did some modifications, than the problem appears and now it doesn't matter if I comment the modifications or go back to other previous versions and the problem persist, while it was checking by other softwares and by the osciloscope that ColdFire is doing everything OK?

Now by complete I mean all data received. For example, imagine that the sender (the ColdFire microprocessor in this case) sends a array of bytes like "0123456789". Now I would expect that readyRead() would either capture at once all the line, or else split it in parts (let us say, first "01234" and than "56789"). But the problem is that readyRead() is simply cutting out the first byte, so what appears is "123456789", without the first character "0", and some times it seems that is ignoring entire groups of data received (when the ColdFire microprocessor is started, it sends 3 lines of bytes and sometimes readyRead() is capturing only one or two and sometimes it shows all, so big parts of data are lost).

I was figuring out if that wouldn't be some sort of incompatibility between the lastest version of QtSDK and QAbstractSerial, since when I was testing my software it was in my friend's computer and his Qt is a newest version than mine. (his: 2.3.1).

Are the bytes showing in your QPlainTextEdit in the mainwindow ?

You could put a qDebug() << "data received";after the emit SendMessage(). This way you can easily see if a message is received in different chuncks (what ChrisW67 is talking about).

Regards,
Marc

P.S. Only 2 hours of debugging serial communication ? I usually count in days for that

Hello marcvanriet,

If I'm not mistaken I did this already, but now I don't remeber the results ^^ I only know it didn't solved the problem neither helped in any way, so I cut it out :P and yes, emit sendMessage() sends the data received to a QPlainTextEdit in the mainwindow. Now about the 2 hors, well the entire project must be ready in 3 days, so that is already quite a tremendous lost of time! xD

Kuzulis,

I will see about this 2.0 library.


Thanks for all the help,


Momergil

[kuzulis]

2 Momergil

In complete with a library have of test GUI examples are: /tests/guiapp, /tests/guiapp2.
Take them and check the reception of data from its ColdFire.

If your data too will be lost - there's a problem in the library, and I will try correct it.
If not - there's a problem in your code.

Try to bring a minimal project that reproduces the problem.

I would also like to know the version and type of your OS, as well as a libraries.