Crash in libssl when using Qt's Postgresql driver

[tigloo]

Hello,

I have just migrated from a 32bit system to a new 64bit system, running Kubuntu Natty and Qt 4.7.2. I have recompiled an application that used to work perfectly on the 32bit system and now I am getting a crash while inserting rows into a Postgresql database.

The following is the code, it just reads a CSV file line by line and inserts each line into a Postgresql table. The database connection has been opened successfully at a different place in the application. The crash occurs at query.exec() inside the while() loop on line 51:

void TickerImportThread::run()
{
    status = true;
 
    QSqlQuery query;
    QFile file(filename);
 
    if ( !file.open(QIODevice::Text | QIODevice::ReadOnly) )
    {
        status = false;
        return;
    }
 
    QSqlDatabase::database().transaction();
 
    if ( !query.exec("DELETE FROM \"bardata_M1\"") )
    {
        status = false;
        QSqlError error = query.lastError();
        qDebug() << Q_FUNC_INFO << "SQL Error: " << error.databaseText() << error.driverText();
        QSqlDatabase::database().rollback();
        return;
    }
 
    QByteArray line;
    QList<QByteArray> items;
    long filesize = file.size();
    long completed = 0;
    int percentage = 0;
 
    if ( !query.prepare("INSERT INTO \"bardata_M1\" (symbol, \"time\", open, high, low, \"close\") VALUES ('EURUSD', :v_time, :v_open, :v_high, :v_low, :v_close)") )
    {
        status = false;
        QSqlError error = query.lastError();
        qDebug() << Q_FUNC_INFO << "SQL Error: " << error.databaseText() << error.driverText();
        QSqlDatabase::database().rollback();
        return;
    }
 
    while ( !file.atEnd() )
    {
        line = file.readLine();
        items = line.split(',');
 
        query.bindValue(":v_time", QString(items[0]) + " " + QString(items[1]));
        query.bindValue(":v_open", items[2]);
        query.bindValue(":v_high", items[3]);
        query.bindValue(":v_low", items[4]);
        query.bindValue(":v_close", items[5]);
 
        if ( !query.exec() )
        {
            status = false;
            QSqlError error = query.lastError();
            qDebug() << Q_FUNC_INFO << "SQL Error: " << error.databaseText() << error.driverText();
            QSqlDatabase::database().rollback();
            return;
        }
 
        completed += line.length();
        double newpercentage = (double)completed / (double)filesize * 100;
        if ( round(newpercentage) > percentage )
        {
            percentage = round(newpercentage);
            emit importProgress(percentage);
        }
    }
 
    emit importProgress(100);
 
    QSqlDatabase::database().commit();
 
    file.close();
}

void TickerImportThread::run()
{
    status = true;

    QSqlQuery query;
    QFile file(filename);

    if ( !file.open(QIODevice::Text | QIODevice::ReadOnly) )
    {
        status = false;
        return;
    }

    QSqlDatabase::database().transaction();

    if ( !query.exec("DELETE FROM \"bardata_M1\"") )
    {
        status = false;
        QSqlError error = query.lastError();
        qDebug() << Q_FUNC_INFO << "SQL Error: " << error.databaseText() << error.driverText();
        QSqlDatabase::database().rollback();
        return;
    }

    QByteArray line;
    QList<QByteArray> items;
    long filesize = file.size();
    long completed = 0;
    int percentage = 0;

    if ( !query.prepare("INSERT INTO \"bardata_M1\" (symbol, \"time\", open, high, low, \"close\") VALUES ('EURUSD', :v_time, :v_open, :v_high, :v_low, :v_close)") )
    {
        status = false;
        QSqlError error = query.lastError();
        qDebug() << Q_FUNC_INFO << "SQL Error: " << error.databaseText() << error.driverText();
        QSqlDatabase::database().rollback();
        return;
    }

    while ( !file.atEnd() )
    {
        line = file.readLine();
        items = line.split(',');

        query.bindValue(":v_time", QString(items[0]) + " " + QString(items[1]));
        query.bindValue(":v_open", items[2]);
        query.bindValue(":v_high", items[3]);
        query.bindValue(":v_low", items[4]);
        query.bindValue(":v_close", items[5]);

        if ( !query.exec() )
        {
            status = false;
            QSqlError error = query.lastError();
            qDebug() << Q_FUNC_INFO << "SQL Error: " << error.databaseText() << error.driverText();
            QSqlDatabase::database().rollback();
            return;
        }

        completed += line.length();
        double newpercentage = (double)completed / (double)filesize * 100;
        if ( round(newpercentage) > percentage )
        {
            percentage = round(newpercentage);
            emit importProgress(percentage);
        }
    }

    emit importProgress(100);

    QSqlDatabase::database().commit();

    file.close();
}

To copy to clipboard, switch view to plain text mode 

This is the backtrace:

*** glibc detected *** /home/till/forex-build-desktop/forex: double free or corruption (fasttop): 0x000000000094aa70 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x78a8f)[0x7ffff5370a8f]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x73)[0x7ffff53748e3]
/lib/libcrypto.so.0.9.8(CRYPTO_free+0x1d)[0x7fffeb6b0efd]
/lib/libssl.so.0.9.8(SSL_free+0x49)[0x7fffeba0e199]
/usr/lib/libpq.so.5(+0x19e8d)[0x7fffebc45e8d]
/usr/lib/libpq.so.5(+0x11bc9)[0x7fffebc3dbc9]
/usr/lib/libpq.so.5(PQgetResult+0x9d)[0x7fffebc3bbcd]
/usr/lib/libpq.so.5(+0xfe6b)[0x7fffebc3be6b]
/usr/lib/qt4/plugins/sqldrivers/libqsqlpsql.so(+0xe327)[0x7fffebe62327]
/usr/lib/libQtSql.so.4(_ZN9QSqlQuery4execEv+0xc5)[0x7ffff718cf15]
/home/till/forex-build-desktop/forex[0x40fc0e]
/usr/lib/libQtCore.so.4(+0x74175)[0x7ffff60bf175]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c)[0x7ffff5e33d8c]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7ffff53de04d]

*** glibc detected *** /home/till/forex-build-desktop/forex: double free or corruption (fasttop): 0x000000000094aa70 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x78a8f)[0x7ffff5370a8f]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x73)[0x7ffff53748e3]
/lib/libcrypto.so.0.9.8(CRYPTO_free+0x1d)[0x7fffeb6b0efd]
/lib/libssl.so.0.9.8(SSL_free+0x49)[0x7fffeba0e199]
/usr/lib/libpq.so.5(+0x19e8d)[0x7fffebc45e8d]
/usr/lib/libpq.so.5(+0x11bc9)[0x7fffebc3dbc9]
/usr/lib/libpq.so.5(PQgetResult+0x9d)[0x7fffebc3bbcd]
/usr/lib/libpq.so.5(+0xfe6b)[0x7fffebc3be6b]
/usr/lib/qt4/plugins/sqldrivers/libqsqlpsql.so(+0xe327)[0x7fffebe62327]
/usr/lib/libQtSql.so.4(_ZN9QSqlQuery4execEv+0xc5)[0x7ffff718cf15]
/home/till/forex-build-desktop/forex[0x40fc0e]
/usr/lib/libQtCore.so.4(+0x74175)[0x7ffff60bf175]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c)[0x7ffff5e33d8c]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7ffff53de04d]

To copy to clipboard, switch view to plain text mode 

I'm not sure how to debug this, my guess is that it is libssl's fault, but I could not find anything related on the web. Is there any problem in my code? If not, any tips for debugging or even fixing this?

[tigloo]

I found the bug by stepping through Qt's sources. The problem is obviously related to concurrent accesses to the database from two different threads. I had one thread running that was performing queries on the database and another thread (my main application thread) that was using a timer to query the database as well. When I disabled the timer, the crash disappeared.

This makes me wonder about the thread safety of QtSql, because the documentation does not mention that two QSqlQuery instances are not allowed to access the same QSqlDatabase instance at the same time. In my case, a work around for the crash is trivial, so I'll just disable the timer while other queries are running.

[ChrisW67]

From Threads and the SQL Module:

A connection can only be used from within the thread that created it. Moving connections between threads or creating queries from a different thread is not supported.

In addition, the third party libraries used by the QSqlDrivers can impose further restrictions on using the SQL Module in a multithreaded program. Consult the manual of your database client for more information

From PostgreSQL 29.17. Behavior in Threaded Programs

One thread restriction is that no two threads attempt to manipulate the same PGconn object at the same time. In particular, you cannot issue concurrent commands from different threads through the same connection object. (If you need to run concurrent commands, use multiple connections.)

Since Qt will be using such a connection, it also inherits PostgreSQL limitations.

Options include using QSqlDatabase::cloneDatabase() to create an independent database connection based on an existing (default) one or creating a named connection for each thread.

[tigloo]

Thanks for the information, I expected to find that in the class documentation itself but did not see it there. Using QSqlDatabase::cloneDatabase() is a great tip.