Memory leak in my threaded SSL server application

[moosa]

Hi All,

I built an SSL server by subclassing the QTcpServer. The stress test I did is connecting and disconnecting from the server about 200,000 times (client connects and disconnects each 100ms). I can see that the memory consumption gets larger as the number of connections and drops increase.

My client application on the other hand doesn't have this phenomena.

I tested the server and client under Windows XP and Ubuntu 9.04.

Honestly I don't know if this is a QT issue or a bug in my application. But I will be happy if someone could guide me where the memory leak could be.

Please feel free to comment on my code as well.

Thank you in advance!

And here is the code:
Pay attention that the defines NO_AUTH_TIME_LIMIT and UNSECURE_CONNECTION are off!

sslserver.h:

#ifndef SSLSERVER_H
#define SSLSERVER_H
 
#include <QTcpServer>
#include <QSslSocket>
 
class SslServer : public QTcpServer
{
    Q_OBJECT
public:
    SslServer(QObject *parent=0);
    ~SslServer();
 
protected:
    void incomingConnection(int socketDescriptor);
 
private slots:
    void error(QAbstractSocket::SocketError err);
};
#endif // SSLSERVER_H

#ifndef SSLSERVER_H
#define SSLSERVER_H

#include <QTcpServer>
#include <QSslSocket>

class SslServer : public QTcpServer
{
    Q_OBJECT
public:
    SslServer(QObject *parent=0);
    ~SslServer();

protected:
    void incomingConnection(int socketDescriptor);

private slots:
    void error(QAbstractSocket::SocketError err);
};
#endif // SSLSERVER_H

To copy to clipboard, switch view to plain text mode 

sslserver.c:

#include "sslserver.h"
#include "messagethread.h"
 
SslServer::SslServer(QObject *parent):QTcpServer(parent)
{
}
 
SslServer::~SslServer()
{
}
 
void SslServer::incomingConnection(int socketDescriptor)
{
    MessageThread *thread = new MessageThread(socketDescriptor, this);
    connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
    thread->start();
 }
void SslServer::error(QAbstractSocket::SocketError err)
{
    qDebug()<<tr("Error: %1").arg(err);
}

#include "sslserver.h"
#include "messagethread.h"

SslServer::SslServer(QObject *parent):QTcpServer(parent)
{
}

SslServer::~SslServer()
{
}

void SslServer::incomingConnection(int socketDescriptor)
{
    MessageThread *thread = new MessageThread(socketDescriptor, this);
    connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
    thread->start();
 }
void SslServer::error(QAbstractSocket::SocketError err)
{
    qDebug()<<tr("Error: %1").arg(err);
}

To copy to clipboard, switch view to plain text mode 

messagethread.h:

#ifndef MESSAGETHREAD_H
#define MESSAGETHREAD_H
 
#include <QThread>
#include <QSslSocket>
#include <QAbstractSocket>
#include <QTimer>
#include <QMutex>
 
#define THREAD_MAX_PENDING_CONN_INTERVAL    (2000)
 
class MessageThread : public QThread
{
    Q_OBJECT
public:
    MessageThread(int socketDescriptor, QObject *parent = 0);
    ~MessageThread();
 
protected:
    void run(void);
 
private:
    int                 m_socketDescriptor;     //  Socket descriptor of the connection
    quint16             m_numOfExpectedBytes;   //  Number of expected bytes from the client
    quint32             m_currentUserId;        //  Current user ID
    bool                m_cleanExitDone;        //  Indicates that an exit procedure was done
    QMutex             *m_mutex;                //  Mutex to perform clean exit without interrupts
 
#ifdef UNSECURE_CONNECTION                      //  Define m_socket to be SSL or TCP socket
    QTcpSocket         *m_socket;
#else
    QSslSocket         *m_socket;
#endif
 
#ifndef NO_AUTH_TIME_LIMIT                      //  If NO_AUTH_TIME_LIMIT is NOT present, the server will reject an unauthorized
    QTimer             *m_authTimer;            //  connection which is held for a defined time frame
    bool                m_authorized;
#endif
 
private slots:
    void processReadyRead(void);
    void checkPending(void);
    void performCleanExit(void);
};
 
#endif // MESSAGETHREAD_H

#ifndef MESSAGETHREAD_H
#define MESSAGETHREAD_H

#include <QThread>
#include <QSslSocket>
#include <QAbstractSocket>
#include <QTimer>
#include <QMutex>

#define THREAD_MAX_PENDING_CONN_INTERVAL    (2000)

class MessageThread : public QThread
{
    Q_OBJECT
public:
    MessageThread(int socketDescriptor, QObject *parent = 0);
    ~MessageThread();

protected:
    void run(void);

private:
    int                 m_socketDescriptor;     //  Socket descriptor of the connection
    quint16             m_numOfExpectedBytes;   //  Number of expected bytes from the client
    quint32             m_currentUserId;        //  Current user ID
    bool                m_cleanExitDone;        //  Indicates that an exit procedure was done
    QMutex             *m_mutex;                //  Mutex to perform clean exit without interrupts

#ifdef UNSECURE_CONNECTION                      //  Define m_socket to be SSL or TCP socket
    QTcpSocket         *m_socket;
#else
    QSslSocket         *m_socket;
#endif

#ifndef NO_AUTH_TIME_LIMIT                      //  If NO_AUTH_TIME_LIMIT is NOT present, the server will reject an unauthorized
    QTimer             *m_authTimer;            //  connection which is held for a defined time frame
    bool                m_authorized;
#endif

private slots:
    void processReadyRead(void);
    void checkPending(void);
    void performCleanExit(void);
};

#endif // MESSAGETHREAD_H

To copy to clipboard, switch view to plain text mode 

messagethread.c:

#include "messagethread.h"
#include <QIODevice>
#include <QDataStream>
#include <QAbstractSocket>
#include <QFile>
#include <QCoreApplication>
#include <QDebug>
 
MessageThread::MessageThread(int socketDescriptor, QObject *parent):QThread(parent)
{
    m_socketDescriptor = socketDescriptor;
    m_numOfExpectedBytes = 0;
    m_cleanExitDone = false;
    m_mutex = new QMutex();
}
 
MessageThread::~MessageThread()
{
    delete m_mutex;
}
 
void MessageThread::run(void)
{
#ifdef UNSECURE_CONNECTION
    m_socket = new QTcpSocket();
#else
    m_socket = new QSslSocket();
#endif
    if (m_socket->setSocketDescriptor(m_socketDescriptor))
    {
        QList<QSslCertificate> certificateList;
        certificateList.append(QSslCertificate());
        connect(m_socket, SIGNAL(readyRead()), this, SLOT(processReadyRead()), Qt::DirectConnection);
        connect(m_socket, SIGNAL(disconnected()), this, SLOT(performCleanExit()));
#ifdef UNSECURE_CONNECTION
        //  Nothing to do if the connection is unsecure
#else
        //  Add the key that will make the handshake
        m_socket->setPrivateKey(":/certs/my-key.pem");
        m_socket->setLocalCertificate(":/certs/my-cert.pem");
 
        //  Add the certificate authority to the list of certificate sent
        QFile f(":/certs/cacert.pem");
        f.open(QIODevice::ReadOnly);
        QSslCertificate cert(f.readAll());
        if (!cert.isValid())
            qDebug("Invaild certificate");
        m_socket->addCaCertificate(cert);
        m_socket->startServerEncryption();
#endif
    }
    else
    {
        qDebug()<<tr("Could not create socket from descriptor");
        qDebug()<<m_socket->errorString();
        performCleanExit();
    }
#ifndef NO_AUTH_TIME_LIMIT
    m_authorized = false;
    m_authTimer = new QTimer();
    connect(m_authTimer, SIGNAL(timeout()), this, SLOT(checkPending()), Qt::DirectConnection);
    m_authTimer->setInterval(THREAD_MAX_PENDING_CONN_INTERVAL);
    m_authTimer->setSingleShot(true);
    m_authTimer->start();
#endif
    /* Call the event loop of the thread */
    exec();
}
 
void MessageThread::performCleanExit(void)
{
    m_mutex->lock();
    if (!m_cleanExitDone)
    {
        QAbstractSocket::SocketState s;
        m_cleanExitDone = true;
        s = m_socket->state();
        // Remove the connection
        if (s != QAbstractSocket::UnconnectedState && s != QAbstractSocket::ClosingState)
        {
            m_socket->disconnectFromHost();
            m_socket->waitForDisconnected();
        }
        disconnect(m_socket, 0, 0, 0);
        m_socket->deleteLater();
    }
#ifndef NO_AUTH_TIME_LIMIT
    m_authTimer->deleteLater();
#endif
    m_mutex->unlock();
    //  Cover the case where we started with this function and the check pending
    //  is called right after quit().
    QCoreApplication::processEvents();
    quit();
}
 
 
void MessageThread::processReadyRead(void)
{
    if (m_socket->bytesAvailable() == 0)
    {
        return;
    }
    m_socket->readAll();
    //performCleanExit();
}
 
void MessageThread::checkPending(void)
{
#ifndef NO_AUTH_TIME_LIMIT
    if (!m_authorized)
    {
        qDebug("Time out baby");
        performCleanExit();
    }
#endif
}

#include "messagethread.h"
#include <QIODevice>
#include <QDataStream>
#include <QAbstractSocket>
#include <QFile>
#include <QCoreApplication>
#include <QDebug>

MessageThread::MessageThread(int socketDescriptor, QObject *parent):QThread(parent)
{
    m_socketDescriptor = socketDescriptor;
    m_numOfExpectedBytes = 0;
    m_cleanExitDone = false;
    m_mutex = new QMutex();
}

MessageThread::~MessageThread()
{
    delete m_mutex;
}

void MessageThread::run(void)
{
#ifdef UNSECURE_CONNECTION
    m_socket = new QTcpSocket();
#else
    m_socket = new QSslSocket();
#endif
    if (m_socket->setSocketDescriptor(m_socketDescriptor))
    {
        QList<QSslCertificate> certificateList;
        certificateList.append(QSslCertificate());
        connect(m_socket, SIGNAL(readyRead()), this, SLOT(processReadyRead()), Qt::DirectConnection);
        connect(m_socket, SIGNAL(disconnected()), this, SLOT(performCleanExit()));
#ifdef UNSECURE_CONNECTION
        //  Nothing to do if the connection is unsecure
#else
        //  Add the key that will make the handshake
        m_socket->setPrivateKey(":/certs/my-key.pem");
        m_socket->setLocalCertificate(":/certs/my-cert.pem");

        //  Add the certificate authority to the list of certificate sent
        QFile f(":/certs/cacert.pem");
        f.open(QIODevice::ReadOnly);
        QSslCertificate cert(f.readAll());
        if (!cert.isValid())
            qDebug("Invaild certificate");
        m_socket->addCaCertificate(cert);
        m_socket->startServerEncryption();
#endif
    }
    else
    {
        qDebug()<<tr("Could not create socket from descriptor");
        qDebug()<<m_socket->errorString();
        performCleanExit();
    }
#ifndef NO_AUTH_TIME_LIMIT
    m_authorized = false;
    m_authTimer = new QTimer();
    connect(m_authTimer, SIGNAL(timeout()), this, SLOT(checkPending()), Qt::DirectConnection);
    m_authTimer->setInterval(THREAD_MAX_PENDING_CONN_INTERVAL);
    m_authTimer->setSingleShot(true);
    m_authTimer->start();
#endif
    /* Call the event loop of the thread */
    exec();
}

void MessageThread::performCleanExit(void)
{
    m_mutex->lock();
    if (!m_cleanExitDone)
    {
        QAbstractSocket::SocketState s;
        m_cleanExitDone = true;
        s = m_socket->state();
        // Remove the connection
        if (s != QAbstractSocket::UnconnectedState && s != QAbstractSocket::ClosingState)
        {
            m_socket->disconnectFromHost();
            m_socket->waitForDisconnected();
        }
        disconnect(m_socket, 0, 0, 0);
        m_socket->deleteLater();
    }
#ifndef NO_AUTH_TIME_LIMIT
    m_authTimer->deleteLater();
#endif
    m_mutex->unlock();
    //  Cover the case where we started with this function and the check pending
    //  is called right after quit().
    QCoreApplication::processEvents();
    quit();
}


void MessageThread::processReadyRead(void)
{
    if (m_socket->bytesAvailable() == 0)
    {
        return;
    }
    m_socket->readAll();
    //performCleanExit();
}

void MessageThread::checkPending(void)
{
#ifndef NO_AUTH_TIME_LIMIT
    if (!m_authorized)
    {
        qDebug("Time out baby");
        performCleanExit();
    }
#endif
}

To copy to clipboard, switch view to plain text mode 

[TMan]

Did you check whether your MessageThread objects are really deleted when finished?

[moosa]

Yes, I printed a qDebug message on the MessageThread destructor when it is deleted and saw that the destructor is being called.

[TMan]

And the QTcpSocket/QSslSocket you create in MessageThread, are those deleted? Also, have you tried running with valgrind?

[moosa]

I am getting into the valgrind tool right now.

About the QSslSocket, yes I am deleting it.
One thing that decreased the memory leak noticeably (but not at all) is moving the following code from the "performCleanExit" function in the thread:

m_socket->deleteLater();
m_authTimer->deleteLater();

m_socket->deleteLater();
m_authTimer->deleteLater();

To copy to clipboard, switch view to plain text mode 

to the following code in thread destructor:

delete m_socket;
delete m_authTimer;

delete m_socket;
delete m_authTimer;

To copy to clipboard, switch view to plain text mode 

It may be related to that that the deleteLater() function does not enter the event loop (in order to be executed) before the thread is deleted.

I will make further investigation and try to provide the client code as well in order to be able to reproduce the memory leak easily.

Thanks in advance

[sadjoker]

Hello, i`ll watch that thread closely for your final investigation about the memory leak. I`ve got the same problem with a multithreaded QSslSocket app myself and the memory of my server app goes up like ~700kb per client connect/disconnect. And my code is similar to yours.

[moosa]

By the way, I am using QT version 4.6.2.

There is a reported bug about QSslSocket memory leak in the following link: http://bugreports.qt.nokia.com/browse/QTBUG-6504
I do not know whether or not this the problem I am having here.

Regards

[moosa]

Finally I found the code causing the memory leak:
In the server side, setting the private key causes a memory leak. It seems like a bug in QT.

The code that causes the memory leak is:

m_socket->setPrivateKey(":/certs/my-key.pem");  //  Add the key that will make the handshake
m_socket->setLocalCertificate(":/certs/my-cert.pem");

m_socket->setPrivateKey(":/certs/my-key.pem");  //  Add the key that will make the handshake
m_socket->setLocalCertificate(":/certs/my-cert.pem");

To copy to clipboard, switch view to plain text mode 

[moosa]

I think this is the bug reported:
http://bugreports.qt.nokia.com/browse/QTBUG-6504