QTreeView + QSortFilterProxyModel take a lot of system resources when scrolling

[arek_pl]

Hello. I've got problem with QTreeView. I need to insert max 45000 rows to this widged, but when i exceed 5000 my program uses almost 100% CPU power.

Creating QStandardItemModel, QTreeView and QSortFilterProxyModel:

model = new QStandardItemModel(0, COLUMN_COUNT, parent);
    model->setHeaderData(0, Qt::Horizontal, QObject::tr("Lp."));
    model->setHeaderData(1, Qt::Horizontal, QObject::tr("Data"));
    model->setHeaderData(2, Qt::Horizontal, QObject::tr("Czas"));
    model->setHeaderData(3, Qt::Horizontal, QObject::tr("Opis modułu"));
    model->setHeaderData(4, Qt::Horizontal, QObject::tr("Zdarzenie"));
    model->setHeaderData(5, Qt::Horizontal, QObject::tr("Priorytet"));
    model->setHeaderData(6, Qt::Horizontal, QObject::tr("0/1"));
    model->setHeaderData(7, Qt::Horizontal, QObject::tr("Adres"));
    model->setHeaderData(8, Qt::Horizontal, QObject::tr("Kod"));
    model->setHeaderData(9, Qt::Horizontal, QObject::tr("Typ"));
 
    proxyModel = new QSortFilterProxyModel(this);
    proxyModel->setDynamicSortFilter(false); // dynamic sort filter is off
    proxyModel->setSourceModel(model);
    proxyModel->setFilterRegExp("[0-3]");
    proxyModel->setFilterKeyColumn(5);
 
    proxyView = new QTreeView(this);
    proxyView->setRootIsDecorated(false);
    proxyView->setAlternatingRowColors(true);
    proxyView->setModel(proxyModel);
    proxyView->setSortingEnabled(true); //sorting enabled
    proxyView->setEditTriggers(QAbstractItemView::NoEditTriggers);
    proxyView->setContentsMargins(0,0,0,0);
    proxyView->sortByColumn(2, Qt::AscendingOrder);
 
    QVBoxLayout *mainLayout = new QVBoxLayout;
    mainLayout->addWidget(proxyView);
    mainLayout->setContentsMargins(0,0,0,0);
 
    setLayout(mainLayout);

model = new QStandardItemModel(0, COLUMN_COUNT, parent);
    model->setHeaderData(0, Qt::Horizontal, QObject::tr("Lp."));
    model->setHeaderData(1, Qt::Horizontal, QObject::tr("Data"));
    model->setHeaderData(2, Qt::Horizontal, QObject::tr("Czas"));
    model->setHeaderData(3, Qt::Horizontal, QObject::tr("Opis modułu"));
    model->setHeaderData(4, Qt::Horizontal, QObject::tr("Zdarzenie"));
    model->setHeaderData(5, Qt::Horizontal, QObject::tr("Priorytet"));
    model->setHeaderData(6, Qt::Horizontal, QObject::tr("0/1"));
    model->setHeaderData(7, Qt::Horizontal, QObject::tr("Adres"));
    model->setHeaderData(8, Qt::Horizontal, QObject::tr("Kod"));
    model->setHeaderData(9, Qt::Horizontal, QObject::tr("Typ"));

    proxyModel = new QSortFilterProxyModel(this);
    proxyModel->setDynamicSortFilter(false); // dynamic sort filter is off
    proxyModel->setSourceModel(model);
    proxyModel->setFilterRegExp("[0-3]");
    proxyModel->setFilterKeyColumn(5);

    proxyView = new QTreeView(this);
    proxyView->setRootIsDecorated(false);
    proxyView->setAlternatingRowColors(true);
    proxyView->setModel(proxyModel);
    proxyView->setSortingEnabled(true); //sorting enabled
    proxyView->setEditTriggers(QAbstractItemView::NoEditTriggers);
    proxyView->setContentsMargins(0,0,0,0);
    proxyView->sortByColumn(2, Qt::AscendingOrder);

    QVBoxLayout *mainLayout = new QVBoxLayout;
    mainLayout->addWidget(proxyView);
    mainLayout->setContentsMargins(0,0,0,0);

    setLayout(mainLayout);

To copy to clipboard, switch view to plain text mode 

Adding row to QTreeView:

void MainEvents::addDeviceEvent(QStandardItemModel *model,  const QDate &date, const QTime &time,
                    const QString &field, const QString &devEvent, const quint8 &priority, const qint8 &bits,
                    const qint8 &address, const qint8 &code, const qint8 &type)
{
 
    model->insertRow(0);
    model->setData(model->index(0,0), model->rowCount());
    model->setData(model->index(0, 1), date);
    model->setData(model->index(0, 2), time.toString("hh:mm:ss.zzz"));
    model->setData(model->index(0, 3), field);
    QStandardItem *eventItem = new QStandardItem(devEvent);
    QFont font;
    if (bits != 0) font.setBold(true);
    eventItem->setFont(font);
    eventItem->setForeground(QBrush(eventNameColor[priority]));
 
    model->setItem(0,4,eventItem);    
    model->setData(model->index(0, 5), priority);
    model->setData(model->index(0, 6), bits);
    model->setData(model->index(0, 7), address);
    model->setData(model->index(0, 8), code);
    model->setData(model->index(0, 9), type);
 
}

void MainEvents::addDeviceEvent(QStandardItemModel *model,  const QDate &date, const QTime &time,
                    const QString &field, const QString &devEvent, const quint8 &priority, const qint8 &bits,
                    const qint8 &address, const qint8 &code, const qint8 &type)
{

    model->insertRow(0);
    model->setData(model->index(0,0), model->rowCount());
    model->setData(model->index(0, 1), date);
    model->setData(model->index(0, 2), time.toString("hh:mm:ss.zzz"));
    model->setData(model->index(0, 3), field);
    QStandardItem *eventItem = new QStandardItem(devEvent);
    QFont font;
    if (bits != 0) font.setBold(true);
    eventItem->setFont(font);
    eventItem->setForeground(QBrush(eventNameColor[priority]));

    model->setItem(0,4,eventItem);    
    model->setData(model->index(0, 5), priority);
    model->setData(model->index(0, 6), bits);
    model->setData(model->index(0, 7), address);
    model->setData(model->index(0, 8), code);
    model->setData(model->index(0, 9), type);

}

To copy to clipboard, switch view to plain text mode 

I get data from serial device or from file in offline mode. In both modes problem exists so i include only reading data from file source:

void MainEvents::openEventsFile(const QString &fileName)
{
    QFile file(fileName);
    if (!file.open(QIODevice::ReadOnly))
    {
        QMessageBox::warning(this,tr("ENAP"),tr("Nie można otworzyć pliku %1:\n%2.").arg(file.fileName()).arg(file.errorString()));
        return;
    }
    QDataStream in(&file);
    in.setVersion(QDataStream::Qt_4_6);
 
    quint32 header;
    in >> header;
    if (header != fileHeader)
    {
        QMessageBox::warning(this, tr("ENAP"),tr("Otwarty plik nie jest prawidłowym plikiem zdarzeń."));
        file.close();
        return;
    }
    QApplication::setOverrideCursor(Qt::WaitCursor);
    do
    {
        QDate date;
        in >> date;
        QTime time;
        in >> time;
        QString field;
        in >> field;
        QString event;
        in >> event;
        quint8 priority;
        in >> priority;
        quint8 bit;
        in >> bit;
        quint8 address;
        in >> address;
        quint8 code;
        in >> code;
        quint8 type;
        in >> type;
        addDeviceEvent(model, date, time, field, event, priority, bit, address, code, type);
    }
    while (!in.atEnd());
    QApplication::restoreOverrideCursor();
    file.close();
    proxyModel->invalidate(); 
}

void MainEvents::openEventsFile(const QString &fileName)
{
    QFile file(fileName);
    if (!file.open(QIODevice::ReadOnly))
    {
        QMessageBox::warning(this,tr("ENAP"),tr("Nie można otworzyć pliku %1:\n%2.").arg(file.fileName()).arg(file.errorString()));
        return;
    }
    QDataStream in(&file);
    in.setVersion(QDataStream::Qt_4_6);

    quint32 header;
    in >> header;
    if (header != fileHeader)
    {
        QMessageBox::warning(this, tr("ENAP"),tr("Otwarty plik nie jest prawidłowym plikiem zdarzeń."));
        file.close();
        return;
    }
    QApplication::setOverrideCursor(Qt::WaitCursor);
    do
    {
        QDate date;
        in >> date;
        QTime time;
        in >> time;
        QString field;
        in >> field;
        QString event;
        in >> event;
        quint8 priority;
        in >> priority;
        quint8 bit;
        in >> bit;
        quint8 address;
        in >> address;
        quint8 code;
        in >> code;
        quint8 type;
        in >> type;
        addDeviceEvent(model, date, time, field, event, priority, bit, address, code, type);
    }
    while (!in.atEnd());
    QApplication::restoreOverrideCursor();
    file.close();
    proxyModel->invalidate(); 
}

To copy to clipboard, switch view to plain text mode 

Is there any posibility to add 45000 elements to QTreeView with QSortFilterProxyModel (with ability to sorting and filtering) and scroll it without glitches?

Thank you in advance.

[Coises]

Since efficiency appears to be a problem here, I think I would start by replacing the QStandardItemModel with a custom subclass of QAbstractItemModel or QAbstractTableModel. Since each column appears to have a fixed data type (and many columns are simple, fixed-length types), you can probably build a much more efficient data structure for a backing store than one based on QStandardItems.

If the strings involved have known maximum lengths, you can even create a single “plain old data” struct to hold each row.

It will probably prove difficult to maintain both fast insertion of new records and fast sorting. If insertion speed is more important, keep one linked list and add to the end, then build an indexed list when you (re)sort. If fast sorting is more important, maintain a sorted list of pointers for each order in which you’ll care to sort and add each new record to all the lists in the appropriate place; but keeping that even reasonably efficient for insertions into a 45000-record set will require some creativity.

Alternatively, it might be that using SQLite to maintain the table would work well for this amount of data.

[arek_pl]

Thanks for fast reply! I'll try to replace this QStandardItemModel.

[arek_pl]

I found simplier solution. I've set UniformRowHeights to true. Now everything works fine.