QTableView sort column with numbers very slow

Hi, i'm trying to sort a QtableView's column containing numbers, using QSortFilterProxyModel::sort(), but is veeery slow. It's ok sorting strings, good speed, but horrible with numbers.
Any way to solve this please?
Thanks.

Please show some code demonstrating the problem.

Its a very simple code:

First i connect the QTableView header to the slot:

Code:

---

connect(gameView->horizontalHeader(), SIGNAL(sectionClicked(int)), this, SLOT(saveSortOrder(int)));

---

and this is the slot at this moment:

Code:

---

void RetroGC::saveSortOrder(int ind) {
 
  romProxy->sort(ind, Qt::AscendingOrder);
// gameView->horizontalHeader()->setSortIndicatorShown(true);
// QSettings sortSettings(QString("Data/SSettings/%1/%2.ini").arg(CurrType).arg(CurrID), QSettings::IniFormat);
// sortSettings.setValue( "Sort/Col", ind );
// sortSettings.setValue("Sort/Mode", gameView->horizontalHeader()->sortIndicatorOrder());
 
}

---

Takes about 5 seconds to sort a table with 1.864 rows.

I've commented some lines for testing. I've tried gameView->sortByColum() and still slow.

EDIT: Ok, after some testings, i've found the problem. Only some rows have a different value because the default value is 0. If i fill the rows with incrementing integers, 1, 2, 3 ,4 ,5... etc., the sort speed its amazing :). So, here is the new question: Is there any way to fix this problem? Any way to ignore that rows with the 0 value in that cell?

What is the source model of the proxy? How is its data() method implemented?

The source model is a reimplemented QAbstractTableModel, beacuse i need lazy population for big (10.000+ rows) tables.

Here is the full data() implementation:

Code:

---

QVariant RomList::data(const QModelIndex &index, int role) const
{
 
    if (!index.isValid())
        return QVariant();
 
    if (index.row() >= ROMs.size() || index.row() < 0)
        return QVariant();
 
    if (role == Qt::SizeHintRole){
        if(index.column() == 2) {
            return QSize(86,16);
            }
        }
    if (role == Qt::TextAlignmentRole){
        if(index.column() == 3) {
            return Qt::AlignHCenter;
            }
 
    if(index.column() == 4) {
        return Qt::AlignHCenter;
        }
    }
    if (role == Qt::DecorationRole) {
 
        if(index.column() == 2) {
            QFileInfo ROMi(ROMs.at(index.row()));
            QSettings setDATA(QString("Data/ROMdata/%1/%2/%3.ini").arg(CurrentTipo).arg(CurrentSystem).arg(ROMi.completeBaseName()), QSettings::IniFormat);
            return QPixmap(QString("Skins/%1/System/icons/rate%2.png").arg(CurrentSkin).arg(setDATA.value( QString("%1/Rating").arg( ROMi.completeBaseName() )).toString()));
        }
}
         if (role == Qt::DisplayRole) {
             if(index.column() == 0) {
                 QFileInfo ROMi(ROMs.at(index.row()));
                 return ROMi.completeBaseName();
             }
             if(index.column() == 2) {
                 QFileInfo ROMi(ROMs.at(index.row()));
                 QSettings setDATA(QString("Data/ROMdata/%1/%2/%3.ini").arg(CurrentTipo).arg(CurrentSystem).arg(ROMi.completeBaseName()), QSettings::IniFormat);
                 if(!setDATA.value(QString("%1/Rating").arg( ROMi.completeBaseName() )).isNull()) {
                 return setDATA.value( QString("%1/Rating").arg( ROMi.completeBaseName() )).toInt();
                   } else return QVariant();
               }
             if(index.column() == 3) {
             QSettings SetEmu(QString("Data/Emulators/%2/%1.ini").arg(CurrentSystem).arg(CurrentTipo), QSettings::IniFormat);
             if(!SetEmu.value(QString("%1/Name").arg(  SetEmu.value("DefaultEmulator/RGCID").toString()) ).toString().isEmpty()) {
             return SetEmu.value(QString("%1/Name").arg(  SetEmu.value("DefaultEmulator/RGCID").toString()) ).toString();
               }
              }
            if(index.column() == 4) { // This is the column i'm sorting, all row's value is 0 if the game is not played, so there are a lot of "0" cells slowing the sort
 
                QFileInfo ROMi(ROMs.at(index.row()));
                QSettings setDATA(QString("Data/ROMdata/%1/%2/%3.ini").arg(CurrentTipo).arg(CurrentSystem).arg(ROMi.completeBaseName()), QSettings::IniFormat);
                if(!setDATA.value(QString("%1/Played").arg( ROMi.completeBaseName() )).isNull()) {
                return setDATA.value(QString("%1/Played").arg(ROMi.completeBaseName())).toInt();
                  } else {
 
                return 0;
                  }
              }
 
            if(index.column() == 9) {
            return ROMs.at(index.row());
              }
 
   }
             return QVariant();
}

---

Your data() implementation is quite slow (because of creating and accessing the instance of QSettings), maybe that's what causes sorting to be slow since each item is being accessed multiple times and each time you're creating new QSettings instance. Maybe you should cache those values you can in the model?

Ok, i'll try to read all the values, or add all values into the game list, with the names, like "Name;played;rating;etc", and use QString::section() instead of QSettings. Probably that will solve the proble, name sorting is very fast, where there is not any QSettings access.

Thanks wysota, you are a life saver again.

The most obvious improvement is to create the settings object just once and keep it opened as a member variable of the model. I can see you're always creating QSettings with the same parameters so keeping the settings object open will at least prevent it from being parsed at each call to data().

I fixed the slow issue, but theres a new problem.
My custom model has a reimplemented setData() :

Code:

---

bool RomList::setData(const QModelIndex &index, const QVariant &value, int role) {
  if(index.isValid() && role == Qt::DisplayRole  ) {
      if(index.column() == 4) {
          int row = index.row();
          QStringList tmp = ROMs.at(row).split(";");
          tmp.replace(4, QString::number(value.toInt()));
          ROMs.replace(row, tmp.join(";"));
          emit dataChanged(index, index);
          return true;
        } else {
    return false;
        }
    }
  return false;
}

---

All the table data loads from ROMs QstringList, so if i change some cell of column 4, it will be updated, but only works with some cells, some works, some doesn't work, its trange.

Here is the data change:

Code:

---

void RetroGC::incPlayed(QString stipo, QString sid, int num, int row) {
  QSettings romList(QString("Data/ROMLists/%1/%2.rgl").arg(stipo).arg(sid), QSettings::IniFormat);
  QStringList tmp = romList.value(QString("ROMS/%1").arg(num)).toString().split(";");
 
  qint32 xt = gameList->index(row, 4).data(Qt::DisplayRole).toInt();
  xt++;
  tmp.replace(4, QString::number(xt));
  romList.setValue(QString("ROMS/%1").arg(num), tmp.join(";"));
  gameList->setData(  gameList->index(row, 4), xt, Qt::DisplayRole );
}

---

i've been 2 hours stucked and investigating, but i can't find where is the problem.

First of all handle EditRole in your setData implementation in addition to DisplayRole. Second of all I suggest you restructure your program. Make it never access any of the ROM data directly without the model. It's better to add methods to the model class that will perform operations on the data itself than to risk modifying the data outside the model. You could simply have a increaseColumn4(const QModelIndex&) method that would bump up the number in the model and emit dataChanged().

But the user will not add nothing manually, all is done by the app, if i use EditRole it seems to nothing happens.
Ok, i'll restructure all the model thing, so all the ROM data will only managed by the model, if i understood crrectly.

thaks again, and sorry my for programming noobness. >.<

Quote:

---

Originally Posted by aguayro

But the user will not add nothing manually, all is done by the app, if i use EditRole it seems to nothing happens.

---

It's just for completeness and to be safe in case you want to interact with one of many other mechanisms in Qt where distinction between DisplayRole and EditRole does matter.

Quote:

---

Ok, i'll restructure all the model thing, so all the ROM data will only managed by the model, if i understood crrectly.

---

That should help you pinpoint the real problem. If you make sure you only access the data through the model (emitting appropriate signals along the way), problems with model aspects working "sometimes" should go away.

Quote:

---

thaks again, and sorry my for programming noobness. >.<

---

No problem. Been there, done that :)

The problem was in QSortFilterProxyModel and sorting, when i sort, the index are different from the source model. Trying to find the solution myself.

It's strange, but the table still has sorting speed problems, whe all columns values are setted, i can take 5 seconds to sort by column. There is not ini access in the model.
Where could be the problem? i'm using a QSortFilterProxyModel too.

I really need solve the QTableView problems, its the central widget of my app, and can't have a good perfomance with 8000+ rows. I've seen an app that make all this REALLY instantly (coded in Delphi), i need to have that perfomance, if not, my app becomes useless in comparission :(.
This table is going to kill me :(

Here is my data implementation just now (note i've commented the DecorativeRole part to check if that helps to speed, but no luck, still slow):

Code:

---

QVariant rs_GameModel::data(const QModelIndex &index, int role) const {                                                              //   DATA
    if (!index.isValid()) {
        return QVariant();
    }
    if (index.row() >= romCount || index.row() < 0 ) {
        return QVariant();
    }
 
    if (role == Qt::TextAlignmentRole) {
        if (index.column() > 0 && index.column() < 10) {
            return  Qt::AlignHCenter;
        }
 
    }
    if (role == Qt::UserRole) {
        if (index.column() == 0) {
 
           return QString("%1;%2;%3").arg(ROMs.at(index.row()).section(";", 12, 12)).arg(ROMs.at(index.row()).section(";", 14, 14)).arg(ROMs.at(index.row()).section(";", 15, 15));
        }
        if (index.column() == 1) {
            return ROMs.at(index.row()).section(";", 1, 1);
        }
        if (index.column() == 2) {
            return ROMs.at(index.row()).section(";", 2, 2);
        }
        if (index.column() == 3) {
            return ROMs.at(index.row()).section(";", 11, 11);
        }
    }
//    if (role == Qt::DecorationRole) {
//        if (index.column() == 0) {
//            return romicon;
//        }
//        if (index.column() == 1) {
//            QString tmpExt = ROMs.at(index.row()).section(";", 1, 1);
 
//            if (tmpExt.compare("zip",  Qt::CaseInsensitive) == 0 || tmpExt.compare("7Z",  Qt::CaseInsensitive) == 0 || tmpExt.compare("r",  Qt::CaseInsensitive) == 0 ) {
//                return QIcon(QString("Skin/%1/icons/sys/compressed16.png").arg(CurrSkin));
//            } else if (tmpExt.compare("iso",  Qt::CaseInsensitive) == 0 || tmpExt.compare("cue",  Qt::CaseInsensitive) == 0 || tmpExt.compare("bin",  Qt::CaseInsensitive) == 0 ||
//                       tmpExt.compare("nrg",  Qt::CaseInsensitive) == 0 || tmpExt.compare("mdf",  Qt::CaseInsensitive) == 0 || tmpExt.compare("mds",  Qt::CaseInsensitive) == 0 ) {
//                return QIcon(QString("Skin/%1/icons/sys/cddvd16.png").arg(CurrSkin));
//            } else {
//            return QIcon(QString("Skin/%1/icons/sys/cartridge16.png").arg(CurrSkin));
//            }
//        }
 
//        if (index.column() == 2) {
//            if (ROMs.at(index.row()).section(";", 2, 2).toInt() != 0) {
//                return QPixmap(QString("Skin/%1/icons/sys/rate%2.png").arg(CurrSkin).arg(ROMs.at(index.row()).section(";", 2, 2).toInt()));
//            }
//        }
//    }
    if (role == Qt::DisplayRole) {
        if (index.column() == 0) {
            return ROMs.at(index.row()).section(";", 0, 0);
        }
 
        if (index.column() == 3) {
            if (ROMs.at(index.row()).section(";", 3,3) == "rsnull") {
            return cEmulator;
            } else {
                return ROMs.at(index.row()).section(";", 3,3); }
        }
        if (index.column() == 4) {
            return ROMs.at(index.row()).section(";", 4, 4);
        }
        if (index.column() == 5) {
 
            return ROMs.at(index.row()).section(";", 5, 5).replace("rsnull", "");
 
        }
        if (index.column() == 6 ) {
            return ROMs.at(index.row()).section(";", 6, 6);
        }
        if (index.column() == 7) {  // Compañia
 
            return ROMs.at(index.row()).section(";", 7, 7);
 
        }
        if (index.column() == 8) {
 
            return ROMs.at(index.row()).section(";", 8, 8);
 
        }
        if (index.column() == 9) {
 
            return ROMs.at(index.row()).section(";", 9, 9);
 
        }
        if (index.column() == 11) {
            return ROMs.at(index.row()).section(";", 10, 10);
        }
 
    }
 
    return QVariant();
}

---

any help, clue, or anything is appreciated, anything that can help me to put end my QTableHell
thanks

Did you try profiling your app? Instead of chasing ghosts, it will tell you which parts of your program take most time to execute.

What does this section() do?

the list of games is a QStringList like name;language;path;etc... so i use section() to get each column data.

About profiling i'm not sure what it means, i'll google for it (i'm newbie in programming in general)
I'll try anything, this problem is killing me. Maybe Qt can't sort 15.000 rows in < 1 second?

EDIT: i did it (at leas that seems), the problem was the section()

How did you solve it?

My model was working wit a QStringList, each QString was a row, Name;Type;Company;etc... then i was using section() to get the data for each row cell. I've replaced the QStringList with QList<QStrngList>, i didn't know that section() were too slow, but with QList<QStringList> and using just MyListQSList.at(index.row()).at( column-nuber ) is very fast, just instantly:

Code:

---

QVariant rs_GameModel::data(const QModelIndex &index, int role) const { ////////////////////////////////////////////////////////////        DATA
    if (!index.isValid()) {
        return QVariant();
    }
    if (index.row() >= romCount || index.row() < 0 ) {
        return QVariant();
    }
 
    if (role == Qt::TextAlignmentRole) {
        if (index.column() > 0 && index.column() < 10) {
            return  Qt::AlignHCenter;
        }
 
    }
    if (role == Qt::UserRole) {
        if (index.column() == 0) {
            return QString("%1;%2;%3").arg(ROMs.at(index.row()).at(12)).arg(ROMs.at(index.row()).at(14)).arg(ROMs.at(index.row()).at(15));
        }
        if (index.column() == 3) {
            ROMs.at(index.row()).at(11);
        }
 
    }
    if (role == Qt::DecorationRole) {
        if (index.column() == 0) {
            return romicon;
        }
        if (index.column() == 2) {
            return QPixmap(QString("Skin/%1/icons/sys/rate%1.png").arg(CurrSkin));
        }
 
    }
    if (role == Qt::DisplayRole) {
        if (index.column() == 0) {
            return  ROMs.at(index.row()).at(0);
        }
        if (index.column() == 3) {
            if (ROMs.at(index.row()).at(3) != "rsnull") {
            return  ROMs.at(index.row()).at(3);
              } else {
                return cEmulator;
              }
        }
        if (index.column() == 4) {
            return  ROMs.at(index.row()).at(4);
        }
        if (index.column() == 5) {
            return  ROMs.at(index.row()).at(5);
        }
        if (index.column() == 6) {
            return  ROMs.at(index.row()).at(6);
        }
        if (index.column() == 7) {
            return  ROMs.at(index.row()).at(7);
        }
        if (index.column() == 8) {
            return  ROMs.at(index.row()).at(8);
        }
        if (index.column() == 9) {
            return  ROMs.at(index.row()).at(9);
        }
        if (index.column() == 11) {
            return  ROMs.at(index.row()).at(10);
        }
 
    }
 
    return QVariant();
}

---

I suposse will discover what is fast and what is slow in qt with the time, and programmng wit it :)

Qt has nothing to do so. In every language search string section will take longer than getting the item from list or array.