QTableView performances

[miraks]

Hi,

I replaced all calls to reset by something based on beginRemoveRows + endRemoveRows + beginInsertRows + endInsertRows.

But I still have the same performances (bad performances ) and the same number of calls on data method:

##methode                                          nb call        millisecondes    moyenne     min      max           propre       moyenne propre   %
##SKGObjectModel::data                             2706598        46342,4          0,02        0           64,25      35899,5         0,01          33,75%
##QAbstractItemModel::refresh-fill                 14             48237         3445,5         0,09     47583,1       19442,2      1388,73          18,28%
##SKGObjectModelBase::rowCount                     2709500        15354,5          0,01        0           53,88      15354,5         0,01          14,44%
##SKGObjectModelBase::index                        2709435        13351,1          0           0           51,89      13351,1         0             12,55%
##SKGObjectModelBase::data                         2353264        11256,9          0           0           64,2       10569,1         0              9,94%

##methode                                          nb call        millisecondes    moyenne     min      max           propre       moyenne propre   %
##SKGObjectModel::data                             2706598        46342,4          0,02        0           64,25      35899,5         0,01          33,75%
##QAbstractItemModel::refresh-fill                 14             48237         3445,5         0,09     47583,1       19442,2      1388,73          18,28%
##SKGObjectModelBase::rowCount                     2709500        15354,5          0,01        0           53,88      15354,5         0,01          14,44%
##SKGObjectModelBase::index                        2709435        13351,1          0           0           51,89      13351,1         0             12,55%
##SKGObjectModelBase::data                         2353264        11256,9          0           0           64,2       10569,1         0              9,94%

To copy to clipboard, switch view to plain text mode 

[wysota]

Did you reimplement data() the way I suggested?

[calmspeaker]

Do not use reset() as you could,because reset() will clear the state of the items in the table, for example, the selected state. And the beginInsertRows(), endInsertRows() will reduce the efficiency. You could emit the signals layoutAboutToBeChanged()before you insert warnings,layoutChanged() after you insert warnings.

The tableview could be very efficent, I have tried to display 100,000 items, and there is no obvious delay.

er, I have a question, what tool do u use to test your codes' efficency?

[miraks]

Did you reimplement data() the way I suggested?

Not yet.

Do not use reset() as you could,because reset() will clear the state of the items in the table, for example, the selected state. And the beginInsertRows(), endInsertRows() will reduce the efficiency. You could emit the signals layoutAboutToBeChanged()before you insert warnings,layoutChanged() after you insert warnings.

In any case, in my scenario, I have to fill the table with around 3400 new lines.
At the beginning the table contains 20 lines.
After a user action, the table must contain around 3400 new lines.
I will try with layoutAboutToBeChanged() and layoutChanged().

The tableview could be very efficient, I have tried to display 100,000 items, and there is no obvious delay.

Did you tried with decorations, colors, alignment ?
I noticed bad performances just by adding background color.

what tool do u use to test your codes' efficency?

My own component.
In fact, my traces system is able (with some build directives activated) to compute the time consumed by a part of code (a method for example).
This system is less time consuming that something like Callgrind.

[calmspeaker]

In any case, in my scenario, I have to fill the table with around 3400 new lines.
At the beginning the table contains 20 lines.
After a user action, the table must contain around 3400 new lines.

I suggest you only call the pair signal(beginlayoutchanged, layoutchanged) once, in your situation, that is a way to improve the efficiency.

Did you tried with decorations, colors, alignment ?
I noticed bad performances just by adding background color.

I only tried to changed all the items' background color every per second.The efficiency is acceptable.

My own component.
In fact, my traces system is able (with some build directives activated) to compute the time consumed by a part of code (a method for example).
This system is less time consuming that something like Callgrind.

Could it be shared?

[wysota]

Have you tried gprof? It's faster than callgrind because it doesn't emulate the cpu and it's standard, so there are tools to use it. You can use it with GCC only, but I take it that's not a problem for you

[miraks]

Hi calmspeaker,

I prepared for you a package containing:
1-A class managing errors with history.
2-A class managing indented traces with error and profiling.
3-A document to explain how to use it.

I hope it will help you.
If you need help, contact me.

I will test your proposal.

[miraks]

Hi,

Here is the result of my test with layoutAboutToBeChanged + layoutChanged.

Without this modification, I get:

##methode                                               nb call       millisecondes  moyenne     min         max          propre        moyenne propre      %  
##SKGObjectModel::data                                  2705890       43950,9           0,02        0           60,99      31727,7        0,01              39,24%
##SKGObjectModelBase::data                              2347017       12969,1           0,01        0           37,98      12309,6        0,01              15,23%
##SKGObjectModelBase::rowCount                          2707786       10864,1           0           0           22,39      10864,1        0                 13,44%
##SKGObjectModelBase::index                             2707746        9032,57          0           0           27,68       9032,57       0                 11,17%
##QAbstractItemModel::reset-fill                             14       21129,6        1509,26        0,19     20674,1        8237,73     588,41              10,19%
##SKGObjectModelBase::headerData                         251522        4541,63          0,02        0           16,52       3054,3        0,01               3,78%
##SKGServices::executeSelectSqliteOrder(QSqlDatabase)       174        2351,37         13,51        0,03       730,59       2351,37      13,51               2,91%
##SKGMainPanel::SKGMainPanel                                  1        4144,91       4144,91     4144,91      4144,91        820,56     820,56               1,01%

##methode                                               nb call       millisecondes  moyenne     min         max          propre        moyenne propre      %  
##SKGObjectModel::data                                  2705890       43950,9           0,02        0           60,99      31727,7        0,01              39,24%
##SKGObjectModelBase::data                              2347017       12969,1           0,01        0           37,98      12309,6        0,01              15,23%
##SKGObjectModelBase::rowCount                          2707786       10864,1           0           0           22,39      10864,1        0                 13,44%
##SKGObjectModelBase::index                             2707746        9032,57          0           0           27,68       9032,57       0                 11,17%
##QAbstractItemModel::reset-fill                             14       21129,6        1509,26        0,19     20674,1        8237,73     588,41              10,19%
##SKGObjectModelBase::headerData                         251522        4541,63          0,02        0           16,52       3054,3        0,01               3,78%
##SKGServices::executeSelectSqliteOrder(QSqlDatabase)       174        2351,37         13,51        0,03       730,59       2351,37      13,51               2,91%
##SKGMainPanel::SKGMainPanel                                  1        4144,91       4144,91     4144,91      4144,91        820,56     820,56               1,01%

To copy to clipboard, switch view to plain text mode 

With the modification:

##methode                                              nb call        millisecondes  moyenne     min         max          propre        moyenne propre      %  
##SKGObjectModel::data                                 1939034        31138,4           0,02        0           32,87     22583,8         0,01              37,16%
##SKGObjectModelBase::data                             1667650         9058,82          0,01        0           27,76      8592,84        0,01              14,14%
##QAbstractItemModel::reset-fill                            14        21189,3        1513,52        0,13     20886,5       8319,58      594,26              13,69%
##SKGObjectModelBase::rowCount                         1941008         7398,6           0           0           35,16      7398,6         0                 12,17%
##SKGObjectModelBase::index                            1940968         6369,83          0           0           20,05      6369,83        0                 10,48%
##SKGServices::executeSelectSqliteOrder(QSqlDatabase)      174         2528,14         14,53        0,03       759,23      2528,14       14,53               4,16%
##SKGObjectModelBase::headerData                        170665         2962,19          0,02        0           12,09      1976,83        0,01               3,25%
##SKGServices::executeSqliteOrder(QSqlDatabase)            379         1612,22          4,25        0,03       533,94       831,15        2,19               1,37%
##SKGMainPanel::SKGMainPanel                                 1         3931,13       3931,13     3931,13      3931,13       715,1       715,1                1,18%

##methode                                              nb call        millisecondes  moyenne     min         max          propre        moyenne propre      %  
##SKGObjectModel::data                                 1939034        31138,4           0,02        0           32,87     22583,8         0,01              37,16%
##SKGObjectModelBase::data                             1667650         9058,82          0,01        0           27,76      8592,84        0,01              14,14%
##QAbstractItemModel::reset-fill                            14        21189,3        1513,52        0,13     20886,5       8319,58      594,26              13,69%
##SKGObjectModelBase::rowCount                         1941008         7398,6           0           0           35,16      7398,6         0                 12,17%
##SKGObjectModelBase::index                            1940968         6369,83          0           0           20,05      6369,83        0                 10,48%
##SKGServices::executeSelectSqliteOrder(QSqlDatabase)      174         2528,14         14,53        0,03       759,23      2528,14       14,53               4,16%
##SKGObjectModelBase::headerData                        170665         2962,19          0,02        0           12,09      1976,83        0,01               3,25%
##SKGServices::executeSqliteOrder(QSqlDatabase)            379         1612,22          4,25        0,03       533,94       831,15        2,19               1,37%
##SKGMainPanel::SKGMainPanel                                 1         3931,13       3931,13     3931,13      3931,13       715,1       715,1                1,18%

To copy to clipboard, switch view to plain text mode 

Conclusion:
It's better because the number of call reduced.
It's not enough.

[wysota]

I really suggest to modify the implementation of data(). It is currently very inefficient and it's one of the most often called methods.

[miraks]

I really suggest to modify the implementation of data(). It is currently very inefficient and it's one of the most often called methods.

I will try but it's not so easy !
I will take you informed.

In any case, thank you for your help.

[miraks]

Hi wysota,

I have got 2 bad news:

1-I optimized data method and after that, I commented in data methods all the code concerning decoration, tooltip, alignment, etc to keep only the text. Even with this, I win only 6% of performances. .
Conclusion: even with perfect code in data, I won't have good performances.

2-The solution based on layoutAboutToBeChanged + layoutChanged generates some regressions.
(indexes seems to be not recomputed and so are still pointing on deleted internal data).

I don't know what to do.

PS1: I attached my new implementation.
PS2: I don't know if it's very important, but I use a proxy (filter & sort) too.

[wysota]

The implementation is still very inefficient. You are still comparing strings instead of enums or numbers and you don't do any serious caching of calculated data.

6% performance gain is already much, don't underestimate it.

[miraks]

Hi wysota,

Today, I have got a good new.
I understood why I have so bad performances.
In fact, it's due to these lines:

ui.kOperationView->verticalHeader()->setResizeMode(QHeaderView::ResizeToContents);
ui.kOperationView->horizontalHeader()->setResizeMode(QHeaderView::ResizeToContents);

ui.kOperationView->verticalHeader()->setResizeMode(QHeaderView::ResizeToContents);
ui.kOperationView->horizontalHeader()->setResizeMode(QHeaderView::ResizeToContents);

To copy to clipboard, switch view to plain text mode 

If I comment these lines, I have got:

##methode                                               , nb call , millisecondes , moyenne   , min        , max     , propre   , moyenne propre
##SKGServices::executeSelectSqliteOrder(QSqlDatabase)   , 173     , 2250.3        , 13.0075    , 0.0319843  , 757.705 , 2250.3  , 13.0075
##SKGObjectModelBase::data                              , 77777   , 2022.72       , 0.0260067  , 0.00187886 , 17.3499 , 2011.33 , 0.0258602
##SKGObjectModelBase::refresh-reset                     , 13      , 4078.05       , 313.696    , 0.00701368 , 3962.67 , 1081.16 , 83.166
##SKGObjectModel::data                                  , 71620   , 2738.77       , 0.0382403  , 0.00292969 , 17.3791 , 776.344 , 0.0108398
##SKGServices::executeSqliteOrder(QSqlDatabase)         , 379     , 1319.9        , 3.48257    , 0.0279003  , 444.459 , 679.188 , 1.79205
##SKGMainPanel::SKGMainPanel                            , 1       , 2419.67       , 2419.67    , 2419.67    , 2419.67 , 372.141 , 372.141
##SKGObjectModelBase::index                             , 73266   , 295.21        , 0.00402929 , 0.00190628 , 10.6721 , 295.21  , 0.00402929
##SKGObjectModelBase::rowCount                          , 73305   , 223.636       , 0.00305076 , 0.00187886 , 11.12   , 223.636 , 0.00305076
##SKGObjectModelBase::clear                             , 17      , 201.88        , 11.8753    , 0.00792778 , 176.784 , 201.88  , 11.8753
##SKGObjectBase::getObjects                             , 62      , 1078.61       , 17.3969    , 0.0670469  , 932.943 , 184.245 , 2.9717
##SKGObjectModelBase::headerData                        , 9244    , 137.556       , 0.0148806  , 0.00188088 , 1.31897 , 104.604 , 0.0113158
##SKGSortFilterProxyModel::filterAcceptsRow             , 4022    , 336.626       , 0.0836961  , 0.023957  , 0.784051 , 90.5983 , 0.0225257

##methode                                               , nb call , millisecondes , moyenne   , min        , max     , propre   , moyenne propre
##SKGServices::executeSelectSqliteOrder(QSqlDatabase)   , 173     , 2250.3        , 13.0075    , 0.0319843  , 757.705 , 2250.3  , 13.0075
##SKGObjectModelBase::data                              , 77777   , 2022.72       , 0.0260067  , 0.00187886 , 17.3499 , 2011.33 , 0.0258602
##SKGObjectModelBase::refresh-reset                     , 13      , 4078.05       , 313.696    , 0.00701368 , 3962.67 , 1081.16 , 83.166
##SKGObjectModel::data                                  , 71620   , 2738.77       , 0.0382403  , 0.00292969 , 17.3791 , 776.344 , 0.0108398
##SKGServices::executeSqliteOrder(QSqlDatabase)         , 379     , 1319.9        , 3.48257    , 0.0279003  , 444.459 , 679.188 , 1.79205
##SKGMainPanel::SKGMainPanel                            , 1       , 2419.67       , 2419.67    , 2419.67    , 2419.67 , 372.141 , 372.141
##SKGObjectModelBase::index                             , 73266   , 295.21        , 0.00402929 , 0.00190628 , 10.6721 , 295.21  , 0.00402929
##SKGObjectModelBase::rowCount                          , 73305   , 223.636       , 0.00305076 , 0.00187886 , 11.12   , 223.636 , 0.00305076
##SKGObjectModelBase::clear                             , 17      , 201.88        , 11.8753    , 0.00792778 , 176.784 , 201.88  , 11.8753
##SKGObjectBase::getObjects                             , 62      , 1078.61       , 17.3969    , 0.0670469  , 932.943 , 184.245 , 2.9717
##SKGObjectModelBase::headerData                        , 9244    , 137.556       , 0.0148806  , 0.00188088 , 1.31897 , 104.604 , 0.0113158
##SKGSortFilterProxyModel::filterAcceptsRow             , 4022    , 336.626       , 0.0836961  , 0.023957  , 0.784051 , 90.5983 , 0.0225257

To copy to clipboard, switch view to plain text mode 

As you can see, the number of calls is drastically reduced and performances are better (before 43s into data method, now 2s into data method ==> it's better than 6% ).

Is it normal to have so bad performances when ResizeToContents is used ?

[wysota]

Yes, that's normal. sizeHint is queried all the time which in turns calls data() all the time, etc. I thought you knew that so I didn't even bother to mention it, sorry

But honestly it's because of your hand-crafted profiler - it doesn't monitor methods which are called automatically by Qt or that are implemented by Qt, hence no references to sizeHint reading and hence my impression you weren't resizing columns to contents.