Results 1 to 8 of 8

Thread: [PyQt] QTableView displying 100.000.000 rows

  1. #1
    Join Date
    Nov 2012
    Posts
    14
    Qt products
    Platforms
    Windows

    Default [PyQt] QTableView displying 100.000.000 rows

    Hi everyone,

    as the title says I'm trying to display 100.000.000 rows in my QTableView but I'm having some problems. When I launch my script, the widget seems to hang. After some investigations it turns out that the problem seems to be related to the headerData function which is called once for each row in the table. It does not seems to be normal, since I expect it to be called only for the currently displayed rows.
    Here a test code displaying the problem:

    Qt Code:
    1. import sys
    2. from PyQt4.QtCore import *
    3. from PyQt4.QtGui import *
    4.  
    5.  
    6. class MyWindow(QWidget):
    7. def __init__(self, *args):
    8. QWidget.__init__(self, *args)
    9.  
    10. # create table
    11. table = QTableView()
    12. tm = MyTableModel(self)
    13. table.setModel(tm)
    14.  
    15. # layout
    16. layout = QVBoxLayout()
    17. layout.addWidget(table)
    18. self.setLayout(layout)
    19.  
    20. class MyTableModel(QAbstractTableModel):
    21. def rowCount(self, parent):
    22. return 100000000
    23.  
    24. def columnCount(self, parent):
    25. return 10
    26.  
    27. def data(self, index, role):
    28. if not index.isValid():
    29. return QVariant()
    30. elif role != Qt.DisplayRole:
    31. return None
    32. return str((index.row(), index.column()))
    33.  
    34. def headerData(self, section, orientation, role):
    35. print "headerData", section
    36. if role == Qt.DisplayRole:
    37. return section
    38. return None
    39.  
    40. if __name__ == "__main__":
    41. app = QApplication(sys.argv)
    42. w = MyWindow()
    43. w.resize(900, 600)
    44. w.show()
    45. sys.exit(app.exec_())
    To copy to clipboard, switch view to plain text mode 

    Anyone can point me out what I'm doing wrong? is it a bug? The test has been run on windows 7 (64 bit), PyQt 4.11.
    Thanks in advance!

  2. #2
    Join Date
    Dec 2009
    Location
    New Orleans, Louisiana
    Posts
    791
    Thanks
    13
    Thanked 152 Times in 150 Posts
    Qt products
    Qt5
    Platforms
    MacOS X

    Default Re: [PyQt] QTableView displying 100.000.000 rows

    Does a user interface with 100 million rows really sound like a good idea?

  3. #3
    Join Date
    Nov 2012
    Posts
    14
    Qt products
    Platforms
    Windows

    Default Re: [PyQt] QTableView displying 100.000.000 rows

    In the years of big data the answer is probably yes

  4. #4
    Join Date
    Dec 2009
    Location
    New Orleans, Louisiana
    Posts
    791
    Thanks
    13
    Thanked 152 Times in 150 Posts
    Qt products
    Qt5
    Platforms
    MacOS X

    Default Re: [PyQt] QTableView displying 100.000.000 rows

    Quote Originally Posted by Gad82 View Post
    In the years of big data the answer is probably yes
    Good luck with that then...

    Edit: Regarding your original question, are you by chance using QHeaderView::ResizeToContents?
    Last edited by jefftee; 15th June 2015 at 16:42.

  5. #5
    Join Date
    Nov 2012
    Posts
    14
    Qt products
    Platforms
    Windows

    Default Re: [PyQt] QTableView displying 100.000.000 rows

    No, as shown in the test script I only set up a simple model and then put it into the QTableView....

  6. #6
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    3,969
    Thanks
    232
    Thanked 627 Times in 617 Posts
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows Android

    Default Re: [PyQt] QTableView displying 100.000.000 rows

    In the years of big data the answer is probably yes
    In the years of big data, no one actually looks at the big data. If they look at it at all, they look at visualizations that condense the data into something a human viewer can comprehend. The individual items are for the most part, nearly irrelevant. No one, absolutely no one, will look at anything more than the first page or two of a table with 100 million entries, so why even bother with such a thing?

    If you feel you must do this, then you should implement a model which retrieves only what the user is currently viewing on the screen and not attempt to build either a model or a view that contains all of it.

  7. #7
    Join Date
    Dec 2009
    Location
    New Orleans, Louisiana
    Posts
    791
    Thanks
    13
    Thanked 152 Times in 150 Posts
    Qt products
    Qt5
    Platforms
    MacOS X

    Default Re: [PyQt] QTableView displying 100.000.000 rows

    Looks like you are not differentiating between horizontal and vertical headers. If you don't want header data for rows, then you should be returning QVariant() when the orientation is not horizontal.

    I don't code in Python, if None is not the same as QVariant(), then you should change your code to return QVariant() instead of None. Just guessing but perhaps your headerData is being repeatedly called because you're not returning QVariant() in the case where you have no data to return.

  8. #8
    Join Date
    Apr 2019
    Posts
    1
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: [PyQt] QTableView displying 100.000.000 rows

    Even when returning QVariant() in header data the QTableView header still tries to allocate a vector the size of the row count.
    Is it possible to have hundred millions of rows in the model without an out of memory crash in the table?

Similar Threads

  1. PyQt - QTableView with comboBox
    By elanari in forum Qt Programming
    Replies: 1
    Last Post: 9th May 2011, 14:13
  2. QTableView with icons in rows
    By kasper360 in forum Newbie
    Replies: 6
    Last Post: 5th April 2011, 06:24
  3. Disable row/rows in QTableView
    By scott_hollen in forum Qt Programming
    Replies: 6
    Last Post: 28th March 2011, 18:16
  4. QTableView 2 rows inside every row
    By skuda in forum Qt Programming
    Replies: 3
    Last Post: 22nd April 2009, 09:23
  5. Copying QTableView rows
    By derrickbj in forum Qt Programming
    Replies: 1
    Last Post: 28th September 2006, 00:00

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.