Results 1 to 7 of 7

Thread: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

  1. #1
    Join Date
    Apr 2019
    Posts
    5
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

    the project is called Vivisect and it has been using PyQt5. Vivisect is a binary analysis framework (eg. a disassembler)
    this is a relatively stable project, but while recently attempting to add in filtering capabilities to various QTreeView widgets using QSortFilterProxyModel, i'm getting somewhat inconsistent segfaults, with very little context (ie. none other than printed debug statements).

    i've looked through this thread but i haven't been able to stop the segfaulting completely: https://www.qtcentre.org/threads/692...Source-segfalt

    when i start up analysis, it always crashes (it populates the various widgets):
    Qt Code:
    1. $ vivbin /bin/chown
    2. Loaded (0.0685 sec) /bin/chown
    3. Beginning analysis...
    4. ...analyzing exports.
    5. 0x0200ad31: Emulation Found 0x0200afd0 (from func: 0x0200ad20) via call 0x0200afd0
    6. libpng warning: iCCP: known incorrect sRGB profile
    7. libpng warning: iCCP: known incorrect sRGB profile
    8. ...
    9. Dynamic Branch found at 0x2001d48 call rax
    10. Segmentation fault (core dumped)
    To copy to clipboard, switch view to plain text mode 
    here's the relative code snippits (links to full code listed below):

    Qt Code:
    1. class VivFilterModel(QSortFilterProxyModel):
    2. def __init__(self, parent=None):
    3. QSortFilterProxyModel.__init__(self, parent=parent)
    4. self.setDynamicSortFilter(True)
    5. self.setFilterKeyColumn(-1)
    6.  
    7. def __getattr__(self, name): # the existing "navModel" includes a couple functions
    8. return getattr(self.sourceModel(), name)
    9.  
    10.  
    11. class VivFilterView(QWidget):
    12. '''
    13. This is the primary window for the VQViv*Views if they want to include filtering
    14. '''
    15. window_title = '__undefined__'
    16. view_type = None
    17.  
    18. def __init__(self, vw, vwqgui, *args, **kwargs):
    19. QWidget.__init__(self)
    20.  
    21. self.view = self.view_type(vw, vwqgui, *args, **kwargs)
    22. self.ffilt = VQFilterWidget(self)
    23.  
    24. layout = vq_basics.VBox(self.view, self.ffilt)
    25. self.setLayout(layout)
    26.  
    27. self.ffilt.filterChanged.connect(self.textFilterChanged)
    28. self.setWindowTitle(self.view.window_title)
    29.  
    30. def textFilterChanged(self):
    31. regExp = QtCore.QRegExp(self.ffilt.text(),
    32. self.ffilt.caseSensitivity(),
    33. self.ffilt.patternSyntax())
    34.  
    35. self.view.filterModel.setFilterRegExp(regExp)
    36.  
    37. class VQTreeModel(QtCore.QAbstractItemModel):
    38. '''
    39. A QT tree model that uses the tree API from visgraph
    40. to hold the data...
    41. '''
    42.  
    43. columns = ( 'A first column!', 'The Second Column!')
    44. editable = None
    45. dragable = False
    46.  
    47. def __init__(self, parent=None, columns=None):
    48.  
    49. if columns != None:
    50. self.columns = columns
    51.  
    52. QtCore.QAbstractItemModel.__init__(self, parent=parent)
    53. self.rootnode = VQTreeItem((), None)
    54.  
    55. if self.editable == None:
    56. self.editable = [False,] * len(self.columns)
    57.  
    58. def vqEdited(self, pnode, col, value):
    59. return value
    60.  
    61. def append(self, rowdata, parent=None):
    62. if parent == None:
    63. parent = self.rootnode
    64.  
    65. pidx = self.createIndex(parent.row(), 0, parent)
    66. i = len(parent.children)
    67. self.beginInsertRows(pidx, i, i)
    68. node = parent.append(rowdata)
    69. self.endInsertRows()
    70. self.layoutChanged.emit()
    71. return node
    72. ...
    73. def sort(self, colnum, order=0):
    74. cmpf = VQTreeSorter(colnum, order)
    75. self.layoutAboutToBeChanged.emit()
    76. self.rootnode.children.sort(cmp=cmpf)
    77. self.layoutChanged.emit()
    78.  
    79. def flags(self, index):
    80. if not index.isValid():
    81. return 0
    82. flags = QtCore.QAbstractItemModel.flags(self, index)
    83. col = index.column()
    84. if self.editable[col]:
    85. flags |= QtCore.Qt.ItemIsEditable
    86. if self.dragable:
    87. flags |= QtCore.Qt.ItemIsDragEnabled
    88. return flags
    89.  
    90. def columnCount(self, parent=None):
    91. return len(self.columns)
    92.  
    93. def data(self, index, role):
    94. if not index.isValid():
    95. return None
    96.  
    97. item = index.internalPointer()
    98. if role == QtCore.Qt.DisplayRole:
    99. return item.data(index.column())
    100.  
    101. if role == QtCore.Qt.UserRole:
    102. return item
    103.  
    104. return None
    105.  
    106. def setData(self, index, value, role=QtCore.Qt.EditRole):
    107.  
    108. node = index.internalPointer()
    109. if not node:
    110. return False
    111.  
    112. # If this is the edit role, fire the vqEdited thing
    113. if role == QtCore.Qt.EditRole:
    114. value = self.vqEdited(node, index.column(), value)
    115. if value == None:
    116. return False
    117.  
    118. node.rowdata[index.column()] = value
    119. self.dataChanged.emit(index, index)
    120.  
    121. return True
    122.  
    123. def headerData(self, column, orientation, role):
    124. if ( orientation == QtCore.Qt.Horizontal and
    125. role == QtCore.Qt.DisplayRole):
    126.  
    127. return self.columns[column]
    128.  
    129. return None
    130.  
    131. def index(self, row, column, parent):
    132.  
    133. if not self.hasIndex(row, column, parent):
    134. return QtCore.QModelIndex()
    135.  
    136. pitem = parent.internalPointer()
    137. if not pitem:
    138. pitem = self.rootnode
    139.  
    140. item = pitem.child(row)
    141. if not item:
    142. return QtCore.QModelIndex()
    143.  
    144. return self.createIndex(row, column, item)
    145.  
    146. def parent(self, index):
    147.  
    148. if not index.isValid():
    149. return QtCore.QModelIndex()
    150.  
    151. item = index.internalPointer()
    152. if not item:
    153. return QtCore.QModelIndex()
    154.  
    155. pitem = item.parent
    156.  
    157. if pitem == self.rootnode:
    158. return QtCore.QModelIndex()
    159.  
    160. if pitem == None:
    161. return QtCore.QModelIndex()
    162.  
    163. return self.createIndex(pitem.row(), 0, pitem)
    164. ...
    To copy to clipboard, switch view to plain text mode 


    the working pull request is here: https://github.com/vivisect/vivisect/pull/237/files
    the full branch is here: https://github.com/atlas0fd00m/vivis...filtered_views

    i expect the bug is some simple thing i'm doing (eg. not providing a parent, or maybe connecting signals in incompatible ways, etc...)
    but i'm at a complete loss. i don't understand why adding in a QSortFilterProxyModel would cause the entire python interpreter to segfault.

    thank you in advance for any help and guidance you can provide? i'm relatively new to pyqt and qt in general, so don't worry about offending me.
    let me know if you need any other context (or clarification)

    @


    Added after 9 minutes:


    i ran out of room for these bits of relevant code.

    Qt Code:
    1. class VQVivFunctionsViewPart(VQVivTreeView):
    2.  
    3. _viv_navcol = 0
    4. window_title = 'Functions'
    5. columns = ('Name','Address', 'Size', 'Ref Count')
    6.  
    7.  
    8. def __init__(self, vw, vwqgui):
    9. VQVivTreeView.__init__(self, vw, vwqgui, withfilter=True)
    10. self.navModel = VivNavModel(self._viv_navcol, self, columns=self.columns)
    11. self.filterModel = VivFilterModel()
    12. self.filterModel.setSourceModel(self.navModel)
    13. self.setModel(self.filterModel)
    14.  
    15. self.vqLoad()
    16. self.vqSizeColumns()
    17.  
    18. class VQVivFunctionsView(VivFilterView):
    19. view_type = VQVivFunctionsViewPart
    20.  
    21. class VQTreeView(QTreeView):
    22.  
    23. def __init__(self, parent=None, cols=None, **kwargs):
    24. QTreeView.__init__(self, parent=parent, **kwargs)
    25. self.setSortingEnabled(True)
    26. self.setAlternatingRowColors(True)
    27.  
    28. if cols != None:
    29. model = VQTreeModel(parent=self, columns=cols)
    30. self.setModel( model )
    31.  
    32. def vqSizeColumns(self):
    33. c = self.model().columnCount()
    34. for i in xrange(c):
    35. self.resizeColumnToContents(i)
    36.  
    37. def setModel(self, model):
    38. model.dataChanged.connect( self.dataChanged )
    39. model.rowsInserted.connect( self.rowsInserted )
    40. return QTreeView.setModel(self, model)
    41.  
    42. class EnviNavModel(vq_tree.VQTreeModel):
    43.  
    44. dragable = True
    45.  
    46. def __init__(self, navcol, parent=None, columns=None):
    47. vq_tree.VQTreeModel.__init__(self, parent=parent, columns=columns)
    48. self.navcol = navcol
    49.  
    50. def mimeData(self, idx):
    51. pnode = idx[0].internalPointer()
    52. expr = pnode.rowdata[self.navcol]
    53. mdata = QtCore.QMimeData()
    54. mdata.setData('envi/expression',str(expr))
    55. return mdata
    To copy to clipboard, switch view to plain text mode 

    if you are looking through the codebase, the PyQt5 files of interest should be:

    vivisect/qt/views.py
    vqt/tree.py
    envi/qt/memory.py
    vqt/common.py
    vivisect/qt/main.py

    i realize it's a little complex, but i appreciate the help, and i'm happy to provide whatever context may make it easier to troubleshoot.

    thanks!
    @
    Last edited by atlas; 3rd April 2019 at 20:29.

  2. #2
    Join Date
    Apr 2019
    Posts
    5
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

    i don't know if it's at all related, but since migrating to PyQt5 (from PyQt4), i've been getting a ton of these messages:

    Qt Code:
    1. QXcbConnection: XCB error: 8 (BadMatch), sequence: 3577, resource id: 190841046, major code: 130 (Unknown), minor code: 3
    To copy to clipboard, switch view to plain text mode 
    (running Linux/Kubuntu 18.04)

  3. #3
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

    The filter model should not cause any problem by itself, but it might be triggering bugs in the source model.

    The code looks pretty good though.
    I can see only one unchecked access of an internal pointer: in data() when returning the value for DisplayRole.

    You also don't need to emit layoutChanged() in append, begin/endInsertRows() handle that correctly.

    I see you are connecting to the model's dataChanged() and rowsInserted() signals in your view.
    Maybe it is holding some data that gets invalidate when the filter removes entries?

    Cheers,
    _

  4. The following user says thank you to anda_skoa for this useful post:

    atlas (16th April 2019)

  5. #4
    Join Date
    Apr 2019
    Posts
    5
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

    thank you for your insights! i somehow missed your post. i'll have to check my notifications.

    i'll look into that. i also had another user hit me off-list, recommending a change to a couple lines which call model() but which should also directly access sourceModel():

    in "vivisect/qt/views.py":

    - idx = self.model().createIndex(pnode.row(), col, pnode)
    + idx = self.model().sourceModel().createIndex(pnode.row() , col, pnode)
    # We are *not* the edit role...
    - self.model().setData(idx, val, role=QtCore.Qt.DisplayRole)
    + self.model().sourceModel().setData(idx, val, role=QtCore.Qt.DisplayRole)
    (thanks Norm!)

    hopefully these pointers can help identify the instability.
    again, i appreciate your insights!
    i'll let you know what i find.


    Added after 17 minutes:


    Quote Originally Posted by anda_skoa View Post
    You also don't need to emit layoutChanged() in append, begin/endInsertRows() handle that correctly.

    I see you are connecting to the model's dataChanged() and rowsInserted() signals in your view.
    Maybe it is holding some data that gets invalidate when the filter removes entries?

    _
    if i remove the call to emit, nothing added to the view. i end up with an empty view. i'm not doubting what you're saying about begin/endInsertRows(), i'm wondering how my code is doing odd stuff which is keeping it from operating correctly. and if that may also cause my problems.
    as for holding data that gets invalidated when the filter removes entries, unfortunately the segfaults are occurring before i get to filter anything.

    thanks again!
    Last edited by atlas; 16th April 2019 at 18:55.

  6. #5
    Join Date
    Apr 2019
    Posts
    5
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

    Quote Originally Posted by anda_skoa View Post
    The filter model should not cause any problem by itself, but it might be triggering bugs in the source model.

    The code looks pretty good though.
    I can see only one unchecked access of an internal pointer: in data() when returning the value for DisplayRole.

    You also don't need to emit layoutChanged() in append, begin/endInsertRows() handle that correctly.

    I see you are connecting to the model's dataChanged() and rowsInserted() signals in your view.
    Maybe it is holding some data that gets invalidate when the filter removes entries?

    Cheers,
    _
    i'm having pretty good luck with the changes requested and no longer segfaulting....
    i'm now struggling with dramatic delays since adding the filter, and i think it may have to do with my event handling.
    i'm trying to remove the self.layoutChanged.emit() and everything is much faster, but the QTreeView displays nothing. i can tell that there's data in the model, because filtering takes a while (this test bin has 20000 entries in the tree) to filter. i've tried connecting things differently, but i'm not seeing changes. i've connected the model to some debugging functions, and i can *see* that rowsAboutToBeInserted and rowsInserted signals are emitted, but nothing seems to cause the QTreeView to visually show any of the data.

    how can i get the QTreeView to adjust what data it thinks is there without emitting layoutChanged?

    i tried going back to the code pre-filtering and removing the emits, and have similar troubles. however, oddly, i get a few entries listed. i can see that there are way more items than listed, because when i change from sort-ascending to sort-descending, the items are completely different ones (ie. i'm looking at but a window of the available entries). is there some way to get QTreeView to refresh how many items it's supposed to be displaying?

    help? thanks in advance

  7. #6
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

    This is very strange, layoutChanged() should definitely not be needed when adding or removing rows.

    When you say "going back to pre-filtering", do you mean you set the main model directly on the view?
    And it still doesn't behave correctly?

    Cheers,
    _

  8. The following user says thank you to anda_skoa for this useful post:

    atlas (3rd May 2019)

  9. #7
    Join Date
    Apr 2019
    Posts
    5
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: PyQt5 SegFault when adding rows to QTreeView using QSortFilterProxyModel

    Quote Originally Posted by anda_skoa View Post
    This is very strange, layoutChanged() should definitely not be needed when adding or removing rows.

    When you say "going back to pre-filtering", do you mean you set the main model directly on the view?
    And it still doesn't behave correctly?

    Cheers,
    _

    correct, that's what i meant.

    however, long story short, i removed it from a different branch (without filtering) but left in two seemingly minor changes, and things *seem* to work properly.

    https://github.com/atlas0fd00m/vivis...e/atlas_merged
    (vqt/tree.py and vivisect/qt/views.py)

    the filter_views branch is here:
    https://github.com/atlas0fd00m/vivis...filtered_views


    i just had to back out the filter_views branch because, while i no longer segaulted, the gui never come up when filtering. i could get the gui to come up if i removed the layoutChanged.emit() line, but then none of the QTrees showed their data.

    any ideas? it seems that we've worked the code around so that the layoutChanged.emit() causes problems with and without filters it some warped way, that feels like progress.

    @

    btw, i apologize again for the delay in response. i just discovered that i wasn't subscribed to this thread corrected.

Similar Threads

  1. Replies: 1
    Last Post: 18th March 2018, 21:16
  2. Replies: 3
    Last Post: 7th October 2016, 19:21
  3. Replies: 0
    Last Post: 26th June 2014, 15:03
  4. Replies: 2
    Last Post: 15th August 2011, 01:26
  5. Replies: 2
    Last Post: 12th June 2007, 18:23

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.