Solved, thanks. The trick is to use QAbstractItemView.openPersistentEditor(index) as render just gives the pixmaps of the widget without functionality. So one has to implement the createEditor and an empty setModelData method in the delegate.
However, if one does that for many items it gets slow. For me it doesn't matter since I anyway just wanted the button to show up if the mouse hovers over it, so its just one persistent editor