a graphicssceene that basically renders thousands of points in a zoomable, pannable graph
I routinely create interactive scientific data scatterplots with thousands of individual QGraphicsItem instances. Zooming, panning, and other operations are nearly instantaneous.

I suspect that what you might be doing is destroying and recreating your scene each time it is zoomed or panned. This is wrong. You should create your scene once (or when there is a massive change in the underlying data) and use transformations on the view to change the region of the scene that is displayed at any one time. You do not have to do anything to the scene; simply set the sceneRect used by the view and the view will automatically update itself to show that part of the scene.

If you aren't changing the scene when zooming or panning, are you moving the scene items when panning or zooming? This is also wrong. Each move of a scene item is going to result in a repaint of the views where the item is displayed. If you are doing this for thousands of items, then you are repainting the view thousands of time for every zoom or pan operations. Again, if the scene items are fixed in world space, then all you have to do is change the region of that space shown in the view, by modifying the view's sceneRect.