QGraphicsScene/QGraphicsView performance problems

[bnilsson]

What would be your primary platform?

I will have to arrange some way for you to pick up the datafiles, they will be too large for an attachment. I think I have some space at my ISP's web hotel, I will do this tomorrow.
What will be your primary project platform?

Here is the project. It is currently tested only for Mac, I have not built this particular project for Linux yet.

[wysota]

What would be your primary platform?

Mine is Linux/x86.

I will have to arrange some way for you to pick up the datafiles, they will be too large for an attachment.

Can't you just provide some small application that would generate such example datafile? They don't have to make sense and they don't have to be larger than a megabyte. It's just a test app.

Here is the project.

It'd be best if you provided something small. It's easier to debug then.

BTW. I just had a quick look at the code you provided and I have some comments:
- I'm not sure if manipulating the update mode the way you do is a good idea, you should set the update mode once and never touch it again
- I'm not sure if beginning by reimplementing the main drawing routine is a good idea. I'd start with the base class implementation first, it's probably trying to be smarter than your implementation
- why do you use the data stream if you only read raw data and seek through the file? Operate on the file directly instead
- try not to fall of the scale with your dimensions, 10nm = 10E-8, pretty close to 32bit precision.

[bnilsson]

Mine is Linux/x86.

Were you able to build the application?

Can't you just provide some small application that would generate such example datafile? They don't have to make sense and they don't have to be larger than a megabyte. It's just a test app.

The example datafile supplided, "testexp_c.j51" should enable you to thest the function, but maybe not show the problem. I don't have a J51 generator ready, but I can try to make one. It would not take long, I hope.

It'd be best if you provided something small. It's easier to debug then.

I can strip it down further if you want. Should I?

BTW. I just had a quick look at the code you provided and I have some comments:
- I'm not sure if manipulating the update mode the way you do is a good idea, you should set the update mode once and never touch it again

If I turn off update permantly it doesn't scroll. If I turn it on a large file will be drawn twice or more, taking maybe 10-30s extra before the user can zoom in to any region of interest.
Do you have any suggestion on how to handle this?

- I'm not sure if beginning by reimplementing the main drawing routine is a good idea. I'd start with the base class implementation first, it's probably trying to be smarter than your implementation

I dont' quite uderstand. Please expalin.
This is my very forst Qt application, I took the outline from a basic example (a text editor) described in the documentation. My plan was to clean it up later, and evenually make a multidocument application. But I got stuck before I got to that point.

- why do you use the data stream if you only read raw data and seek through the file? Operate on the file directly instead

Sure. In one of my later versions I make a linked list from the data analyzed.

- try not to fall of the scale with your dimensions, 10nm = 10E-8, pretty close to 32bit precision.

10-8 compared to 1 meter, yes. In the drawing unity means 1 micrometer, so the precision is no problem here.

[wysota]

Were you able to build the application?

I had no data so I didn't even try.

I can strip it down further if you want. Should I?

You could get rid of the loading routines and just generate polygons and rectangles in the application.

Do you have any suggestion on how to handle this?

Yes, find out why it draws twice.

I dont' quite uderstand. Please expalin.

Don't reimplement anything from the scene or the view. Just you the classes provided only changing their properties and adding items to the scene.

Edit:
The application works just fine with the provided data. It uses about 15MB of RAM which I assume is quite fine.

[bnilsson]

Here is a stripped down version with no loader and a limited set of redundant functions. I did not want to change the UI part so there are some empty action handling methods, please ignore them.

It draws an area set by "chipsize", divided into fields of size "fieldsize", whuch are in turn divided into subfields of size "subfieldsize", and in each subfield there is one rectangle and one polygon. Chip, field and subfield boundaries are drawn in blue color, while the rectangles and polygons are drawn in black.

Chipsize can range from 50,000 microns up to 125,000 microns, the field size is 800 microns, and subfield size is 100 microns. I kept this sceme to be able to relate to my real needs.

Here, 125,000 micron chipsize is beyond the capacity of any of my machines (1 Mac G5 1GB RAM, and 1 amd64 1GB RAM) while 50,000 micron chipsize can be managed with some patience.
Note that Qt4.3.3 has real problems when zooming in on the amd64, the Qt4.4.0 snapshot is much better. They really must have done something there.

Some logs:
Chipsize 50000.000 um, 3844 fields, 246016 subfields, 246016 rects, 246016 polys
Chipsize 100000.000 um, 15625 fields, 1000000 subfields, 1000000 rects, 1000000 polys
Chipsize 125000.000 um, 24336 fields, 1557504 subfields, 1557504 rects, 1557504 polys

[bnilsson]

Don't reimplement anything from the scene or the view. Just you the classes provided only changing their properties and adding items to the scene.

Please define "reimplement".
I need to tell you I never took the course in C++ so sometimes I have problems with the programming lingo. I just look at examples and try to do the same, and then fill in my own stuff.

It would be great if you could explain with "more words".

[wysota]

Please define "reimplement".

"Subclass and rewrite a method which is defined in the base class and has the same signature (return value, name and arguments)".

Use QGraphicsView and QGraphicsScene instead of MyQGraphics... classes.

[bnilsson]

Ok, I think I understand.
But this removes the possibility to add new methods for the class, right?
Now I have a myQGraphicsView member pointing at the coordinate display in the main window, and I give it a value by the method setCoordDisplay(QLineEdit *in);
Please suggest how I should transfer the result form QGraphicsView::mouseMoveEvent to the coordinate display field in the main window using some alternate method.

If new methods can be added, please advice me on how the header should be written.

[wysota]

But this removes the possibility to add new methods for the class, right?

You can add new methods, just don't reimplement the ones that already exist and do something - namely drawItems().

[bnilsson]

So you mean that the problem was NOT the new class myQGraphicsView, but the fact that drawItems was re-implemented?
If so, what about mouse*Event methods?

[wysota]

I didn't say anything like that. I said you shouldn't start programming by reimplementing methods that do something useful. The base class implementation is probably sufficient for you.

[bnilsson]

I am sorry, but I am getting confused. Let's take this very slowly.

Use QGraphicsView and QGraphicsScene instead of MyQGraphics... classes.

You suggest that I should not create a derived class, but instead override the existing methods when needed, such as mouseMoveEvent, mousePressEvent and mouseReleaseEvent, and not override drawItems.

Did I understand this correctly?

I appreciate your patience.

[bnilsson]

Let's close the discussion about "reimplementation", after reading all of your comments again I now understand your point. Thanks for your patience.

[wysota]

If you want to override anything, then you need to subclass. The rest is correct.

[pherthyl]

This may be useful to you.

http://labs.trolltech.com/blogs/2008...graphicsitems/

[wysota]

Ok, I finally managed to take a look at the app. The example you gave us occupies about 200MB of memory, which is not that bad for 0.5M items ( ~400B per item). The application is horribly slow, but I see some ways to speed it up. I don't know if this will improve the speed significantly, but it's worth to try. I'm going to get rid of your polygon items first.

Edit: Ok, I'm not going to do that, because it'd be quite hard for me as I don't know your code. But I already know the difference should be huge. From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon. As you have only four points, there is no point in creating a path from them - you should draw them manualy and return a very tiny bounding rectangle. Furthermore you should make use of the levelOfDetail while painting items - from a big distance you could paint polygons as rects or even just points. It should make a huge difference. And it's not a matter of memory, it's consumption is fine.

[Gopala Krishna]

Ok, I finally managed to take a look at the app. The example you gave us occupies about 200MB of memory, which is not that bad for 0.5M items ( ~400B per item). The application is horribly slow, but I see some ways to speed it up. I don't know if this will improve the speed significantly, but it's worth to try. I'm going to get rid of your polygon items first.

Edit: Ok, I'm not going to do that, because it'd be quite hard for me as I don't know your code. But I already know the difference should be huge. From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon. As you have only four points, there is no point in creating a path from them - you should draw them manualy and return a very tiny bounding rectangle. Furthermore you should make use of the levelOfDetail while painting items - from a big distance you could paint polygons as rects or even just points. It should make a huge difference. And it's not a matter of memory, it's consumption is fine.

Very true.. Level of detail really makes lot of difference.. The chip demo which qt provides is really a nice demo on how graphicsview applications can be optimized atleast to some extent.

Actually your app didn't run on my system(1Gb ram, 3GHz, pentium 4 HT) until i changed chip size to 5k.
But to be frank enough, I personally feel gv framework is overkill. Infact your app didn't run(50k) even when i turned of all possible bells and whistles. Also you don't need those unnecessary memory strain as well.
If you are sure that you don't need to select or move items here and there, definitely implementing a widget is best option. Remember you always don't have to draw everything as drawing everything at < .25 scale is hardly visible.

[bnilsson]

First, I want to thank you for your engagement in this. This is my VERY first Qt project, and surely must have missed and misunderstood a lot of things.

Ok, I'm not going to do that, because it'd be quite hard for me as I don't know your code.

You have all of my relevant code in the example. What is missing?

But I already know the difference should be huge. From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon.

Here it seems I have missed something really important. I was unaware of pos().

Furthermore you should make use of the levelOfDetail while painting items - from a big distance you could paint polygons as rects or even just points.

Again. I was unaware of levelofDetail, I will study it. However, the description in Assistant is a bit cryptic, so I will have to test different settings. Will a higher value improve?

Gopala:
But to be frank enough, I personally feel gv framework is overkill.

I tend to have the same opinion. But the gv framework is so convenient! But on the other hand, it is the only thing I have looked at so far. Is it as easy to implement scroll and zoom for a regular widget?

pherthyl:
This may be useful to you.

http://labs.trolltech.com/blogs/2008...graphicsitems/

I definitely see this on Linux, but for some reason not on my MacOSX. On Linux, displaying the full chip took time, but was ok finally. Zooming in locked it up completely. On Linux, this was eliminated using Qt4.4.0. Or is this a note of a further improvement?

[bnilsson]

First:

From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon.

This statement puzzles me.
I just do

QVector<QPointF> Tpz(4);
<fill in the 4 points>
QPolygonF Trapezoid(Tpz);
scene->addPolygon(Trapezoid,AreaShotPen);

QVector<QPointF> Tpz(4);
<fill in the 4 points>
QPolygonF Trapezoid(Tpz);
scene->addPolygon(Trapezoid,AreaShotPen);

To copy to clipboard, switch view to plain text mode 

Where does the pos() come in?

Secondly, can you apply levelofDetail to a high-level draw such as scene->addRect();?
Or do you have to create a custom QGraphicsItem and apply it to painter->drawRect() to do that?

[Gopala Krishna]

Secondly, can you apply levelofDetail to a high-level draw such as scene->addRect();?
Or do you have to create a custom QGraphicsItem and apply it to painter->drawRect() to do that?

You have to create a custom item. Have a look at the code for demos/chip/chip.cpp for reference.