QGraphicsScene/QGraphicsView performance problems

[jacek]

Unfortunately, now the app started gobbling up all memory even for a smaller file, so it got worse. I will have to figure out why...

Use valgrind. The Massif tool might be especially helpful.

[bnilsson]

Use valgrind. The Massif tool might be especially helpful.

For the moment I'm on MacOSX, so this is not applicable.

[bnilsson]

The behaviour was a bit better using Qt4.4.0 than Qt4.3.3, so maybe it's not in my code entirely. But still in the 1Gb range for a 40Mb file.
The behaviour was very strange indeed (excessive cpu and memory load when zooming IN) on my amd46 Debian using Qt4.3.3, and this particular problem was fixed by the Qt4.4.0 snapshot.

Anyway, standard usage of QGraphicsView/QGraphiscScene was not dramatically improved using Qt4.4.0.

[wysota]

Could you provide a minimal compilable example reproducing the problem so that we might try it ourselves?

[bnilsson]

Sure, the project is not very big, I will try to prepare something.
I am new to this forum, what are the provisions for uploading?

[wysota]

Use the attachments feature ("Manage attachments" button below the advanced editor).

[bnilsson]

What would be your primary platform?

I will have to arrange some way for you to pick up the datafiles, they will be too large for an attachment. I think I have some space at my ISP's web hotel, I will do this tomorrow.
What will be your primary project platform?

Here is the project. It is currently tested only for Mac, I have not built this particular project for Linux yet.

[wysota]

What would be your primary platform?

Mine is Linux/x86.

I will have to arrange some way for you to pick up the datafiles, they will be too large for an attachment.

Can't you just provide some small application that would generate such example datafile? They don't have to make sense and they don't have to be larger than a megabyte. It's just a test app.

Here is the project.

It'd be best if you provided something small. It's easier to debug then.

BTW. I just had a quick look at the code you provided and I have some comments:
- I'm not sure if manipulating the update mode the way you do is a good idea, you should set the update mode once and never touch it again
- I'm not sure if beginning by reimplementing the main drawing routine is a good idea. I'd start with the base class implementation first, it's probably trying to be smarter than your implementation
- why do you use the data stream if you only read raw data and seek through the file? Operate on the file directly instead
- try not to fall of the scale with your dimensions, 10nm = 10E-8, pretty close to 32bit precision.

[bnilsson]

Mine is Linux/x86.

Were you able to build the application?

Can't you just provide some small application that would generate such example datafile? They don't have to make sense and they don't have to be larger than a megabyte. It's just a test app.

The example datafile supplided, "testexp_c.j51" should enable you to thest the function, but maybe not show the problem. I don't have a J51 generator ready, but I can try to make one. It would not take long, I hope.

It'd be best if you provided something small. It's easier to debug then.

I can strip it down further if you want. Should I?

BTW. I just had a quick look at the code you provided and I have some comments:
- I'm not sure if manipulating the update mode the way you do is a good idea, you should set the update mode once and never touch it again

If I turn off update permantly it doesn't scroll. If I turn it on a large file will be drawn twice or more, taking maybe 10-30s extra before the user can zoom in to any region of interest.
Do you have any suggestion on how to handle this?

- I'm not sure if beginning by reimplementing the main drawing routine is a good idea. I'd start with the base class implementation first, it's probably trying to be smarter than your implementation

I dont' quite uderstand. Please expalin.
This is my very forst Qt application, I took the outline from a basic example (a text editor) described in the documentation. My plan was to clean it up later, and evenually make a multidocument application. But I got stuck before I got to that point.

- why do you use the data stream if you only read raw data and seek through the file? Operate on the file directly instead

Sure. In one of my later versions I make a linked list from the data analyzed.

- try not to fall of the scale with your dimensions, 10nm = 10E-8, pretty close to 32bit precision.

10-8 compared to 1 meter, yes. In the drawing unity means 1 micrometer, so the precision is no problem here.

[wysota]

Were you able to build the application?

I had no data so I didn't even try.

I can strip it down further if you want. Should I?

You could get rid of the loading routines and just generate polygons and rectangles in the application.

Do you have any suggestion on how to handle this?

Yes, find out why it draws twice.

I dont' quite uderstand. Please expalin.

Don't reimplement anything from the scene or the view. Just you the classes provided only changing their properties and adding items to the scene.

Edit:
The application works just fine with the provided data. It uses about 15MB of RAM which I assume is quite fine.

[bnilsson]

Here is a stripped down version with no loader and a limited set of redundant functions. I did not want to change the UI part so there are some empty action handling methods, please ignore them.

It draws an area set by "chipsize", divided into fields of size "fieldsize", whuch are in turn divided into subfields of size "subfieldsize", and in each subfield there is one rectangle and one polygon. Chip, field and subfield boundaries are drawn in blue color, while the rectangles and polygons are drawn in black.

Chipsize can range from 50,000 microns up to 125,000 microns, the field size is 800 microns, and subfield size is 100 microns. I kept this sceme to be able to relate to my real needs.

Here, 125,000 micron chipsize is beyond the capacity of any of my machines (1 Mac G5 1GB RAM, and 1 amd64 1GB RAM) while 50,000 micron chipsize can be managed with some patience.
Note that Qt4.3.3 has real problems when zooming in on the amd64, the Qt4.4.0 snapshot is much better. They really must have done something there.

Some logs:
Chipsize 50000.000 um, 3844 fields, 246016 subfields, 246016 rects, 246016 polys
Chipsize 100000.000 um, 15625 fields, 1000000 subfields, 1000000 rects, 1000000 polys
Chipsize 125000.000 um, 24336 fields, 1557504 subfields, 1557504 rects, 1557504 polys

[wysota]

Ok, I finally managed to take a look at the app. The example you gave us occupies about 200MB of memory, which is not that bad for 0.5M items ( ~400B per item). The application is horribly slow, but I see some ways to speed it up. I don't know if this will improve the speed significantly, but it's worth to try. I'm going to get rid of your polygon items first.

Edit: Ok, I'm not going to do that, because it'd be quite hard for me as I don't know your code. But I already know the difference should be huge. From what I understand you make each item very big - it's pos() is set to (0,0) and it's rect is defined by the polygon. As you have only four points, there is no point in creating a path from them - you should draw them manualy and return a very tiny bounding rectangle. Furthermore you should make use of the levelOfDetail while painting items - from a big distance you could paint polygons as rects or even just points. It should make a huge difference. And it's not a matter of memory, it's consumption is fine.

[bnilsson]

Don't reimplement anything from the scene or the view. Just you the classes provided only changing their properties and adding items to the scene.

Please define "reimplement".
I need to tell you I never took the course in C++ so sometimes I have problems with the programming lingo. I just look at examples and try to do the same, and then fill in my own stuff.

It would be great if you could explain with "more words".

[wysota]

Please define "reimplement".

"Subclass and rewrite a method which is defined in the base class and has the same signature (return value, name and arguments)".

Use QGraphicsView and QGraphicsScene instead of MyQGraphics... classes.

[pherthyl]

This may be useful to you.

http://labs.trolltech.com/blogs/2008...graphicsitems/