Speed of Qwt plotting slows with many points

[Momergil]

Hello!

I'm having a problem concerning the regularity of the speed in which points in a Qwt graph are being plotted, i.e. after some points, the speed decreases.

My software contains a QTimer that is connecte through timeout() to a slot that reads the Y value to be plot in a QVector::front() and uses QwtPlotCurve::setData() and QwtPlot::replot() to plot it each timeout(). After reading the QVector, the first position is deleted.

The problem is that the QVector is huge - at the end something like 90 000 Y-coordinates should be in it - and therefore the Qwt should plot a graf with 90 000 points, and this seems to be causing a delay in the speed of the plotting, so when it starts, the speed is OK and the QTimer seems to be firing in the correct speed, but when the size of the Qwt graph begins to grow significantly (that is over 13 000 - 15 000 points plotted, but more significantly after 18 000 - 20 000 points), the speed begins to slow a little bit and gradually, so when the 90 000 point arrives, the plotting speed is less than half of what is programmed.

I and my friends imagine that this is maybe due to the internal buffer that Qwt possess that even allow me to stop the plotting and verify what has being plot - and I simply don't know if there is a function that allows to erase specific points in a QwtPlot; the only thing I know is QwtPlot::clear(), that doesn't work for this -. I think that the big size of this internal buffer is what is making the plotting "more difficult", i.e. more slow.

So I have some questions related to this problem:

1) Is truly the internal buffer what is creating the "slow" problem? yes or no, how could I solve this and force the plotting to occur always in the same speed - using the same functions that I use today?
2) Is there any other way to plot the points in the graph without using the QwtPlotCurve::setData() function but that still produces the same type of graph? I imagine that if there is a function that can plot point by point instead of creating a line between points some of my problems would be solved; I could just erase the previous point after a pre-determined number of points have being plotted. That would disable the possibility to check back what was plotted, but at least would discard the buffer big size - if that is what is truly making this problem appears.


Thanks very much,

Momergil.

[Spitfire]

You didn't say which Qwt you're using and that's quite imrpotant.

On Windows I've noticed significant difference between Qwt 6.x.x and 5.x.x where latter is faster.
It was discussed on the forum.

In the end I've managed to plot 5 milion points in less than 150ms.
You can't rely on Qwt to do all the optimizations.
In my case I'm 'compressing' my points to fit width of the screen and draw few hunderd lines instead of milions of points.
No loss in details at all.

Depending on your data you'll have to do something similar, otherwise the speed will degrade as Qwt draws every point every time you redraw, and you can't add/remove points without redrawing the whole thing.

There's few other things you can try to improve speed:
- polygon clipping,
- use Qwt 5.x.x if you're using windows
and few other things I can't remember at the moment :/

[Momergil]

Hello Spitfire,

thanks for the reply, but I have the impression that your case don't apply to mine.

First, I'm using Linux Ubuntu 10.X.X, and probably Qwt 5 (the guy who downloaded it for me in my pc downloaded the package that was avaliable at that time, and he sad it wasn't the Qwt 6.X.X).

Second, my situation is far different than yours. I have the impression that I read the post in which you (or someone else) discussed the problem you talked about, but my situation is far different. I don't pretend to display a large quantity of points at one time, but to add point by point (while those points are only a int variable for the Y-axis plus a int constantly-increased variable for the X-axis), while using replot() for update the graph. The unique situation in which your point would be valid, I imagine, is that if Qwt interprets each new replot() as if it was a complete different new image, in which case it would, after all, led to time difficulties as you mentioned, for that would mean that the graph when it comes about 90 000 points Qwt would understand it as if a was plotting a complete different image as that one with 30 000 points, not with just the difference that it has some extra points.

But even so 90 000 points don't compare to your 5 million! Only if all the black empty space of the background is counted by qwt as being points as well. If that is the case, that its quite obvious than when it comes about 90 000 points plotted actually I got several millions of them, and that would perfectly explain the speed the delay.

So, in case that's it, how do you did the compress thing?

Thanks!

Momergil


Note: in terms of speed, each clock timeout() is supposed to occur in 6.7 msec.

---

Added after 59 minutes:

Hmmm, it seems that my problem is half solved.

I managed to erase the first point in the vectors with the points to be ploted when the vector reaches the size of 10000 points, so now each time the replot() function is called, it has to plot at maximum 10 000 points, maintaining the plot speed. Of course, this came with the sacrifice of the "hitoric buffer"; if I want to consult what was plot, I can only go back till - 10 000 points; before that, the graph is clean.

Now its only a matter of putting part of the cpuplot qwt example and it will work fine.

Any case, doubt solved.

Thanks,

Momergil

[Uwe]

... and I simply don't know if there is a function that allows to erase specific points in a QwtPlot;

Just to have it clear for others reading the archive: you can't erase parts of a widget, all you can do is to repaint them. This is by design of Qt and has nothing to with Qwt. ( With Qt3 you were able to draw/erase with the XOR mode - but not with Qt4 anymore ).

What you can do with Qwt is to paint new points/lines on top. But to make use of this you might have to design your plot widget like in the oscilloscope example. For a curve, where the points are shifted along an axis ( like in the cpuplot example ) you can't use this.

Before complaining about performance issues always check the refreshtest example to see what is possible on your system. If you have high frame rates in this example ask yourself, what makes the difference to your application.

Uwe

[Spitfire]

Hmmm If i understood you correctly, you have a vector with lets say 50k points, but you want to plot only last 2k (or some other small part) and possibly scroll to any other part of the vector data, Similar to oscilloscope example but with keeping data off the screen, correct?

[jetsu]

You didn't say which Qwt you're using and that's quite imrpotant.

On Windows I've noticed significant difference between Qwt 6.x.x and 5.x.x where latter is faster.
It was discussed on the forum.

In the end I've managed to plot 5 milion points in less than 150ms.
You can't rely on Qwt to do all the optimizations.
In my case I'm 'compressing' my points to fit width of the screen and draw few hunderd lines instead of milions of points.
No loss in details at all.

Depending on your data you'll have to do something similar, otherwise the speed will degrade as Qwt draws every point every time you redraw, and you can't add/remove points without redrawing the whole thing.

There's few other things you can try to improve speed:
- polygon clipping,
- use Qwt 5.x.x if you're using windows
and few other things I can't remember at the moment :/

How could you compress millions of points to few hundred lines?
could you tell me the data processing method?

[Spitfire]

It's quite simple.

Imagine you have 1M points and 1K pixels wide canvas (for sake of simplicity).
You can't draw all 1M points, many of them will overlap in the same pixel.

Divide amount of points you have by the width of the canvas (in pixels).
In example above you'll end up with 1K points per canvas pixel (width-wise).
All this 1K points can be represented using only two points as a single vertical line resulting in 98% compression.
Just go through the points and record lowest and highest Y value, then create a line using those two values and X of the canvas pixel and you're done.
Repeat it 1K times and you'll end up with 98% less data to draw.

You have to do that every time data or canvas size changes. Also when you zoom in, you effectivley change the data available to draw so you have to recalculate.

[Uwe]

The easiest way is to use QwtWeedingCurveFitter. It has to be used according to scale changes ( don't use QwtPlotCurve::setCurveFitter ). Depending on the tolerance parameter you can control the algorithm.

Of course such an algorithm is for line plots and not very effective for scatter plots.

Uwe

PS: note that using Qwt from SVN trunk reintroduces several optimizations, that have been removed in Qwt 6.0. They should have a notable impact on the performance of a line plot with many, many points.

[yashkumar]

In my case it is not changing width of canvas.

I check plot->canvas()->width(), but i am getting constant value.

which parameter can i use for changing chunk size with respect to Zoom in and Zoom out?

It's quite simple.

Imagine you have 1M points and 1K pixels wide canvas (for sake of simplicity).
You can't draw all 1M points, many of them will overlap in the same pixel.

Divide amount of points you have by the width of the canvas (in pixels).
In example above you'll end up with 1K points per canvas pixel (width-wise).
All this 1K points can be represented using only two points as a single vertical line resulting in 98% compression.
Just go through the points and record lowest and highest Y value, then create a line using those two values and X of the canvas pixel and you're done.
Repeat it 1K times and you'll end up with 98% less data to draw.

You have to do that every time data or canvas size changes. Also when you zoom in, you effectivley change the data available to draw so you have to recalculate.

In my case it is not changing width of canvas.

I check plot->canvas()->width(), but i am getting constant value.

which parameter can i use for changing chunk size with respect to Zoom in and Zoom out?