QwtPlotCurve draw it's full contents without using current zoom information

[Uwe]

The only thing that puts me off using this is the time needed to compute the pyramid of scales prior to displaying the data.

As you already have the optimization ( according to the ROI ) for the detailled zoom levels in place you can start with a larger tolerance value that should be very successful with weeding out points. How successful depends on the characteristics of your curve, but if it is not noise only you should have a result that is below 10% of the original size. For the following sets ( maybe 1 or 2 more ) you can use the result of the first run.

So there should be no memory problem and performancewise it boils down to the first run only. How long does it take to run the algo for a huge set on your system: more than a second ?

Note that you can split your set into parts and run the algorithm one by one reuniting the results later. This way you can avoid the need of a temporary copy and you can distribute the calculation to different threads. On multicore systems I would expect to have almost a linear effect for each core.

Are any of the performance improvements likely to benefit Windows?

First of all note, that when you have your implentation of the levels of detail I would expect that you are fine with or without optimizations.

Qwt 5.x had a integer based painting engine ( because of Qt3 compatibility ), Qt 6.0 runs on floats. Unfortunatly Qt 4.7 ( raster paint engine ) had a performance issue so that QPainter::drawPolylineF was 3 times slower than QPainter::drawPolyline - for the same values !

So I modified the code in SVN trunk to use integers for paint devices with integer based coordinate systems ( f.e on screen ) and floats for the others ( PDF, SVG ). Having integers ( or rounded floats since Qt 4.8 ) makes it possible to remove duplicates before passing them to Qt. F.e. when drawing a curve with 256,000,000 points to a widget with ~1000 pixels width almost all points will be duplicates.

Here it would also be possible to reduce the memory for the translated points heavily - but this is not done yet ( see qwtToPolylineFiltered in qwt_point_mapper.cpp ). But if you want to avoid a huge temporary buffer you could implement YourCurve::drawSeries calling QwtPlotCurve::drawSeries for intervals of f.e 10000 samples each: from -> from + 10000, from +10000 -> from + 20000 ...

Caching of symbols is another optimization - but this is more important for scatter plots than for your use case.

Using OpenGL is the only way to have hardware accelerated graphics on Windows, but this is more for oscilloscope alike use cases and as QwtPlotGLCanvas is in an experimental state ( f.e no caching yet ) I wouldn't recommend it to you. But you can have a try: setting an OpenGL canvas is one line of code only.

HTH,
Uwe

[mike_the_tv]

As you already have the optimization ( according to the ROI ) for the detailled zoom levels in place you can start with a larger tolerance value that should be very successful with weeding out points. How successful depends on the characteristics of your curve, but if it is not noise only you should have a result that is below 10% of the original size. For the following sets ( maybe 1 or 2 more ) you can use the result of the first run.

So there should be no memory problem and performancewise it boils down to the first run only. How long does it take to run the algo for a huge set on your system: more than a second ?

I tried an experiment with one channel run through the QwtWeedingCurveFitter and the other through the min/max weeding routine I have. With 1,000,000 samples the QwtWeedingCurveFitter with a tolerance of 50 (not sure if that's large or small) took about 20+ seconds to weed the 1,000,000 points to about 1557. The routine I used was less than a second and generated about 1600 points (2 x number of plot pixels). This was at full zoom out so no ROI optimisations at this stage. To be fair this was plotting two waveforms, the time domain, which had the weeders attached, and a frequency domain FFT, which had no weeders attached. However the FFT plot time was the same complexity for both time domain plots and from a visual guess was maybe 1-2 seconds to display.

I think the killer for the QwtWeedingCurveFitter routine is the heavy use of qSqrt(). Also from a visual perspective, because I had used what seemed to be a fairly high tolerance, the plot was not all that accurate to the general profile of the data, there were noticable gaps in the plot envelope where sample peaks had obvously been optimised out but where there was data. The data being plotted is simply a sine wave with a slight amount of modulated noise and phase variation to simulate the hardware variations.

Note that you can split your set into parts and run the algorithm one by one reuniting the results later. This way you can avoid the need of a temporary copy and you can distribute the calculation to different threads. On multicore systems I would expect to have almost a linear effect for each core.

Yeh had thought about doing that but given the speed test I tried, even distributing over the 4 cores in my Intel Core i5 would still not be fast enough for the number of points I need. I gave up timing 10,000,000 samples.

Here it would also be possible to reduce the memory for the translated points heavily - but this is not done yet ( see qwtToPolylineFiltered in qwt_point_mapper.cpp ). But if you want to avoid a huge temporary buffer you could implement YourCurve::drawSeries calling QwtPlotCurve::drawSeries for intervals of f.e 10000 samples each: from -> from + 10000, from +10000 -> from + 20000 ...

If you are overloading the drawSeries() could you also restrict the to/from at this level also to solve the ROI data point reduction?

Using OpenGL is the only way to have hardware accelerated graphics on Windows, but this is more for oscilloscope alike use cases and as QwtPlotGLCanvas is in an experimental state ( f.e no caching yet ) I wouldn't recommend it to you. But you can have a try: setting an OpenGL canvas is one line of code only.

What is the magic code loine for this and does it require the latest trunk code?


Thanks for the help.