QVector Explosion

[baray98]

Hi,

I have a compressed data file as input to my app with the following characteristics

filesize = 100 Meg
composition = divided into frames ,each frame has all the info on decompressing,frame size .etc...
no of frames in file is around 5000 frames

frame to real datasize ratio is about 1:7 hence real data size for a data file is 700 Meg

My solution 1:
My app divided the files into frames an decompress, save decompressed data into QVector as i plot the data , and cleared the vector (uncompressed data container) after getting what i need, but this is really slowing my plot response. my app did not crush ever

My solution 2:
My app divided the files into frames an decompresses it , save decompressed data into QVector as i plot the data , did not clear the vector (uncompressed data container) after getting what i need, My purpose to this is that the next time i will come into this frame, it wont decompress the raw data again since my uncompress data values is still valid.

Now, My solution 2 was faster in plotting but after so many frames (about 1000) my apps hung it seems that the PC runs out of memory , my guess is that Qvector (uncompressed data container) can not allocate memory anymore. It exploded thats what i call it ....

Can someone point me out a better approach into this BIG data that i need to process....

baray98

[wysota]

But what is your goal? Do you need to access all of the data at the same time? Can't you read and process one "frame" at a time?