Writing numbers to file fast! (Need Faster float to char implementation)

[d_stranz]

With these performance statistics on 30 seconds worth of data (1.2 s), it looks like you might be able to write your data in real time, as opposed to storing 30 seconds worth and then writing. Also would give you some protection against data loss if something fails - can you afford to lose 30 seconds worth of data? Writing in real time, you lose at most a second or so.

Not that you should think in units of seconds in the first place - if you are sampling a continuous data stream at 200kS/s, then -the smallest- time interval at which you can write in real time without data loss would be better than buffering for some human-defined period like 1, 5, or 30 seconds before writing.

This is where you might make use of a memory-mapped file. If you know how long you will be acquiring data in total (say 10 minutes), then you allocate the memory-mapped file to a size of 200000 * 600 * sizeof ( double ) and simply acquire to that "array" as if it were the double * array you allocate in your example code. The file system takes care of flushing it to disk at appropriate times (or you can do it manually every second or whatever if you don't trust the file system to do it for you).

With memory-mapping, you can completely ignore all the details of file I/O except those involved in originally opening and then closing the memory-mapped file. Everything else just looks like filling up an array.

David

[drhex]

I notice that you tried earlier to multiply your aquired values with 1000000 to get an integer. How much precision does your data have? Using float rather than integer might be sufficient, letting you store twice as many values or write them faster.

[philwinder]

Hi guys,
I have been using the previous routine to save data now for quite a few months, and everything seems to be fine in Real time.

@d_stranz: Ok, but the reason I had to move to a file anyway was because my system could only handle a minutes worth of data in memory. Since there are 4 channels, sampling at 200kS/s = 800kS/s * 32 bit for float ~= 24.4mB/s. One minute = 1.5 Gig. A lot of data.

Also, I cannot find a qt related class that deals with it, so would I have to write my own?

@drhex: Thats was just because I thought that saving an int would be smaller/quicker. I now know that was wrong. The final piece of code is what I have been using. (Or something similar)