Huge Text File

[mcosta]

Hi to all!

I have to show content of huge Text file (about 70 MB and 100000 lines).

I tried with QTextEdit and QTextEdit::setPlainText but it's very slow.
Also iterating the Text File and appending line like

...
QFile file(fileName);
file.open(QIODevice::ReadOnly | QIODevice::Text);
 
while(!file.atEnd())
{
    textEdit->append(file.readLine());
}
file.close();
...

...
QFile file(fileName);
file.open(QIODevice::ReadOnly | QIODevice::Text);

while(!file.atEnd())
{
    textEdit->append(file.readLine());
}
file.close();
...

To copy to clipboard, switch view to plain text mode 

The file contents are truncated in textEdit.

Help please

[stevey]

This is just a brain dump of an approach I'd consider.
Maybe there's another way of doing it.

70mb really isn't a huge file by today's standards, but if it does seem slow then I'd recommend implementing a memory buffering mechanism that only reads enough to fill the QTextEdit.

If your control is sized with 20 lines, then load 20 lines of text and display it.
You could buffer 20 before and 20 after as well so as to have something to display whilst loading the next segment.

One issue you'd need to deal with is the scrollbars. The bars by default will reflect upto 20 lines but it needs to think there's how ever many are actually in the file.
Also if you need to edit the file then this technique would get much more complicated. You could have a structure that tracks the changes then separately from the control. Consider that the change disappears from sight you need to keep the change and redisplay it if it comes into focus again.
It could also be very IO intensive so you'd be trading off the initial load time for another performance problem. Maybe increasing the buffer before and after to 2 or 3 times the size of the QTextEdit would help minimise IO.

To implement this, I'd subclass QTextEdit to something like QtBufferedIOTextEdit to make it a reusable technique.

[wysota]

If you use plain text, maybe you should try with QPlainTextEdit (it's probably available since 4.4) or with QTextBrowser?

[fullmetalcoder]

If you use plain text, maybe you should try with QPlainTextEdit (it's probably available since 4.4) or with QTextBrowser?

I've not try QPlainTextEdit, which may perform well but I can ensure you than QTextEdit performs really bad when it comes to loading big files (even 1.2Mb requires a couple of seconds for the text to be laid out) and QTextBrowser does even worse... Looks like the doc for Qt 4.4 is available online yet btw...

If you are looking for a faster alternatives you may have a look at QCodeEdit. It performs better than QTextEdit on average, in terms loading time and memory usage.QScintilla may also be worth a look but the API is very different which may make it harder to port your code...