Text file parsing too slow

[Potch]

Hello,

I need to parse a tab delimited (or csv) file and display it in a QTableWidget.
So far, I do it this way :

while (!MyStream.atEnd())
	{
		General_Interface.MainTable->insertRow(Max_Number_of_Lines+1);
		buffer_line = MyStream.readLine();
		buffer_list = buffer_line.split(Delimiter);
 
		for (int column = 0; column <= Max_Number_of_Columns; column ++)
		{
			QTableWidgetItem *Cell_Content = new QTableWidgetItem();
			Cell_Content->setText(buffer_list[column]);
			General_Interface.MainTable->setItem(Max_Number_of_Lines, column,
					Cell_Content);
 
		}
 
		Max_Number_of_Lines++;
	}

while (!MyStream.atEnd())
	{
		General_Interface.MainTable->insertRow(Max_Number_of_Lines+1);
		buffer_line = MyStream.readLine();
		buffer_list = buffer_line.split(Delimiter);

		for (int column = 0; column <= Max_Number_of_Columns; column ++)
		{
			QTableWidgetItem *Cell_Content = new QTableWidgetItem();
			Cell_Content->setText(buffer_list[column]);
			General_Interface.MainTable->setItem(Max_Number_of_Lines, column,
					Cell_Content);

		}

		Max_Number_of_Lines++;
	}

To copy to clipboard, switch view to plain text mode 

The problem is that the kind of file I have to parse can contains more than 30 columns and 5000 lines, and in these cases it's awfully slow.

How can I optimize this ?
Is it relevant to use a QTextStream as I did ?

Thanks in advance...

[wysota]

I wouldn't use a text stream in this situation - it only slows everything down without giving any benefits. You can operate on QFile directly.

The problem might be using QTableWidget instead of QTableView. I'd suggest switching to model based approach - this should make things much faster, especially if you first fill the whole model and then set it on the view and not the other way round.

[Potch]

Thanks for your fast response Wysota.

By

You can operate on QFile directly.

do you mean by using QByteArray ?

Concerning the model/view approach, I will rethink my program in this way. For it is the first time I use Qt, item-based approach seemed more friendly to me...
But it seems that the easy way is not always the best !

[wysota]

do you mean by using QByteArray ?

No, I mean using QIODevice::readLine(). QFile is a QIODevice.

[Potch]

Thanks for your response. I will go in this direction....

[Potch]

Hi all,

As advised, I finally used QIODevice::readLine() with a model / view approach, and it works very fast. I easily load a 3 or 4 Mo text file quasi instantaneaous.

For those that it can help :

InputDataModel.setRowCount(0) ;
InputDataModel.setColumnCount(Max_Number_of_Columns + 3) ;
 
	while (InputDataFile.canReadLine())
	{
		if (Max_Number_of_Lines == 0)
		{
			buffer_list = buf.split(Delimiter) ;
		}
 
		else
		{
			QByteArray line = InputDataFile.readLine();
			QString buffer_line(line);
			buffer_list = buffer_line.split(Delimiter);
		}
 
		for (int column = 0; column <= Max_Number_of_Columns; column ++)
		{
			QStandardItem *item = new QStandardItem(buffer_list[column]);
			InputDataModel.setItem(Max_Number_of_Lines, column, item) ;
		}
	}
 
	MytableView->setModel(&InputDataModel);
	MytableView->show() ;

InputDataModel.setRowCount(0) ;
InputDataModel.setColumnCount(Max_Number_of_Columns + 3) ;

	while (InputDataFile.canReadLine())
	{
		if (Max_Number_of_Lines == 0)
		{
			buffer_list = buf.split(Delimiter) ;
		}

		else
		{
			QByteArray line = InputDataFile.readLine();
			QString buffer_line(line);
			buffer_list = buffer_line.split(Delimiter);
		}

		for (int column = 0; column <= Max_Number_of_Columns; column ++)
		{
			QStandardItem *item = new QStandardItem(buffer_list[column]);
			InputDataModel.setItem(Max_Number_of_Lines, column, item) ;
		}
	}
	
	MytableView->setModel(&InputDataModel);
	MytableView->show() ;

To copy to clipboard, switch view to plain text mode 

Thanks again for your clever advices.....