QDomDocument Speed by 24MB file

[patrik08]

I have each week big xml file to read, QFile have 2 minute to open file.
setContent from QDomDocument have 30 sec. to parse this file.
On read xml QSqlDatabase play all attribute to db on 15 sec. ...

is the QXmlDefaultHandler faster? or How?

void PlaySincro::ValidateFile( QString xmlfile )
{
    QFileInfo fi(xmlfile);
    bool cvalid = true;
    QString re;  
    QFile file( xmlfile );
    const char *dat;
 
    std::cout << "### loading file wait... " << fi.fileName().toAscii().data() << " Size/" << BiteorMega(fi.size()).toAscii().data() << std::endl;
 
    if( file.open( QFile::ReadOnly | QFile::Text ) ) {
       QTextStream in( &file );
       int cur = 0;
 
       while( ! in.atEnd() ) {
            QString line = QString( in.readLine() );
            re.append(line);
            cur++;
 
             switch(cur)
                {
                case 1:
                dat ="-";
                break;
                case 2:
                dat ="\\";
                break;
                case 3:
                dat ="|";
                break;
                case 4:
                dat ="/";
                cur = 0;
                break;
              }
 
            std::cout <<  "Read file " <<  fi.fileName().toAscii().data() << " [" << dat << "]\r";
            fflush ( stdin );
 
       }
    }
 
    Qxml doc;  /* class Qxml : public QDomDocument */
    QString errorStr;
    int errorLine, errorColumn, position;
 
    if (!doc.setContent(re,false, &errorStr, &errorLine, &errorColumn)) {
    std::cout << "### Unable to read XML file! errorLine" << errorLine << " errorColumn" << errorColumn << std::endl;
    return;
    }
 
           QDomElement root = doc.root();
           QDomElement query = root.firstChildElement("query");
           QString user = doc.GetAtt(query,"user");
           QString summs = doc.GetAtt(query,"sumrow");
           QString cools = doc.GetAtt(query,"sumcool");
           QString data = doc.GetAtt(root,"build");
           QString org = doc.GetAtt(root,"org");
 
           if (data.size() < 1 && user.size() < 1 && summs.size() < 1 && cools.size() < 1 && org.size() < 1) {
            cvalid = false;
           }
 
           if (root.tagName() != "odbc_root" ) {
             cvalid = false;
           }
 
 
        if (cvalid) {
          std::cout << "###" << xmlfile.toAscii().data() << " valid Ok! From " << user.toAscii().data() << std::endl;
          PlayFileDom(root,org);
          doc.clear();   
        } else {
          std::cout << "###" << xmlfile.toAscii().data() << " not valid!" << std::endl; 
        }
}

void PlaySincro::ValidateFile( QString xmlfile )
{
    QFileInfo fi(xmlfile);
    bool cvalid = true;
    QString re;  
    QFile file( xmlfile );
    const char *dat;
    
    std::cout << "### loading file wait... " << fi.fileName().toAscii().data() << " Size/" << BiteorMega(fi.size()).toAscii().data() << std::endl;

    if( file.open( QFile::ReadOnly | QFile::Text ) ) {
       QTextStream in( &file );
       int cur = 0;
        
       while( ! in.atEnd() ) {
            QString line = QString( in.readLine() );
            re.append(line);
            cur++;
           
             switch(cur)
                {
                case 1:
                dat ="-";
                break;
                case 2:
                dat ="\\";
                break;
                case 3:
                dat ="|";
                break;
                case 4:
                dat ="/";
                cur = 0;
                break;
              }
              
            std::cout <<  "Read file " <<  fi.fileName().toAscii().data() << " [" << dat << "]\r";
            fflush ( stdin );
            
       }
    }
    
    Qxml doc;  /* class Qxml : public QDomDocument */
    QString errorStr;
    int errorLine, errorColumn, position;
    
    if (!doc.setContent(re,false, &errorStr, &errorLine, &errorColumn)) {
    std::cout << "### Unable to read XML file! errorLine" << errorLine << " errorColumn" << errorColumn << std::endl;
    return;
    }
  
           QDomElement root = doc.root();
           QDomElement query = root.firstChildElement("query");
           QString user = doc.GetAtt(query,"user");
           QString summs = doc.GetAtt(query,"sumrow");
           QString cools = doc.GetAtt(query,"sumcool");
           QString data = doc.GetAtt(root,"build");
           QString org = doc.GetAtt(root,"org");
           
           if (data.size() < 1 && user.size() < 1 && summs.size() < 1 && cools.size() < 1 && org.size() < 1) {
            cvalid = false;
           }
           
           if (root.tagName() != "odbc_root" ) {
             cvalid = false;
           }
           
           
        if (cvalid) {
          std::cout << "###" << xmlfile.toAscii().data() << " valid Ok! From " << user.toAscii().data() << std::endl;
          PlayFileDom(root,org);
          doc.clear();   
        } else {
          std::cout << "###" << xmlfile.toAscii().data() << " not valid!" << std::endl; 
        }
}

To copy to clipboard, switch view to plain text mode 

[marcel]

Why do you read line by line in a QString, when you can use:
bool setContent ( QIODevice * dev, QString * errorMsg = 0, int * errorLine = 0, int * errorColumn = 0 )?

QFile is a subclass of QIODevice so you can pass the file after you open in instead of the QTextStream. Should be considerably faster.

Regards

[patrik08]

Why do you read line by line in a QString, when you can use:
bool setContent ( QIODevice * dev, QString * errorMsg = 0, int * errorLine = 0, int * errorColumn = 0 )?

QFile is a subclass of QIODevice so you can pass the file after you open in instead of the QTextStream. Should be considerably faster.

Regards

Yes now the console stay frozen 40 sec. .... and before I see turn [-\|/-] .. chars .

Have you find a class to display Postcript file? or PDF? .. I put XPDF Libs on ice to Window...

http://poppler.freedesktop.org/ have a QT4 libs ... but i found bug on automake from mingw MSYS .... this gay make it only to build on automake autoconf & other tools... and not simply a *.pro file ...

[marcel]

Yes now the console stay frozen 40 sec. .... and before I see turn [-\|/-] .. chars

Well, it is a pretty big file... But still, is faster than you said it was before... I am not sure if you can make it load faster. Probably this is the best solution, because if you were to load it in an intermediate buffer, you would still loose some time with the loading part. Then, the QDomDocument would still have to parse your buffer, so you're back at square one...

To provide feedback to the user while loading large files, you could do the loading in a separate thread ( or use a QTimer and notify the user every 5 secs, let's say, that the file is still loading ). of course, this depends on how much need the user has to be notified .

Have you find a class to display Postcript file? or PDF? .. I put XPDF Libs on ice to Window...

No, unfortunately. I still plan to modify xpdf on my own and integrate it with qt, but this as soon as I find some spare time ( a lot of free time ).
I don't know of any other pdf library as complete as xpdf out there ( well, except the Adobe pdf Library ).

Regards