Results 1 to 13 of 13

Thread: QThread and PostgreSQL - a problem

  1. #1
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default QThread and PostgreSQL - a problem

    I'm building some QT application to convert DBF files to SQL database.
    I have a class derived from QThread to convert one DBF file containing information from one month. This class is defined as :
    Qt Code:
    1. class KonwertJedenMiesiac :
    2. public QThread
    3. {
    4. Q_OBJECT
    5. public:
    6. KonwertJedenMiesiac(QString katDBF,QString bazaDBF);
    7. ~KonwertJedenMiesiac(void);
    8. void KonwertJedenMiesiac::stopKonwerter( void );
    9. void zrobIncProgressBarOperacja( void );
    10. signals:
    11. void incProgressBarOperacja( void );
    12. protected:
    13. inline long nowyKod( char litera, long kod )
    14. { if(litera == ' ' )
    15. return 0;
    16. return(litera*1000000L + kod);
    17. }
    18. protected:
    19. bool run_konwerter;
    20. mutable QMutex mutex;
    21. Code4 cb;
    22. QDate d88;
    23. QString katDBF, bazaDBF;
    24. };
    To copy to clipboard, switch view to plain text mode 
    The constructor look s like :
    Qt Code:
    1. KonwertJedenMiesiac::KonwertJedenMiesiac(QString katDBF, QString bazaDBF )
    2. {
    3. run_konwerter = true;
    4. d88 = QDate(1988,01,01);
    5. this->bazaDBF = bazaDBF;
    6. this->katDBF = katDBF;
    7.  
    8. // wczytujemy parametry bazy danych
    9. QSettings settings( "dworzec.ini", QSettings::IniFormat);
    10. settings.beginGroup("DbSetup");
    11.  
    12. db = QSqlDatabase::addDatabase(settings.value("DbType").toString(),bazaDBF);
    13.  
    14. db.setHostName(settings.value("HostName").toString());
    15. db.setDatabaseName(settings.value("DatabaseName").toString());
    16. db.setUserName(settings.value("UserName").toString());
    17. db.setPassword(settings.value("Password").toString());
    18. }
    To copy to clipboard, switch view to plain text mode 

    SQL database is opened in method run().
    All monthly DBF files must be merged to one SQL table.
    When I create only one thread KonwerterJedenMiesiac all is working correctly. But when I create more than one thread from time to time INSERT rows generated in program are broken like this :
    Qt Code:
    1. QSqlError(-1, "", "")
    2. "INSERT INTO pozycje_paragonow (litera,kod,pozycja,id_towaru,nazwa_towaru,jm,ilosc,cena,kod_vat,stawka_vat,rabat1,rabat2,bonifikata,kwota_ulgi,sww) VALUES ('F',7429,1,65000031,'A
    To copy to clipboard, switch view to plain text mode 

    I don't observe this behavior when SQL database is Firebird.

    Is something special to do with PostgreSQL driver ?

    Qt 4.3.3 OpenSource on Windows XP compiled with Visual C++ 2005 Express Edition. PostgreSQL developer library v.8.2.

  2. #2
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    Make sure that each thread uses an unique connection name.

  3. #3
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    Quote Originally Posted by jacek View Post
    Make sure that each thread uses an unique connection name.
    200% yes. DBF file name is used as connection name and for each thread it is unique.

  4. #4
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    5,372
    Thanks
    28
    Thanked 976 Times in 912 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    How do you start those threads? Did you implement the run() method?

  5. #5
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    Quote Originally Posted by jacek View Post
    How do you start those threads? Did you implement the run() method?
    Yes, as I wrote this same code is working perfect with Firebird.

  6. #6
    Join Date
    Feb 2006
    Location
    Oslo, Norway
    Posts
    6,264
    Thanks
    36
    Thanked 1,519 Times in 1,389 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows Symbian S60 Maemo/MeeGo

    Default Re: QThread and PostgreSQL - a problem

    I don't see KonwertJedenMiesiac::run() being implemented anywhere. Furthermore, you're calling QSqlDatabase::addDatabase() in KonwertJedenMiesiac constructor which is NOT executed in the newly created thread. You might want to consult QThread docs for how to use QThread. There is an example in the detailed description.
    J-P Nurmi

  7. #7
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    Quote Originally Posted by jpn View Post
    I don't see KonwertJedenMiesiac::run() being implemented anywhere. Furthermore, you're calling QSqlDatabase::addDatabase() in KonwertJedenMiesiac constructor which is NOT executed in the newly created thread. You might want to consult QThread docs for how to use QThread. There is an example in the detailed description.
    KonwertJedenMiesiac is parent class for next class which have method run(). One of experiments was calling QSqlDatabase::addDatabase() in run() method but no difference.
    Here is a definition of class derived from KonwertJedenMiesiac
    Qt Code:
    1. class KonwertBJM :
    2. public KonwertJedenMiesiac
    3. {
    4. public:
    5. KonwertBJM(QString katDBF, QString bazaDBF);
    6. ~KonwertBJM(void);
    7. protected:
    8. void run(void);
    9. void robKonwersje( char *kat_dbf, char *miesiac );
    10. };
    To copy to clipboard, switch view to plain text mode 

    and implementation of method run()
    Qt Code:
    1. void KonwertBJM::run(void)
    2. {
    3. if( !db.open() )
    4. {
    5. qDebug() << "KonwerterBJM"
    6. << QObject::tr("Can't connect to database")
    7. << db.lastError().text();
    8. return;
    9. }
    10. char miesiac[7];
    11.  
    12. strcpy_s(miesiac,sizeof(miesiac),bazaDBF.mid(1,6).toLocal8Bit().data());
    13. robKonwersje(katDBF.toLocal8Bit().data(),miesiac);
    14.  
    15. db.close();
    16. }
    To copy to clipboard, switch view to plain text mode 

  8. #8
    Join Date
    Feb 2006
    Location
    Oslo, Norway
    Posts
    6,264
    Thanks
    36
    Thanked 1,519 Times in 1,389 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows Symbian S60 Maemo/MeeGo

    Default Re: QThread and PostgreSQL - a problem

    Even if this had nothing to do with the original problem, you are calling QSqlDatabase::addDatabase() in one thread and QSqlDatabase::open() in another thread. You should not do so when it's the same connection in question.

    Threads and the SQL Module:
    A connection can only be used from within the thread that created it. Moving connections between threads or creating queries from a different thread is not supported.
    J-P Nurmi

  9. #9
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    OK, all code related to creating connection to database is moved to run() method :
    Qt Code:
    1. void KonwertBJM::run(void)
    2. {
    3.  
    4. QSettings settings( "dworzec.ini", QSettings::IniFormat);
    5. settings.beginGroup("DbSetup");
    6. db_connection_name = UniqueDbName();
    7. db = QSqlDatabase::addDatabase(settings.value("DbType").toString(),db_connection_name);
    8.  
    9. db.setHostName(settings.value("HostName").toString());
    10. db.setDatabaseName(settings.value("DatabaseName").toString());
    11. db.setUserName(settings.value("UserName").toString());
    12. db.setPassword(settings.value("Password").toString());
    13. if( !db.open() )
    14. {
    15. qDebug() << "KonwerterBJM"
    16. << QObject::tr("Can't connect to database")
    17. << db.lastError().text();
    18. return;
    19. }
    20. char miesiac[7];
    21.  
    22. strcpy_s(miesiac,sizeof(miesiac),bazaDBF.mid(1,6 ).toLocal8Bit().data());
    23. robKonwersje(katDBF.toLocal8Bit().data(),miesiac);
    24.  
    25. db.close();
    26. }
    To copy to clipboard, switch view to plain text mode 
    No difference.

  10. #10
    Join Date
    Feb 2006
    Location
    Oslo, Norway
    Posts
    6,264
    Thanks
    36
    Thanked 1,519 Times in 1,389 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows Symbian S60 Maemo/MeeGo

    Default Re: QThread and PostgreSQL - a problem

    Quote Originally Posted by Lesiok View Post
    Qt Code:
    1. char miesiac[7];
    2. strcpy_s(miesiac,sizeof(miesiac),bazaDBF.mid(1,6).toLocal8Bit().data());
    To copy to clipboard, switch view to plain text mode 
    Doesn't this leave the last character uninitialized? What does KonwertBJM::robKonwersje() do? Why does it take char* and not QString in the first place?
    J-P Nurmi

  11. #11
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    Quote Originally Posted by jpn View Post
    Doesn't this leave the last character uninitialized? What does KonwertBJM::robKonwersje() do? Why does it take char* and not QString in the first place?
    1. No, strcpy_s writes to the new location string WITH ending 0x00 byte.

    2. KonwertBJM::robKonwersje() is a main method to doing conversion from DBF to SQL. It opens 3 DBF files named as Byyyymm.DBF, KByyyymm.DBF and Pyyyymm.DBF, where yyyymm is saved in var miesiac, and record by record convert it to SQL INSERT statements. A few records from KB and P is related to one record in B. So every record from B produce some INSERT statements in one transaction.

    As I wrote in post opening this thread, on Firebird I can do this in many threads and all is OK. Problem is only with PostgreSQL.

  12. #12
    Join Date
    Jan 2008
    Posts
    40
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: QThread and PostgreSQL - a problem

    In addition, the third party libraries used by the QSqlDrivers can impose further restrictions on using the SQL Module in a multithreaded program. Consult the manual of your database client for more information
    This may be the problem

  13. #13
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QThread and PostgreSQL - a problem

    Quote Originally Posted by C167 View Post
    This may be the problem
    Yes I know.
    Theoretically libpq is compiled in thread-safe mode. PQisthreadsafe() returns 1 (the libpq is thread-safe).

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.