How to achieve fastest database insertion

[Lesiok]

Maybe a solution is to write the samples to a regular file and upload them to the database in the other thread. In this way, You are immune to changes in the instantaneous performance of the server. For example, one minute records are going to the first file. After a minute, the records are going to a new file and the previous one is pushed to the database. And so on...

[ChrisW67]

You are receiving 12000 records in 1.0 second.
You are inserting 12000 records in 0.8 seconds.
Seems to me to me are keeping up just fine although I do wonder why your figure is 3 times longer than mine (Especially given your super-duper MySql backend vs my general MyISAM tables).

Here are some more questions to answer for your own benefit:
Where is the bottleneck? You still have not identified that. If you marshal the data into the query without actually executing it: how long does it take? What portion of the total time is this?
Does this data need to be stored in an RDBMS at all?
Is text the most efficient method of storing a date or time?
Can data be aggregrated on-the-fly and only the aggregrate data stored?
Can Lesiok's approach work for your application?

[karankumar1609]

Hello ChrisW67

I have implemented the same code as yours and its the total time of insertion of record into the table.

Where is the bottleneck? You still have not identified that. If you marshal the data into the query without actually executing it: how long does it take? What portion of the total time is this?

Problem is why my code takes ~0.8 second for insertion.
Before execution of query it hardly takes ~0-1 millisecond.
~0.8 second is the total time

QTime time;
    time.start();
 
    QSqlQuery query;
    database_manager_.transaction();
    for(int i = 0; i < queue.size(); ++i) {
        ...
        ...
        query.exec();
    }
    database_manager_.commit();
 
    qDebug() << ">> " << time.elapsed();

QTime time;
    time.start();

    QSqlQuery query;
    database_manager_.transaction();
    for(int i = 0; i < queue.size(); ++i) {
        ...
        ...
        query.exec();
    }
    database_manager_.commit();

    qDebug() << ">> " << time.elapsed();

To copy to clipboard, switch view to plain text mode 

Does this data need to be stored in an RDBMS at all?

Yes data need to be stored in RDBMS

Is text the most efficient method of storing a date or time?

I will change these variables from string to date, time. It dosn't matter to me.

Can data be aggregrated on-the-fly and only the aggregrate data stored?

Data will be aggregated on-the-fly

Can Lesiok's approach work for your application?

Yes it works, but execution time is little high, Should i need any compiler optimization?

[Lesiok]

Yes it works, but execution time is little high, Should i need any compiler optimization?

What does it mean "execution time is little high" ? Maybe your computer is too weak for this job or misconfigured ?

[karankumar1609]

Thanks for your help i have changed the way i am inserting data into table.
Now i have to insert 300 records per second which is quite less as compared to 12000.