Big Problem search through huge amount of data ca 30 000 strings takes to long

[Ini]

Yes I did a search that shows me only items or so. With only 1 character in the searchlines i have thousands of items and that takes minutes.
If I remove processevents I have an unresponsive GUI, makes no sense.

I can make a small Project that illustrates the problem, maybe then its much clearer. I see at some comments that its not very clear what it does. I do that in a few hours.

[Lesiok]

Show code for listWidget_addItemPlusTooltip() method.

[anda_skoa]

I don't see the point of the timer, but nvm here is the data:

It is to verify that the filtering itself is not the problem.
As the data nicely demonstrates it is always done in less than 100ms.

Its as expected first loop always constant and second loop depends on what i typed in. If I type in only 1 character second loop takes like 3 minutes or so

That's because then you have thousands of items that match.
The more matches, the longer it takes to create all these list widget items.

I dont understand why should nobody see them.

I didn't know you had a magical screen with nearly unlimited screen space.
Usually a list only shows tens of items.

But even if your list widget really shows all 30000 items at the same time, creating list widget items is still more wasteful than using a model.

Thank you for all help, but the hate stuff is not very helpfull

It is my only explanation why you would like to continue to suffer.
If you are not trying to punish yourself are you Trying to win a bet?

And thats true what you say it would be faster to store the previous data. But that does not solve the problem.

Hence this being an additional, secondary, improvement suggestion.

Cheers,
_

[Ini]

"I didn't know you had a magical screen with nearly unlimited screen space.
Usually a list only shows tens of items.

But even if your list widget really shows all 30000 items at the same time, creating list widget items is still more wasteful than using a model."

The Listwidget has a Scrollbar.
Why is a Model faster?

"Show code for listWidget_addItemPlusTooltip() method. "

inline void listWidget_addItemPlusTooltip(QListWidget* paramListWidget,QString paramText)
{
paramListWidget->addItem(paramText);
paramListWidget->item(paramListWidget->count()-1)->setToolTip(paramText);
}

[anda_skoa]

The Listwidget has a Scrollbar.

If it has a scrollbar then this usually means not all items are visible.
Do you have a list that can show all 30000 items without scrolling or do you have a limit on how many items can be shown at any given time?

Why is a Model faster?

Even in you case with the giant screen that can show all 30000 items at the same time, it wouldn't have to create QListViewItem instances.
In a normal case, when the screen size does not allow to show 30000 list items at the same time, it becomes even better, since the view only needs to deal with data for entries that it shows.

But even in your case, with your list view showing actually all 30000 items at once, it should be faster.

Cheers,
_

[Lesiok]

Why every time You create the paramListWidget from scratch ?
Is it not enough to do show / hide on the elements of the Storage::identifier list ?

[Ini]

mainwindow.cpp

#include "mainwindow.h"
#include "ui_mainwindow.h"
 
MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);
 
    ui->identifier_progressBar->setVisible(false);
 
    for (int i = 0; i < 10000; i++) {
        list << QString("test"+QString::number(i));
        ui->identifier_listWidget->addItem(list.at(i));
    }
 
    connect(ui->identifier_lineEdit_search_0,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_1,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_2,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_3,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_4,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
}
 
MainWindow::~MainWindow()
{
    delete ui;
}
 
void MainWindow::searchIn_identifier_listWidget()
{
    searchIn_identifier_listWidget_eventLoopControl = false;
 
    QList<QString> tempList;
    int max = 0;
 
    QString text0 = ui->identifier_lineEdit_search_0->text();
    QString text1 = ui->identifier_lineEdit_search_1->text();
    QString text2 = ui->identifier_lineEdit_search_2->text();
    QString text3 = ui->identifier_lineEdit_search_3->text();
    QString text4 = ui->identifier_lineEdit_search_4->text();
 
    for (int i = 0; i < list.count(); i++) {
        qApp->processEvents();
        if (searchIn_identifier_listWidget_eventLoopControl) {
            return;
        }
        if (list.at(i).contains(text0,Qt::CaseInsensitive) &&
            list.at(i).contains(text1,Qt::CaseInsensitive) &&
            list.at(i).contains(text2,Qt::CaseInsensitive) &&
            list.at(i).contains(text3,Qt::CaseInsensitive) &&
            list.at(i).contains(text4,Qt::CaseInsensitive))
        {
            tempList << list.at(i);
        }
    }
 
    ui->identifier_progressBar->setVisible(true);
    ui->identifier_listWidget->clear();
    ui->identifier_progressBar->setValue(0);
    max = tempList.count();
    ui->identifier_progressBar->setMaximum(max-1);
    for (int i = 0; i < max; i++) {
        qApp->processEvents();
        if (searchIn_identifier_listWidget_eventLoopControl) {
            return;
        }
        ui->identifier_listWidget->addItem(tempList.at(i));
        ui->identifier_progressBar->setValue(i);
    }
    ui->identifier_progressBar->setVisible(false);
 
    searchIn_identifier_listWidget_eventLoopControl = true;
}

#include "mainwindow.h"
#include "ui_mainwindow.h"

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    ui->identifier_progressBar->setVisible(false);

    for (int i = 0; i < 10000; i++) {
        list << QString("test"+QString::number(i));
        ui->identifier_listWidget->addItem(list.at(i));
    }

    connect(ui->identifier_lineEdit_search_0,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_1,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_2,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_3,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
    connect(ui->identifier_lineEdit_search_4,SIGNAL(textChanged(QString)),this,SLOT(searchIn_identifier_listWidget()));
}

MainWindow::~MainWindow()
{
    delete ui;
}

void MainWindow::searchIn_identifier_listWidget()
{
    searchIn_identifier_listWidget_eventLoopControl = false;

    QList<QString> tempList;
    int max = 0;

    QString text0 = ui->identifier_lineEdit_search_0->text();
    QString text1 = ui->identifier_lineEdit_search_1->text();
    QString text2 = ui->identifier_lineEdit_search_2->text();
    QString text3 = ui->identifier_lineEdit_search_3->text();
    QString text4 = ui->identifier_lineEdit_search_4->text();

    for (int i = 0; i < list.count(); i++) {
        qApp->processEvents();
        if (searchIn_identifier_listWidget_eventLoopControl) {
            return;
        }
        if (list.at(i).contains(text0,Qt::CaseInsensitive) &&
            list.at(i).contains(text1,Qt::CaseInsensitive) &&
            list.at(i).contains(text2,Qt::CaseInsensitive) &&
            list.at(i).contains(text3,Qt::CaseInsensitive) &&
            list.at(i).contains(text4,Qt::CaseInsensitive))
        {
            tempList << list.at(i);
        }
    }

    ui->identifier_progressBar->setVisible(true);
    ui->identifier_listWidget->clear();
    ui->identifier_progressBar->setValue(0);
    max = tempList.count();
    ui->identifier_progressBar->setMaximum(max-1);
    for (int i = 0; i < max; i++) {
        qApp->processEvents();
        if (searchIn_identifier_listWidget_eventLoopControl) {
            return;
        }
        ui->identifier_listWidget->addItem(tempList.at(i));
        ui->identifier_progressBar->setValue(i);
    }
    ui->identifier_progressBar->setVisible(false);

    searchIn_identifier_listWidget_eventLoopControl = true;
}

To copy to clipboard, switch view to plain text mode 

mainwindow.h

#ifndef MAINWINDOW_H
#define MAINWINDOW_H
 
#include <QMainWindow>
 
namespace Ui {
class MainWindow;
}
 
class MainWindow : public QMainWindow
{
    Q_OBJECT
 
public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();
 
    QList<QString> list;
    bool searchIn_identifier_listWidget_eventLoopControl = false;
 
private:
    Ui::MainWindow *ui;
 
private slots:
    void searchIn_identifier_listWidget();
};
 
#endif // MAINWINDOW_H

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();

    QList<QString> list;
    bool searchIn_identifier_listWidget_eventLoopControl = false;

private:
    Ui::MainWindow *ui;

private slots:
    void searchIn_identifier_listWidget();
};

#endif // MAINWINDOW_H

To copy to clipboard, switch view to plain text mode 

This is how it works and should work. But code is so ugly. I would need a different solution to the nested events.

edit: AHHH WAIT i need to add GUI code

edit: heres the GUI code, idk if its OK like this if not I upload the project somewhere. Is it ok like this?

mainwindow.ui

<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
 <class>MainWindow</class>
 <widget class="QMainWindow" name="MainWindow">
  <property name="geometry">
   <rect>
    <x>0</x>
    <y>0</y>
    <width>809</width>
    <height>768</height>
   </rect>
  </property>
  <property name="windowTitle">
   <string>MainWindow</string>
  </property>
  <widget class="QWidget" name="centralWidget">
   <widget class="QListWidget" name="identifier_listWidget">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>310</y>
      <width>541</width>
      <height>261</height>
     </rect>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_4">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>230</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_3">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>190</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QPushButton" name="identifier_pushButton_clear_search">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>270</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="text">
     <string>Clear search</string>
    </property>
   </widget>
   <widget class="QProgressBar" name="identifier_progressBar">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>30</y>
      <width>541</width>
      <height>23</height>
     </rect>
    </property>
    <property name="value">
     <number>24</number>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_0">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>70</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_1">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>110</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_2">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>150</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
  </widget>
  <widget class="QMenuBar" name="menuBar">
   <property name="geometry">
    <rect>
     <x>0</x>
     <y>0</y>
     <width>809</width>
     <height>21</height>
    </rect>
   </property>
  </widget>
  <widget class="QToolBar" name="mainToolBar">
   <attribute name="toolBarArea">
    <enum>TopToolBarArea</enum>
   </attribute>
   <attribute name="toolBarBreak">
    <bool>false</bool>
   </attribute>
  </widget>
  <widget class="QStatusBar" name="statusBar"/>
 </widget>
 <layoutdefault spacing="6" margin="11"/>
 <resources/>
 <connections/>
</ui>

<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
 <class>MainWindow</class>
 <widget class="QMainWindow" name="MainWindow">
  <property name="geometry">
   <rect>
    <x>0</x>
    <y>0</y>
    <width>809</width>
    <height>768</height>
   </rect>
  </property>
  <property name="windowTitle">
   <string>MainWindow</string>
  </property>
  <widget class="QWidget" name="centralWidget">
   <widget class="QListWidget" name="identifier_listWidget">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>310</y>
      <width>541</width>
      <height>261</height>
     </rect>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_4">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>230</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_3">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>190</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QPushButton" name="identifier_pushButton_clear_search">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>270</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="text">
     <string>Clear search</string>
    </property>
   </widget>
   <widget class="QProgressBar" name="identifier_progressBar">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>30</y>
      <width>541</width>
      <height>23</height>
     </rect>
    </property>
    <property name="value">
     <number>24</number>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_0">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>70</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_1">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>110</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
   <widget class="QLineEdit" name="identifier_lineEdit_search_2">
    <property name="geometry">
     <rect>
      <x>40</x>
      <y>150</y>
      <width>541</width>
      <height>31</height>
     </rect>
    </property>
    <property name="placeholderText">
     <string>Search</string>
    </property>
   </widget>
  </widget>
  <widget class="QMenuBar" name="menuBar">
   <property name="geometry">
    <rect>
     <x>0</x>
     <y>0</y>
     <width>809</width>
     <height>21</height>
    </rect>
   </property>
  </widget>
  <widget class="QToolBar" name="mainToolBar">
   <attribute name="toolBarArea">
    <enum>TopToolBarArea</enum>
   </attribute>
   <attribute name="toolBarBreak">
    <bool>false</bool>
   </attribute>
  </widget>
  <widget class="QStatusBar" name="statusBar"/>
 </widget>
 <layoutdefault spacing="6" margin="11"/>
 <resources/>
 <connections/>
</ui>

To copy to clipboard, switch view to plain text mode 

[anda_skoa]

Why every time You create the paramListWidget from scratch ?

Ini doesn't like simple solutions, probably working for a company that punishes people for easy to maintain code.

Is it not enough to do show / hide on the elements of the Storage::identifier list ?

That might work, as it fullfills the requirement of making the search result faster and the requirement of involving lots of code with loops.

Cheers,
_

[Ini]

sry but not helpful. try a solution with hide and you see it works much worse/slower.

if not upload it pls instead of fooling me

---

Added after 18 minutes:

how can it be possible to show a result after every character typed in without an nested event. I know no solution. Its not requirement to make it faster requirement is to make code more beatiful

[anda_skoa]

I'll let Lesiok handle that since you are ignoring my advice anyway.

Cheers,
_

[Ini]

no I do not I work on the problem and not insert code that just speed it up my few ms. I can do that later

[anda_skoa]

no I do not I work on the problem and not insert code that just speed it up my few ms. I can do that later

Ok, if you don't want Lesiok's help to speed up things either, then I guess you'll have to wait for someone new that you can then ignore.
Good luck with that.

Cheers,
_

[Lesiok]

QListWidget has such a cool method QListWidget::insertItems.

[wysota]

Regarding search and waiting for the results,

Your algorithm is naive and far away from being optimal. It would be much better to preprocess the original data into some kind of prefix tree and search that tree instead.

[d_stranz]

@wysota - I'm afraid you don't understand. The OP doesn't want a better solution that works faster. He wants his current solution to work faster without changing anything about what he's written.