Results 1 to 5 of 5

Thread: Llarge set of data in QML List view/C++ List model

  1. #1
    Join Date
    Oct 2016
    Posts
    3
    Thanks
    2

    Default Llarge set of data in QML List view/C++ List model

    Hello All,

    In my project, I need to display list of some data using QML List view.And, the QT version I am using is QT 5.0.2. Since, scroll bar view is not available in this version of Qt, I managed to implement it by following this example.

    Now, the data source is an other component which has provided C++ APIs to request for range of data. And the number of elements in the data set can be huge (in terms of tens of thousands).

    I have followed below steps to achieve my requirements :

    1. I have a created a C++ class that inherits from QAbstractListModel and I have overridden rowCount, data, canfetchmore and fetch more functions.
    2. And in the QML, I have referred the instance C++ as model.
    3. Whenever fetchmore() function is called, I am requesting next set of data using C++ APIs that are provided by another component and appending the same to my list model.


    So far, it is good. But I need few clarifications -

    1. If I keep appending the data to my list model,then I will end up storing thousands of elements in the list (I don't want to do that). I want to limit the number of elements in the list and use it as a ring buffer. Is this possible?
    2. QModelIndex parameter in fetchmore is received as invalid Index. Why?
    3. Assume a case : alphabets are mentioned on scrollbar, and when user selects Y on scrollbar, I should fetch elements that starts with Y. Here, I can fetch the data from my external component and reset the list model with new data. But, if I do that,then my list will start with Y-elements and I will not be able to scroll up to get previous elements. -- Is there any other way to achieve this?
    4. Also, my other component takes considerable time if I ask huge set of data (for example in cases as mentioned in point 3) - Any suggestions to make a better design to handle this case?
    5. Is there any way, other than fetchmore(), to update the model dynamically.



    I have spent good amount of time on google to get answers to my questions and I have successfully FAILED.

    Any help appreciated. Thanks in advance.

  2. #2
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,420
    Thanks
    37
    Thanked 1,545 Times in 1,495 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Llarge set of data in QML List view/C++ List model

    Quote Originally Posted by ashwasimha View Post
    If I keep appending the data to my list model,then I will end up storing thousands of elements in the list (I don't want to do that). I want to limit the number of elements in the list and use it as a ring buffer. Is this possible?
    The model only needs to be able to provide access to data, it doesn't have to store the data.
    If you actual data allows you to query for specific data then the model can use that as well.

    Quote Originally Posted by ashwasimha View Post
    QModelIndex parameter in fetchmore is received as invalid Index. Why?
    A list is a flat structure, there are not parent nodes.

    Quote Originally Posted by ashwasimha View Post
    Assume a case : alphabets are mentioned on scrollbar, and when user selects Y on scrollbar, I should fetch elements that starts with Y. Here, I can fetch the data from my external component and reset the list model with new data. But, if I do that,then my list will start with Y-elements and I will not be able to scroll up to get previous elements. -- Is there any other way to achieve this?
    It depends on what you use the reset for.
    When you reset the model and only provide Y elements, then that is all the data you have as far as the view is concerned.

    If you want to reset the model to only see a specific subset, e.g. to make a scrollbar not work on the whole set but only on the subset, then you need to detect when you need to switch out of that "detail" mode.
    E.g. some UI to "go back" to the full list, or header/footer items that trigger this return to the full mode.

    If you reset for some other reason the maybe the reset is not the right way to achieve the target result.

    Quote Originally Posted by ashwasimha View Post
    Also, my other component takes considerable time if I ask huge set of data (for example in cases as mentioned in point 3) - Any suggestions to make a better design to handle this case?
    That's difficult to answer generically, depends on what these components to and how they interact with the data.

    Quote Originally Posted by ashwasimha View Post
    Is there any way, other than fetchmore(), to update the model dynamically.
    A model can at any given times annouce that it is adding or removing rows, or announce that data of existing rows has changed.
    Adding/removing can be done through protected helper methods, data change is notified via the dataChanged() signal.

    Cheers,
    _

  3. The following user says thank you to anda_skoa for this useful post:

    ashwasimha (26th October 2016)

  4. #3
    Join Date
    Oct 2016
    Posts
    3
    Thanks
    2

    Default Re: Large set of data in QML List view/C++ List model

    Hi anda_skoa,

    Thank you for your response.

    Quote Originally Posted by anda_skoa View Post
    The model only needs to be able to provide access to data, it doesn't have to store the data.
    If you actual data allows you to query for specific data then the model can use that as well.
    _
    Agreed. But, my external component takes considerable time to retrieve the data. Hence, a buffer of data is maintained in model to achieve smooth scrolling.

    Quote Originally Posted by anda_skoa View Post
    A list is a flat structure, there are not parent nodes.
    _
    Ok. Understood! Thanks!

    Quote Originally Posted by anda_skoa View Post
    It depends on what you use the reset for.
    When you reset the model and only provide Y elements, then that is all the data you have as far as the view is concerned.

    If you want to reset the model to only see a specific subset, e.g. to make a scrollbar not work on the whole set but only on the subset, then you need to detect when you need to switch out of that "detail" mode.
    E.g. some UI to "go back" to the full list, or header/footer items that trigger this return to the full mode.

    If you reset for some other reason the maybe the reset is not the right way to achieve the target result.

    That's difficult to answer generically, depends on what these components to and how they interact with the data.


    A model can at any given times annouce that it is adding or removing rows, or announce that data of existing rows has changed.
    Adding/removing can be done through protected helper methods, data change is notified via the dataChanged() signal.

    Cheers,
    _
    My case is like this,

    1. Data source is an external component (another process) that provides data in chunks (It has provided APIs to request data in chunks). Number of elements in the source can be huge (in terms of tens of thousands).
    2. I have a class (say CDataFetcher) that interfaces with the external data source and fetches the data on need basis.
    3. Now, CListModel is my model that has inherited from QAbstractListModel.
    Qt Code:
    1. class CListModel : public QAbstractListModel
    2. {
    3. /*other methods and data members....*/
    4. int rowCount(const QModelIndex &parent = QModelIndex())
    5. const Q_DECL_OVERRIDE;
    6. QVariant data(const QModelIndex &index, int role = Qt::DisplayRole)
    7. const Q_DECL_OVERRIDE;
    8. QHash<int ,QByteArray>roleNames() const Q_DECL_OVERRIDE;
    9.  
    10. QStringList myListData;
    11.  
    12. };
    To copy to clipboard, switch view to plain text mode 

    Here myListData is the buffer of data that I am talking about. As the data can be huge, I am implementing a logic to depict myListData as ring buffer.

    Now, if, user scrolls the list either by dragging up or down on the list (please note this action is not done using the scroll bar), then I can provide the data to list view that is available in myListData. If there is no data available in myListData, then,I will fetch the data using CDataFetcher.

    Qt Code:
    1. QVariant data(const QModelIndex &index, int role = Qt::DisplayRole)
    2. {
    3. /*check if the index is in with in the number of elements in myListData*/
    4. /*If yes, return the data*/
    5. /*else, fetch the data from local copy in CDataFetcher*/
    6. }
    7.  
    8. QStringList CDataFetcher::FetchData()
    9. {
    10. /*This class works in a different thread*/
    11. /*Local copy is always maintained in this class*/
    12. /*If the copy is already used by model, then this class fetches the data and makes a local copy*/
    13. }
    To copy to clipboard, switch view to plain text mode 

    This way, though I am not storing the entire data in memory, I am achieving smooth scrolling by maintaining small chunks of data. This works fine for linear scrolling on the list.


    Now, in an other case (case 3 as I mentioned in my previous post), suppose user is viewing elements that starts with alphabet "C" and user selects alphabet "Y" on the scrollbar, I should show elements that starts from "Y". Here, this behaviour is similar to contact list behaviour in iphone.
    I can get list of elements that starts with "Y", but if I have to show it, (as per my design) I have to buffer it in my model. To do this, I will have to erase previous contents and store my current contents.

    When I do this, list view assumes that my list starts with elements "Y" and when user scrolls up on the list to view elements that starts with "X" or "W" or "V"..., I will not get any request to get previous data How can I fix this? Similar problem is explained in this forum.

    Ideally, if I had an API like scrollToItem or JumpToIndex, I would not have ran in to this problem.

    Let me know your comments on this. Please note that QT version that I am using 5.0.2

  5. #4
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,420
    Thanks
    37
    Thanked 1,545 Times in 1,495 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Large set of data in QML List view/C++ List model

    You must have really long strings if a couple dozend thousand is "huge".

    I see a couple options:

    1) you maintain the cache differently, e.g. by caching the data in a file and maintaining just indexes into the file inside the model
    2) you maintain a "sliding window" which has more rows than can possibly be shown. when the view starts requesting outside of the cached range, add new rows on that end and remove from the other
    3) similar to 2 but instead of adding/removing rows change content of existing rows to achieve the same end result

    Cheers,
    _

  6. The following user says thank you to anda_skoa for this useful post:

    ashwasimha (26th October 2016)

  7. #5
    Join Date
    Oct 2016
    Posts
    3
    Thanks
    2

    Default Re: Large set of data in QML List view/C++ List model

    Hi anda_skoa,

    Apologies for delay in response. I was in vacation.

    Quote Originally Posted by anda_skoa View Post
    You must have really long strings if a couple dozend thousand is "huge".

    I see a couple options:

    1) you maintain the cache differently, e.g. by caching the data in a file and maintaining just indexes into the file inside the model
    2) you maintain a "sliding window" which has more rows than can possibly be shown. when the view starts requesting outside of the cached range, add new rows on that end and remove from the other
    3) similar to 2 but instead of adding/removing rows change content of existing rows to achieve the same end result

    Cheers,
    _

    Thank you for your options. Yes, I have followed option that is similar to "option-3" that you have mentioned.

    Brief explanation of strategy that was followed (hopefully it will help someone) :
    There will be a separate thread that requests the data from other component and fills in to a buffer(buffer size if configurable,for ex : an array of 100 elements). When qt framework requests for data using "data(const QModelIndex &index, int role = Qt:: DisplayRole)" function, get the data that is available in buffer. I have implemented a sliding window logic to make sure that data is always available in the buffer when user is scrolling up/down. This way I managed to have only 100 elements in memory at any instance of time.

    Thank you for your support.

Similar Threads

  1. Replies: 9
    Last Post: 8th March 2011, 08:35
  2. List View with sections for alphabetialy sorted list
    By woodtluk in forum Qt Programming
    Replies: 4
    Last Post: 12th October 2010, 11:50
  3. Replies: 8
    Last Post: 6th May 2010, 11:17
  4. How to map tree model data to list view
    By msopanen in forum Qt Programming
    Replies: 0
    Last Post: 10th November 2009, 19:56
  5. [QT4.1.1 XP] multi-columns list model/view
    By incapacitant in forum Newbie
    Replies: 1
    Last Post: 26th February 2006, 13:38

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.