Results 1 to 6 of 6

Thread: Waterfall Function Graph UI for Massive RF Signal Data

  1. #1
    Join Date
    Jun 2018
    Location
    Silicon Valley
    Posts
    6
    Thanks
    3
    Qt products
    Qt5
    Platforms
    Windows

    Question Waterfall Function Graph UI for Massive RF Signal Data

    Hello,
    I am relatively new to Qt, but my first on the job assignment is quite formidable. Any comments or help are welcome, but I also do not expect that many people will be able to help me without having the code in front of them. As I said, I am relatively new to Qt, so I may also not be able to understand your answers very well. Any way, here is my problem;
    I have been tasked with processing RF Signals (Currently in the form of a data file, but eventually will be able to interpret real world data live), which transmit approximately 4 gigabytes of data per second. As it stands now, I have a functional application written primarily in C++ using a Qt UI that can handle the data, but at a speed nowhere close to real time. After speaking with a rep from Qt, they have indicated that Qt 3d may be capable of handling the massive amounts of data, but I am not entirely sure how to begin designing with Qt 3d. I would like to take the RF signals, and plot them on a 2d waterfall spectrogram graph, with a color component indicating a z-axis.

    Other issues are as follows;
    Whenever I resize the window all my waterfall graph data disappears, but the data file itself does not loop to the beginning (nor should it)
    If I zoom in, the program crashes
    There is significant lag in the data speed, and the UI designed to track the data.
    While I am sure some of these issues are bug related (the code is far from flawless), my primary concern is getting the data to process realtime. (A speed increase of about 20x). The project is also 50+ files, and about 10,000 lines of code or so, and I'm currently working by myself on the project, so a solution that says something along the line of "in main just implement this two line fix" probably isn't gonna work.

    So, if you have any idea how to design a waterfall graph in Qt that plots massive amounts of data per second, I'd love some help. I am also not sure if it is possible to perhaps use my GPU to run the program, so that may also be a part of a solution. Truthfully, I have no idea if any one will be able to help, but even telling me what you think I should google may provide some insight.

  2. #2
    Join Date
    Jan 2006
    Location
    Munich, Germany
    Posts
    4,708
    Thanks
    21
    Thanked 418 Times in 411 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows

    Default Re: Waterfall Function Graph UI for Massive RF Signal Data

    As you wrote your self, with such very general information it is hard to give you any practical help.
    That said, here are some thoughts that might help nudge something on your side on the right direction.
    which transmit approximately 4 gigabytes of data per second.
    This is a huge amount of data/sec.
    Qt company was right to point you to Qt3d in the sense, that the only chance with today's HW (with reasonable cost) is to do this directly on the GPU.
    But Qt3D is not the only option.
    You can also use OpenGL for example or even Qt's GraphicsView framework - and be sure to set the HW rendering on.
    [QUOTE]but I am not entirely sure how to begin designing with Qt 3d[QUOTE]
    You know what's coming as an answer here right? There is no another way but to read the documentation, examples, and work your way up.

    . (A speed increase of about 20x).
    Well, a factor of 20 is no joke.
    If I were you, I would first analyze to see where is the bottleneck.
    Also, since I don't know how your code is designed - make sure your code has two parts: the backend which is crunching the data, and a UI part, which is dumb and only does display.
    This has to do with the analysis too.
    You need to optimize both, and you better do that separately.
    One way of doing the data crunching would be to use shaders on the GPU (General purpose GPU programming - google it)

    I would start with the data crunching first - there you can also use low level CPU specific functions to move large data chunks in more efficient way.(like SSE2)
    Bare in mind that your non UI part needs to be faster than that factor of 20, since the UI will also take its price.
    If you manage the backend part. you know you might have a chance with the UI, but if the backend part is not optimized to reach the performance goal, no chance that with UI it will be any faster.

    Probably no what you wanted, but its the best I can offer.
    ==========================signature=============== ==================
    S.O.L.I.D principles (use them!):
    https://en.wikipedia.org/wiki/SOLID_...iented_design)

    Do you write clean code? - if you are TDD'ing then maybe, if not, your not writing clean code.

  3. #3
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    3,927
    Thanks
    229
    Thanked 615 Times in 605 Posts
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows Android

    Default Re: Waterfall Function Graph UI for Massive RF Signal Data

    which transmit approximately 4 gigabytes of data per second.
    These kinds of data rates are not uncommon in some areas of scientific data acquisition, too. In typical applications I am aware of, there is a heavy amount of signal averaging that occurs between receiving the data from the detector and displaying it or saving it to disk. GPUs are great at this, and the first rule of GPU processing is get the data onto the GPU and never move it back - data transfers between GPU and CPU are the slowest part of CPU/GPU applications. So this implies that you want to use GPU-based graphics (eg. OpenGL) if possible.

    The unfortunate thing about Qt 3d is that it seems to be designed mostly for use for games or similar things requiring photo-realistic scene rendering. No one has yet written any kind of 2- or 3-D data plotting layer for it.

    You might want to look at something like VTK. It also has a steep learning curve, but it is designed for scientific data visualization and does have a Qt wrapper so you can embed VTK graphics into a Qt GUI.
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

  4. #4
    Join Date
    Jun 2018
    Location
    Silicon Valley
    Posts
    6
    Thanks
    3
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: Waterfall Function Graph UI for Massive RF Signal Data

    Very helpful actually my friend. The data is relatively fixed, but I think you may be on to something with HW rendering using Qt's framework/OpenGL. Do you happen to have any experience in this area? I agree with you that I need to work my way up, but I am a tad lost on the best place to start as many of the examples and docs from Qt.io are far too simple to be helpful. If you can't no worries, but just pointing me in the right direction with OpenGL/Qt3d would be great. OpenGL rendering sounds like the right option. My boss also is under the opinion that qml files have too much overhead from Java, and that I am better off sticking to C++, any idea if there is a feasible application for qml that I am missing out on? Your help is very appreciated, and thank you very much for your reply.
    Last edited by Yippiyak; 27th June 2018 at 19:11.

  5. #5
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    3,927
    Thanks
    229
    Thanked 615 Times in 605 Posts
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows Android

    Default Re: Waterfall Function Graph UI for Massive RF Signal Data

    I'd say the Qt3D Extras and their C++ examples would be a good place to start. A bit higher lever than naked Qt3D, and not based on QML. IMHO, QML is good for toy interfaces on mobile devices, but not for serious desktop GUIs, and I wish the Qt folks would stop putting out so many new examples in QML only.
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

  6. #6
    Join Date
    Jun 2018
    Location
    Silicon Valley
    Posts
    6
    Thanks
    3
    Qt products
    Qt5
    Platforms
    Windows

    Thumbs up Re: Waterfall Function Graph UI for Massive RF Signal Data

    Quote Originally Posted by d_stranz View Post
    IMHO, QML is good for toy interfaces on mobile devices, but not for serious desktop GUIs, and I wish the Qt folks would stop putting out so many new examples in QML only.
    Right? Its pretty much useless! Thank you for the advice

Similar Threads

  1. displaying a recorded signal as a graph
    By Aiswarya in forum Qwt
    Replies: 1
    Last Post: 21st April 2011, 10:03
  2. QTableView massive model data updates issue
    By cyberbob in forum Newbie
    Replies: 3
    Last Post: 8th December 2010, 00:09
  3. Replies: 5
    Last Post: 21st March 2009, 10:10
  4. QTableWidget massive select -> massive signals
    By zarkzervo in forum Qt Programming
    Replies: 0
    Last Post: 16th March 2009, 15:01
  5. Signal and slot connection graph
    By fahlen in forum General Discussion
    Replies: 4
    Last Post: 27th November 2007, 14:47

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.