Results 1 to 4 of 4

Thread: bluetooth LE receive bandwidth

  1. #1
    Join Date
    May 2015
    Posts
    3
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default bluetooth LE receive bandwidth

    i have upgraded my hardware from a china ble module to a dedicated northern semiconductor arm cortex-m4f mcu which also holds now all of the firmware together with the nordic serial data protocol (20 bytes max / gatt pkg)

    this hw sends out binary data pkgs, either to an android app running on a nexus-7 2013 or a ubuntu 16.04 or 18.04 desktop computer

    data rcve rate is about 30 bytes every 50 ms, so roughly 600 bytes / sec (6 k baud)

    for comparison, the android app consists of the same c++ bin pkg decode code in the JNI portion of the app and java is getting the decoded data, stuffs it into it's data array and displays it on the tablet

    the QT app uses the exact same c++ bin pkg decode code and puts the data into the data structs ... a timer updates this data on the appropriate dialog window

    the android app with a humble quad core arm cpu can keep up with a pkg rate of 20 ms / 30 byte pkgs, which was a big improvement when i switched the china ble modules against the nordic mcus

    the QT desktop apps dies slowly even with the 50 ms data pkg rate ... what i mean by it is observing the system monitor cpu load and having some dbg info (single char print) for each pkg reception as well as a 1000 rcv char cnt timestamp

    it all starts out working fine for a few secs and then the cpu load on the 8 cores (evenly spread out, one at a time) goes up and goes down again the next time it goes up higher and after 30 plus secs this game happened about 10 plus times and then one cpu reaches 100 % of it's slice for too long and the whole thing hangs ... also QT debug mode with breakpoints doesn't seem to like ble based apps since it never connects

    i don't know how else to describe it

    on another test app i only count 1000 rcv bytes and print a timestamp and nothing else and i can reach a 30 ms pkg rate ... if i want to display the data in 2 byte hex format in a text edit it dies the same way

    is this all i can expect from an amd 8 core or and intel i7 8 core cpu on a QT desktop computer running the above mentioned ubuntu versions ???

    at this point i wouldn't even know how to spit the whole app up into threads or similar to even the load

    for the simple bandwidth test app i see no reason what part to put on a separate thread

    for the data processing app i have 2 solutions, one being single threaded and the other one as follows

    the code for "characteristicChanged" slot sticks the data into a very simple queue and a 2nd thread checks to see if there is something in this que and takes it out and does the pkg decode and sticks it into the data struct
    the main app updates the gui based upon a timer from this data struct
    note, it's interesting that the que never gets full, overflow ... how comes that the ble rcve from the qt library doesn't run independent in it's own thread (environment) and produces the rcv data regardless of the main app ... this data has to come from a system driver ???

    can this QT ble data rcv library be run in it's own thread ... right now i wouldn't know how

    there seemed no performance difference for both app versions (single vs double threaded)

    sorry for the lengthy description

    i feel that others might run eventually into similar issues and face the same ble bandwidth dilemma.

    i have not found anything applicable to my performance issue, but maybe i'm not good at gooooogling

    any thoughts and suggestions are highly appreciated

    cheers efiLabs

  2. #2
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    5,230
    Thanks
    302
    Thanked 864 Times in 851 Posts
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: bluetooth LE receive bandwidth

    the QT desktop apps dies slowly even with the 50 ms data pkg rate ... what i mean by it is observing the system monitor cpu load and having some dbg info (single char print) for each pkg reception as well as a 1000 rcv char cnt timestamp

    it all starts out working fine for a few secs and then the cpu load on the 8 cores (evenly spread out, one at a time) goes up and goes down again the next time it goes up higher and after 30 plus secs this game happened about 10 plus times and then one cpu reaches 100 % of it's slice for too long and the whole thing hangs ... also QT debug mode with breakpoints doesn't seem to like ble based apps since it never connects
    This sounds like a classic case of a memory leak where your app continuously allocates more memory without freeing any up. Eventually your system chokes and dies because all available RAM has been allocated. If you are running a 64-bit app with a 64-bit memory address space, your program could request terabytes of space, far beyond physical RAM.

    Look for places where you are pushing data into buffers or other data structures that continue to grow. At a 50ms receive rate, this could happen pretty quickly.

    Also look for algorithms where you are either repeatedly copying data structures that get larger and larger, or repeatedly processing large data structures over and over starting from their beginning, etc. The bigger things get, the more time it takes to process them, and eventually your program is spinning its wheels copying data around or rehashing the same old thing millions of times.
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

  3. #3
    Join Date
    May 2015
    Posts
    3
    Qt products
    Qt5
    Platforms
    Unix/X11 Windows

    Default Re: bluetooth LE receive bandwidth

    Quote Originally Posted by efiLabs View Post
    i have upgraded my hardware from a china ble module to a dedicated northern semiconductor arm cortex-m4f mcu which also holds now all of the firmware together with the nordic serial data protocol (20 bytes max / gatt pkg)

    this hw sends out binary data pkgs, either to an android app running on a nexus-7 2013 or a ubuntu 16.04 or 18.04 desktop computer

    data rcve rate is about 30 bytes every 50 ms, so roughly 600 bytes / sec (6 k baud)

    for comparison, the android app consists of the same c++ bin pkg decode code in the JNI portion of the app and java is getting the decoded data, stuffs it into it's data array and displays it on the tablet

    the QT app uses the exact same c++ bin pkg decode code and puts the data into the data structs ... a timer updates this data on the appropriate dialog window

    the android app with a humble quad core arm cpu can keep up with a pkg rate of 20 ms / 30 byte pkgs, which was a big improvement when i switched the china ble modules against the nordic mcus

    the QT desktop apps dies slowly even with the 50 ms data pkg rate ... what i mean by it is observing the system monitor cpu load and having some dbg info (single char print) for each pkg reception as well as a 1000 rcv char cnt timestamp

    it all starts out working fine for a few secs and then the cpu load on the 8 cores (evenly spread out, one at a time) goes up and goes down again the next time it goes up higher and after 30 plus secs this game happened about 10 plus times and then one cpu reaches 100 % of it's slice for too long and the whole thing hangs ... also QT debug mode with breakpoints doesn't seem to like ble based apps since it never connects

    i don't know how else to describe it

    on another test app i only count 1000 rcv bytes and print a timestamp and nothing else and i can reach a 30 ms pkg rate ... if i want to display the data in 2 byte hex format in a text edit it dies the same way

    is this all i can expect from an amd 8 core or and intel i7 8 core cpu on a QT desktop computer running the above mentioned ubuntu versions ???

    at this point i wouldn't even know how to spit the whole app up into threads or similar to even the load

    for the simple bandwidth test app i see no reason what part to put on a separate thread

    for the data processing app i have 2 solutions, one being single threaded and the other one as follows

    the code for "characteristicChanged" slot sticks the data into a very simple queue and a 2nd thread checks to see if there is something in this que and takes it out and does the pkg decode and sticks it into the data struct
    the main app updates the gui based upon a timer from this data struct
    note, it's interesting that the que never gets full, overflow ... how comes that the ble rcve from the qt library doesn't run independent in it's own thread (environment) and produces the rcv data regardless of the main app ... this data has to come from a system driver ???

    can this QT ble data rcv library be run in it's own thread ... right now i wouldn't know how

    there seemed no performance difference for both app versions (single vs double threaded)

    sorry for the lengthy description

    i feel that others might run eventually into similar issues and face the same ble bandwidth dilemma.

    i have not found anything applicable to my performance issue, but maybe i'm not good at gooooogling

    any thoughts and suggestions are highly appreciated

    cheers efiLabs

    i would like to provide some code snippets on the extremely simple byte counting and display app

    ble connect to service funct setting up signal / slot for "characteristicChanged", which is the byte rcve part

    Qt Code:
    1. void TDevice::ConnectToService (void)
    2. {
    3. setUpdMsg ("Connecting to service ...") ;
    4.  
    5. SioChariF = RxChariF = TxChariF = false ;
    6.  
    7. /*! [ble-service 2] */
    8. connect (pService, SIGNAL (error (QLowEnergyService::ServiceError)),
    9. this, SLOT (SvcError (QLowEnergyService::ServiceError))) ;
    10. connect (pService, SIGNAL (stateChanged (QLowEnergyService::ServiceState)),
    11. this, SLOT (SvcDetailsDisc (QLowEnergyService::ServiceState))) ;
    12. connect (pService, SIGNAL (characteristicChanged (QLowEnergyCharacteristic, QByteArray)),
    13. this, SLOT (RdyRead (QLowEnergyCharacteristic, QByteArray))) ;
    14. /*connect (pService, SIGNAL (characteristicWritten (QLowEnergyCharacteristic, QByteArray)),
    15. this, SLOT (WrData (QLowEnergyCharacteristic, QByteArray))) ; */
    16.  
    17. pService->discoverDetails () ;
    18.  
    19. setUpdMsg ("Discovering details ...") ;
    20. /*! [ble-service 2] */
    21. }
    To copy to clipboard, switch view to plain text mode 

    i like to have a dedicated read funct in the ble device module and provide rdyRead read signal to outside
    contains still some now unused debug statements

    Qt Code:
    1. void TDevice::RdyRead (const QLowEnergyCharacteristic &chari, const QByteArray &val)
    2. {
    3. //qDebug () << "dev.ChariValChg" << chari.uuid () << val ;
    4.  
    5. if (chari != TxChari) return ;
    6.  
    7. //MSG_PRN (".") ;
    8. emit rdyRead (val) ;
    9.  
    10. //qDebug () << "dev.ChariChg" << chari.uuid () << val ;
    11. }
    To copy to clipboard, switch view to plain text mode 

    providing the signal / slot connect funct in the module of the data count and display processing

    Qt Code:
    1. void TTestDlg::Connect (void)
    2. {
    3. Com.Lim (Sze) ;
    4.  
    5. connect (Device, SIGNAL (rdyRead (QByteArray)),
    6. this, SLOT (Put (QByteArray))) ;
    7. }
    To copy to clipboard, switch view to plain text mode 

    this is the final data byte counting and display funct ... nothing more which would to my opinion produce a memory leak ... there is nowhere any dynamic "new / delete" operation used

    " if (!DispF) return ;" determines the counting only or also subsequent data display

    Qt Code:
    1. void TTestDlg::Put (const QByteArray &data)
    2. {
    3. //if (!this->isVisible ()) return ;
    4. //if (!this->hasFocus ()) return ;
    5.  
    6. static qint64 t64 = QDateTime::currentMSecsSinceEpoch () ;
    7.  
    8. enum { eCnt = 1000 } ;
    9. static int cnt = 0 ;
    10. int len = data.length () ;
    11. for (char *p = (char *) data.data (),
    12. *q = &p [len] ; p < q ; p++) {
    13. if (!(cnt++ % eCnt)) {
    14. qint64 t = QDateTime::currentMSecsSinceEpoch () ;
    15. MSG_PRN ("%d-%04d ", cnt / eCnt, t - t64) ; t64 = t ;
    16. }
    17. }
    18. if (!DispF) return ;
    19.  
    20. if (!BinF && data.length () == 1 && data [0] == '\n') return ;
    21.  
    22. // qDebug () << data ;
    23.  
    24. if (!BinF) ui->MsgTxe->insertPlainText (QString (data)) ;
    25. else {
    26. uint sz = data.length () ;
    27. char buf [0x200], *p = buf ;
    28. for (uint i = 0 ; i < sz ; i++) {
    29. uint x = data [i] ;
    30. *p++ = i2hex (x >> 4) ;
    31. *p++ = i2hex (x) ;
    32. *p++ = ' ' ;
    33. } *p = '\0' ;
    34. ui->MsgTxe->insertPlainText (buf) ;
    35. }
    36. QScrollBar *bar = ui->MsgTxe->verticalScrollBar () ;
    37. bar->setValue (bar->maximum ()) ;
    38. }
    To copy to clipboard, switch view to plain text mode 

    Qt Code:
    1. static char i2hex (uint x) { return "0123456789abcdef" [x & 0x0f] ; }
    To copy to clipboard, switch view to plain text mode 

    i realize that i have 2 signal / slots chained together, but i don't know the penalty for doing this as compared to one

    see attached pict :

    console is my debug outlet with "1-2144 2-2306 3-2249 ..." the 1st number being the 1 k byte count (1 2 3 ...) and the second (2144 2306 2249) the time in msec it took to rcve 1 k bytes, which is roughly 450 / sec

    Test-Dlg will print the incoming hex values (bin check-box) if the disp check-box is checked

    this code handles the 450 bytes / sec for counting only fine, as soon as i start displaying it it freezes slowly over a period of minutes

    at this point i do not see any use of dynamic mem allocation like "new / delete" which would cause mem leaks and eventually consuming all avail mem

    i realize that the Test-Dlg gui has to update a hex pair and space 450 times a sec ... so every 2 ms ... is this asking too much ???

    this is more or less the whole byte count and display app and a bit glue fluff around it ... it would provide also data transmit, which is not being used here

    and why does it take several minutes to freeze


    Attachment 12962

    here are pics from the system monitor showing the gradual increase until this simple byte counting / display app freezes

    Attachment 12963Attachment 12964Attachment 12965

    i also ran this simple test app in valgrind (i'm new to it) for a short while watching the data coming in and there were no leaks detected while running

    upon closing my simple app, 5 leaks showed up, where 4 of them appeared purely within some libs ... basically not directly called by my code

    one line pointed out as a leak was my code calling "discoverServices ()" which is a QT bluetooth lib funct to start discovering services

    Qt Code:
    1. void TDevice::DevConnect (void)
    2. {
    3. setUpdMsg ("Discovering services ...") ;
    4. ConnF = true ;
    5. /*! [ble-service 1] */
    6. pControl->discoverServices () ;
    7. /*! [ble-service 1] */
    8. }
    To copy to clipboard, switch view to plain text mode 

    here is the valgrind info related to those leaks

    valgrind-errs0.jpgvalgrind-errs1.jpg

    again, i'm new to valgrind, which seems to be a cooooooool tool

    why does it freeze after a few minutes receiving data with gradually increasing cpu usage

    and if there is a mem leak, how can this be found as long as my code doesn't make unbalanced "new / delete" calls

    yes, i use "new / delete" in the apps ctor to instantiate the various dialogs and close (delete) them upon app close

    well, i ran valgrind now directly from the cli (not through valkyrie gui) and saved the cli output

    and posted it as BleTerm-leaks0.txt and BleTerm-leaks2.txt attachment (one exceeded the up[load size limit)

    anyway, the only real leak possibly caused by my code would in the QT ble lib usage setup and it seems to be in error, since i delete the allocated "pControl = new QLowEnergyController (DevInfo.getDevice ().address()) ;" later at several places, including the dtor

    and now i'm worried that no one will answer after such a lengthy explanation
    Attached Files Attached Files
    Last edited by efiLabs; 9th September 2018 at 03:49.

  4. #4
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    5,230
    Thanks
    302
    Thanked 864 Times in 851 Posts
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: bluetooth LE receive bandwidth

    at this point i do not see any use of dynamic mem allocation like "new / delete" which would cause mem leaks and eventually consuming all avail mem
    There may be memory allocations internal to Qt or whatever other libraries you are using where ownership of the allocated memory is transferred to your program. It also doesn't have to be a classic unpaired new / delete problem. It could be that there are buffers being allocated that should be cleared and aren't, so they just continue to grow.

    so every 2 ms ... is this asking too much ???
    There is a reason why refresh rates for video displays are in the less than 100 Hz range. The human visual system doesn't respond consciously to anything that occurs faster than 30 Hz, and even then merges things into a continuous stream, like video. Your reaction time to something that happens suddenly is on the order of 0.3 s or greater. So updating a UI with a value that changes at 450 Hz is sort of useless. It would make more sense to display a moving average at less than 30 Hz, and even then if the value changes each time all your eye will see is blurred numbers.

    If the number you are updating is something that occurs in a well-defined range (X +/- Y, where Y is small compared to X), then a histogram might be a more useful display. So instead of displaying a constantly changing X, you count the number of times each X occurs and display the counts vs. X in a histogram. Even then, you don't update the replotting of that more than a few time a second at most.

    and now i'm worried that no one will answer after such a lengthy explanation
    There is that problem. Sometimes less is more - distilling the problem down to a few lines usually gets a better response than pages of detailed explanation, all in lowercase...

    Besides that, you are dealing with a hardware / software configuration that probably very few people who follow this forum have any experience with. I don't. I am just suggesting areas to look, based on my long experience trying to track down these kinds of bugs.
    Last edited by d_stranz; 15th September 2018 at 17:19.
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

Similar Threads

  1. QNetworkReply - how to set bandwidth limit properly ?
    By #Dragon in forum Qt Programming
    Replies: 1
    Last Post: 19th August 2015, 18:24
  2. bandwidth
    By Ali Reza in forum General Programming
    Replies: 6
    Last Post: 14th November 2012, 16:11
  3. Moniter the internet bandwidth
    By gaurp in forum Newbie
    Replies: 1
    Last Post: 5th June 2012, 15:56
  4. Distribute bandwidth with QTcpSocket
    By danc81 in forum Qt Programming
    Replies: 2
    Last Post: 19th October 2009, 13:17
  5. QNetworkAccessManager bandwidth throttling
    By bluefly in forum Qt Programming
    Replies: 6
    Last Post: 14th April 2009, 20:49

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.