Results 1 to 14 of 14

Thread: ERROR with QSemaphores on a real time display application

  1. #1
    Join Date
    Apr 2009
    Posts
    25
    Thanks
    4
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: ERROR with QSemaphores on a real time display application

    Hey guys!

    I have some major issues on the synchronisation of two threads. The problem is common producer - consumer but the speed of the two threads is variable and depends on an external device.

    The producer is reading constantly datagrams from a UDPServer and after some validations it stores them to a boost::circular_buffer
    Qt Code:
    1. producer ( QSemphore* freeBytes, QSemaphore * usedBytes, *BUFFER)
    2. while(true)
    3. {
    4. if (udpSocket.waitForReadyRead(10) && !paused)
    5. {
    6.  
    7. while (udpSocket.hasPendingDatagrams())
    8. {
    9. datagramSize = udpSocket.pendingDatagramSize();
    10. datagram.resize(datagramSize);
    11. udpSocket.readDatagram(datagram.data(), datagramSize);
    12.  
    13. hexDatagram.resize(datagramSize*2);
    14.  
    15. hexDatagramSize = hexDatagram.size();
    16. hexDatagram = datagram.toHex();
    17. blocksAcquired = datagramSize/1280;
    18.  
    19. READING_CHECKING();
    20. if( OK_FILL_A_TEMP_BUFFER())
    21. if(CERTAIN_AMOUNT_OF_INFORMATION)
    22. {
    23. freeBytes->acquire(1); // Pointer to a QSemaphore on the main thread
    24. for (j=0; j<320; j++)
    25. {
    26. iBuffer->push_back(tempBuffer[j]);
    27. }
    28. tempIndex=0;
    29. usedBytes->release(1);// Pointer to a QSemaphore on the main thread
    30. }
    31. }
    32. datagram.clear();
    33. hexDatagram.clear();
    34. }
    35. }
    To copy to clipboard, switch view to plain text mode 

    The consumer checks the usedBytes->available() in order to exceed a certain level and then start to process them.

    Qt Code:
    1. while(true)
    2. {
    3. availablePacks = usedBytes->available();
    4.  
    5. if(availablePacks>20)
    6. {
    7. for(i=0; i<availablePacks-5; i++)
    8. {
    9. usedBytes->acquire(1);
    10.  
    11. for(j=0; j<80; j++)
    12. {
    13. cha[j] = iBuffer->front();
    14. iBuffer->pop_front();
    15.  
    16. chb[j] = iBuffer->front();
    17. iBuffer->pop_front();
    18.  
    19. chc[j] = iBuffer->front();
    20. iBuffer->pop_front();
    21.  
    22. chd[j] = iBuffer->front();
    23. iBuffer->pop_front();
    24.  
    25. }
    26. //printf("Con: %d \n",iBuffer->size());
    27. freeBytes->release(1);
    28.  
    29. for(j=0; j<80; j++)
    30. {
    31. PROCCESSING();
    32.  
    33. }
    34.  
    35. }
    36. }
    To copy to clipboard, switch view to plain text mode 

    But all the time the consumer overruns the produces leading to a SIGSEVG error.


    Added after 12 minutes:


    I should also note that the BUFFER is 200 times larger than the transfering packages. and that I have only 100 Semaphores in order to ensure somehow that the threads won't overlap. Even if finally they do so.
    Last edited by ^Nisok^; 9th January 2012 at 22:49.

  2. #2
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,013 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: ERROR with QSemaphores on a real time display application

    I would assume some of your magic numbers calculations are incorrect. I don't see the point of "if(availablePacks>20)" as well.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  3. #3
    Join Date
    Apr 2009
    Posts
    25
    Thanks
    4
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: ERROR with QSemaphores on a real time display application

    Thank you for your reply.
    What do you mean magic numbers??

    Because I don't need to display the data as fast as they come, I wait to have gathered several packages. But this argument can be removed....

  4. #4
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,013 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: ERROR with QSemaphores on a real time display application

    Quote Originally Posted by ^Nisok^ View Post
    What do you mean magic numbers??
    5, 20, 80, 320, 1280
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  5. #5
    Join Date
    Apr 2009
    Posts
    25
    Thanks
    4
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: ERROR with QSemaphores on a real time display application

    I am afraid that the numbers are correct. The application runs even for several hours before the crash.

    Any other idea??

  6. #6
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,013 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: ERROR with QSemaphores on a real time display application

    I don't see how the amount of time the application runs for is related to any of those numbers. The application crashes when there are conditions met which cause a buffer underrun. The only time related aspect is that the overall probability of the fault manifesting itself increases as the time passes.

    For instance you have a statement: blocksAcquired = datagramSize/1280;

    Since datagramSize is an integer, I assume blocksAcquired is also an integer. If datagramSize is not constant, it might sometimes have a size that cannot be divided by 1280. When this happens, you leak blocks (since if datagramSize = 1281 then blocksAcquired = 1, whereas it is likely you'd expect it to be 2). You have to check such border cases. If your application crashes, it is not a direct fault of a buffer underrun, since the semaphore should block an attempt to read from an empty buffer. So first check where exactly the crash occurs (on reading from the buffer or from processing the data) using a debugger. And simplify your code --- if you don't need to strange conditions then throw them away.

    As a side note, I would advise you to manipulate the semaphores in terms of bytes instead of blocks, if you later read or write bytes and not whole blocks. You'll have less places where you can make a mistake in your code.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  7. #7
    Join Date
    Apr 2009
    Posts
    25
    Thanks
    4
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: ERROR with QSemaphores on a real time display application

    I followed your suggestions and I can say that for the past 2 hours it hasn't crash.
    But still I am a bit nervous ...

    Because till now all the crashes took place randomly.
    The debugger wasn't helpfull at all.

    Thank you for your help !!

    I will update the thread if necessary...

  8. #8
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,013 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: ERROR with QSemaphores on a real time display application

    If it crashes, come back with a debugger backtrace
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  9. #9
    Join Date
    Apr 2009
    Posts
    25
    Thanks
    4
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: ERROR with QSemaphores on a real time display application

    One important note that I would like to submit for all the other developers which will read this thread is that the boost::circular_buffer.push_back() check the new size of the buffer using the size().
    The size of the buffer is stored to a particular place in the memory.
    So if a consumer thread checks periodically the size of the buffer you have to use a mutex to lock the concurrent access.

  10. #10
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,013 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: ERROR with QSemaphores on a real time display application

    The consumer should never check the size of the buffer. That's what you have a semaphore for, right? And the value will be strictly for information purposes only anyway as it can change immediately after being read.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  11. #11
    Join Date
    Apr 2009
    Posts
    25
    Thanks
    4
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: ERROR with QSemaphores on a real time display application

    yep... after several hours it crashed again here is the debugger information:

    it crashed on this point
    Qt Code:
    1. iBuffer->pop_front();
    To copy to clipboard, switch view to plain text mode 

    on the pop_front() at the destroy_item
    Qt Code:
    1. void pop_front() {
    2. BOOST_CB_ASSERT(!empty()); // check for empty buffer (front element not available)
    3. destroy_item(m_first);
    4. increment(m_first);
    5. --m_size;
    6. }
    To copy to clipboard, switch view to plain text mode 

    at invalidate_iterators
    Qt Code:
    1. void destroy_item(pointer p) {
    2. m_alloc.destroy(p);
    3. #if BOOST_CB_ENABLE_DEBUG
    4. invalidate_iterators(iterator(this, p));
    5. std::memset(p, cb_details::UNINITIALIZED, sizeof(value_type));
    6. #endif
    7. }
    To copy to clipboard, switch view to plain text mode 

    at remove(p,previous)
    Qt Code:
    1. void invalidate_iterators(const Iterator& it) {
    2. const debug_iterator_base* previous = 0;
    3. for (const debug_iterator_base* p = m_iterators; p != 0; p = p->next()) {
    4. if (((Iterator*)p)->m_it == it.m_it) {
    5. p->invalidate();
    6. remove(p, previous);
    7. continue;
    8. }
    9. previous = p;
    10. }
    11. }
    To copy to clipboard, switch view to plain text mode 

    at m_iterators = m_iterators->next();
    Qt Code:
    1. //! Remove the current iterator from the iterator chain.
    2. void remove(const debug_iterator_base* current,
    3. const debug_iterator_base* previous) const {
    4. if (previous == 0)
    5. m_iterators = m_iterators->next();
    6. else
    7. previous->set_next(current->next());
    8. }
    9. };
    To copy to clipboard, switch view to plain text mode 

    Qt Code:
    1. inline const debug_iterator_base* debug_iterator_base::next() const { return m_next; }
    To copy to clipboard, switch view to plain text mode 

  12. #12
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,013 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: ERROR with QSemaphores on a real time display application

    Are you protecting the buffer itself from concurrent access? Or is it thread-safe already? Because if not then it might be happening that you are modifying the buffer from two threads at once causing it to lose coherence. Surround the calls to push_back and pop_front with QMutex::lock() and QMutex::unlock() and all should be fine.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  13. #13
    Join Date
    Apr 2009
    Posts
    25
    Thanks
    4
    Qt products
    Qt4
    Platforms
    Unix/X11

    Default Re: ERROR with QSemaphores on a real time display application

    Its not a thread safe buffer.

    If I put mutexes (which I had) what is the purpose of the semaphores?

  14. #14
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,364
    Thanks
    3
    Thanked 5,013 Times in 4,791 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: ERROR with QSemaphores on a real time display application

    To prevent buffer underruns/overruns and get rid of the producer and consumer constantly checking whether there is work available for them.

    The semaphore doesn't grant exclusive access. It makes sure no more than N threads can access the section (where N is the value of the semaphore).
    Last edited by wysota; 11th January 2012 at 15:26.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


Similar Threads

  1. real time data display
    By hammer256 in forum Qt Programming
    Replies: 13
    Last Post: 25th March 2013, 17:47
  2. Real time QT application?
    By marc2050 in forum Newbie
    Replies: 1
    Last Post: 8th June 2011, 07:08
  3. Replies: 1
    Last Post: 23rd September 2010, 21:16
  4. Replies: 10
    Last Post: 21st July 2009, 15:22
  5. Display the camera frame in real time
    By alex_lue in forum Qt Programming
    Replies: 8
    Last Post: 27th July 2007, 11:31

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.