Results 1 to 8 of 8

Thread: Can't create QSharedMemory on macOS Catalina

  1. #1
    Join Date
    May 2020
    Posts
    4
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X

    Default Can't create QSharedMemory on macOS Catalina

    Hello.

    I'm trying to create a shared memory segment between two processes.
    Calling QSharedMemory::create() returns false with QSharedMemory::errorString() set to "QSharedMemory::create: out of resources".
    strerror(errno) returns "No space left on device" (I have 150 GB available), sometimes it returns "Too many open files".

    Same code works fine on Windows system. Don't know what to do, hope for your help.

  2. #2
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    4,476
    Thanks
    252
    Thanked 709 Times in 699 Posts
    Qt products
    Qt5
    Platforms
    Windows Android

    Default Re: Can't create QSharedMemory on macOS Catalina

    Let's see, I am trying to peer into my crystal ball at your invisible code, but I am not having much luck.

    So if you are getting an error that indicates there are no resources available, this would seem to indicate that maybe you have opened too many files or other things that use a resource handle, but haven't closed or released those handles after you are done with them. Is every fopen() matched with an fclose() (or the equivalent if you are writing a pure Qt solution)? If a FILE pointer goes out of scope, it doesn't close the file in the process, the file (and the resource handle it uses) stay open until your program exits.
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

  3. #3
    Join Date
    May 2020
    Posts
    4
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X

    Default Re: Can't create QSharedMemory on macOS Catalina

    Sorry, I didn't posted the code because I don't think it's something code related, but maybe (hopefully) I'm wrong. Here's the minimal code reproducing the problem:

    Qt Code:
    1. #include <QCoreApplication>
    2. #include <QSharedMemory>
    3. #include <QDebug>
    4.  
    5. int main(int argc, char *argv[]) {
    6. QCoreApplication a(argc, argv);
    7. auto memory = new QSharedMemory;
    8. memory->setKey("foo");
    9. if(!memory->create(8, QSharedMemory::AccessMode::ReadWrite)) {
    10. qDebug() << memory->errorString();
    11. qDebug() << strerror(errno);
    12. }
    13. return a.exec();
    14. }
    To copy to clipboard, switch view to plain text mode 
    The first line of the output is always "QSharedMemory::create: out of resources"
    The second line is funny because at first it was "No space left on device", but after a couple of debug builds it has changed to "Undefined error: 0". Also, before today I saw it as "Too many open files". Don't know what it depends on.

  4. #4
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    4,476
    Thanks
    252
    Thanked 709 Times in 699 Posts
    Qt products
    Qt5
    Platforms
    Windows Android

    Default Re: Can't create QSharedMemory on macOS Catalina

    For one thing, shared memory does not use an HDD, so it doesn't matter if you have 150 GB of disk space. It is memory-based.

    Second, shared memory is allocated by the OS and is shared among processes. In your code, you allocate a new QSharedMemory instance (using new), but you never delete it when the program exits. If the shared memory segment named "foo" is still around, then you can't create a new one. If you are running multiple instances of your program, then the first time through it may be able to "create()" the segment, but the second running instance of the app can't because it already exists. You can "attach()" to it.

    "Undefined error: 0"
    The QSharedMemory docs say that an error code of 0 means "no error".
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

  5. #5
    Join Date
    May 2020
    Posts
    4
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X

    Default Re: Can't create QSharedMemory on macOS Catalina

    Since heap-allocated memory is freed on program exit there's no need for explicit free(). Anyway, the allocation can be static, as QSharedMemory memory() with the same result. There are no other instances of this process, calling attach() returns "QSharedMemory::handle:: UNIX key file doesn't exist". "Undefined error: 0" is a result of strerror(errno), not memory->errorString(). Latter returns "QSharedMemory::create: out of resources" which has a code 7.

  6. #6
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    4,476
    Thanks
    252
    Thanked 709 Times in 699 Posts
    Qt products
    Qt5
    Platforms
    Windows Android

    Default Re: Can't create QSharedMemory on macOS Catalina

    Since heap-allocated memory is freed on program exit there's no need for explicit free().
    That isn't the case for shared memory if there are other processes attached to it. It is probably reference counted. As the documentation says, unless the QSharedMemory destructor is called (decrementing the reference count), you cannot be guaranteed that the OS will release the memory. Heap that is freed when a process exits is simply freed without calling any destructors. That's why you can have a program with a memory leak and have all the leaked memory freed on exit.

    i don't know what the problem is with your program. Maybe your choice of "foo" as the key? Pretty common thing for a programmer to use as a name for testing purposes.
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

  7. The following user says thank you to d_stranz for this useful post:

    swante (25th May 2020)

  8. #7
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,620
    Thanks
    13
    Thanked 1,592 Times in 1,520 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: Can't create QSharedMemory on macOS Catalina

    According to the Qt docs, for UNIX (which I am assuming will include OS X):
    Unix: QSharedMemory "owns" the shared memory segment. When the last thread or process that has an instance of QSharedMemory attached to a particular shared memory segment detaches from the segment by destroying its instance of QSharedMemory, the Unix kernel release the shared memory segment. But if that last thread or process crashes without running the QSharedMemory destructor, the shared memory segment survives the crash.
    If I execute your code on my Linux box you see the shared segment created:
    Qt Code:
    1. $ ipcs -m
    2.  
    3. ------ Shared Memory Segments --------
    4. key shmid owner perms bytes nattch status
    5. ...
    6. 0x5102002c 57 chrisw 600 8 1
    To copy to clipboard, switch view to plain text mode 
    If I then exit with Control-C (because there's no other way) the program is "crashed":
    Qt Code:
    1. $ ipcs -m
    2.  
    3. ------ Shared Memory Segments --------
    4. key shmid owner perms bytes nattch status
    5. ...
    6. 0x5102002c 57 chrisw 600 8 0
    To copy to clipboard, switch view to plain text mode 
    you see that the number of attached processes goes to zero but the shared memory persists. The OS will free memory allocated on the heap when terminated, i.e. the space occupied by the QSharedMemory controlling object, but not the resource it controlled because the destructor is not called.

    A second attempt to run your code then fails:
    Qt Code:
    1. $ ./test
    2. "QSharedMemory::create: already exists"
    3. No such file or directory
    To copy to clipboard, switch view to plain text mode 

    If I modify your code so the program terminates gracefully and ensures QSharedMemory object is destroyed:
    Qt Code:
    1. #include <QCoreApplication>
    2. #include <QSharedMemory>
    3. #include <QDebug>
    4. #include <QTimer>
    5.  
    6. int main(int argc, char *argv[]) {
    7. QCoreApplication a(argc, argv);
    8. auto memory = new QSharedMemory(&a); // <<< parented to ensure destruction
    9. memory->setKey("foo");
    10. if(!memory->create(8, QSharedMemory::AccessMode::ReadWrite)) {
    11. qDebug() << memory->errorString();
    12. qDebug() << strerror(errno);
    13. }
    14. QTimer::singleShot(5000, &a, &QCoreApplication::quit); // <<< quit in 5 sec
    15. return a.exec();
    16. }
    To copy to clipboard, switch view to plain text mode 
    Then after deleting the persistent shared memory block with ipcrm, the program will run to a graceful completion and deallocate the shared memory object.


    Qt Code:
    1. $ ipcs -m
    2.  
    3. ------ Shared Memory Segments --------
    4. key shmid owner perms bytes nattch status
    5. 0x00000000 32773 chrisw 600 20480 2 dest
    6. 0x00000000 32774 chrisw 600 20480 2 dest
    7. ...
    8.  
    9. chrisw@newton:/tmp/tt$ ./test &
    10. [1] 5023
    11. chrisw@newton:/tmp/tt$ ipcs -m
    12.  
    13. ------ Shared Memory Segments --------
    14. key shmid owner perms bytes nattch status
    15. 0x00000000 32773 chrisw 600 20480 2 dest
    16. 0x00000000 32774 chrisw 600 20480 2 dest
    17. 0x5102002c 32778 chrisw 600 8 1
    18. ...
    19.  
    20. # program terminates after 5 seconds
    21. chrisw@newton:/tmp/tt$ ipcs -m
    22.  
    23. ------ Shared Memory Segments --------
    24. key shmid owner perms bytes nattch status
    25. 0x00000000 32773 chrisw 600 20480 2 dest
    26. 0x00000000 32774 chrisw 600 20480 2 dest
    27. ...
    To copy to clipboard, switch view to plain text mode 

    This approach will be more resilient:
    Qt Code:
    1. #include <QCoreApplication>
    2. #include <QSharedMemory>
    3. #include <QDebug>
    4. #include <QTimer>
    5.  
    6. int main(int argc, char *argv[]) {
    7. QCoreApplication a(argc, argv);
    8. QSharedMemory memory("foo"); // scope ensures destructor is called on normal exit
    9. // attach to shared mem foo if it exists
    10. if(!memory.attach(QSharedMemory::AccessMode::ReadWrite)) {
    11. // does not already exist so create it
    12. if(!memory.create(8, QSharedMemory::AccessMode::ReadWrite)) {
    13. qDebug() << memory.errorString();
    14. qDebug() << strerror(errno);
    15. }
    16. }
    17. QTimer::singleShot(5000, &a, &QCoreApplication::quit); // <<< quit in 5 sec
    18. return a.exec();
    19. }
    To copy to clipboard, switch view to plain text mode 
    You can run ten of these simultaneously. The first to start will create, the others attach, and the last one to terminate gracefully will remove the shared memory.

  9. The following 2 users say thank you to ChrisW67 for this useful post:

    d_stranz (24th May 2020), swante (25th May 2020)

  10. #8
    Join Date
    May 2020
    Posts
    4
    Thanks
    2
    Qt products
    Qt5
    Platforms
    MacOS X

    Default Re: Can't create QSharedMemory on macOS Catalina

    Yep, you were right that created memory segments were outliving the process, because I never exited it gracefully, only by terminating the debugger/killing the process. I've tried to call detach() on QSharedMemory object before a.exec() and it also worked.

    Also, my simplified example isn't correct because in the real code I create 10 memory segments and this is the true reason for "QSharedMemory::create: out of resources" and "Too many open files". macOS by default has a limit of 8 shared memory segments (and 4MB total) per user, that's why attempts to create last two segments always returned these errors. Fortunately, these limits can be changed as described here: http://www.spy-hill.com/help/apple/SharedMemory.html

    So we can either change the limits per-machine or just don't attach to more than 8 segments simultaneously.

    Thanks for the help!

Similar Threads

  1. Compatibility of El Capitan Qt-based apps on new catalina
    By ghorwin in forum Installation and Deployment
    Replies: 0
    Last Post: 17th October 2019, 14:33
  2. Qt Creator Unable to create/open qmake project under MacOS X
    By googie in forum Qt Tools
    Replies: 3
    Last Post: 27th August 2014, 06:45
  3. Replies: 1
    Last Post: 8th May 2013, 11:09
  4. Create new QsharedMemory after crash
    By dashesy in forum Qt Programming
    Replies: 1
    Last Post: 30th June 2012, 07:01
  5. Create Qt Application Bundle in MacOs
    By joy in forum Installation and Deployment
    Replies: 2
    Last Post: 25th February 2008, 18:06

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.