Results 1 to 11 of 11

Thread: Serializing a QHash leads to SEGFault

  1. #1
    Join Date
    Jan 2015
    Posts
    5
    Thanks
    1
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11

    Question Serializing a QHash leads to SEGFault

    I have a problem with a program I am working on where serializing a QHash gets me a segfault. As the project is too much code to show, I isolated what created the problem and could reproduce the segfault.

    Does anyone have a clue what I am doing wrong here? I am sorry that it's a lot of code, it's not very complex though (I am newbie at both C++ and Qt).
    It runs until the statement "data2 >> datum2". After that I get a segfault.

    Qt Code:
    1. 0000 Program start.
    2. 0001 Datum1 written to buffer.
    3. 0002 Buffer about to open for reading.
    4. 0003 data2 about to initialize with buffer.
    5. 0004 data2 initialized with buffer.
    6. 0005 data2 about to read buffer into datum2.
    7. Segmentation fault (core dumped)
    To copy to clipboard, switch view to plain text mode 

    This code is main.cpp.

    Qt Code:
    1. #include <memory>
    2. #include <iostream>
    3. #include <exception>
    4.  
    5. #include <QCoreApplication>
    6. #include <QBuffer>
    7. #include <QIODevice>
    8.  
    9. #include "datum.h"
    10.  
    11. int main(int argc, char *argv[])
    12. {
    13. QCoreApplication a(argc, argv);
    14.  
    15. std::cout << "0000 Program start." << std::endl;
    16.  
    17. std::shared_ptr<Datum> datum = std::make_shared<Datum>();
    18. std::shared_ptr<Thing> thing = std::make_shared<Thing>();
    19.  
    20. thing->name("name of thing");
    21. datum->add(thing);
    22.  
    23.  
    24. QBuffer buffer;
    25.  
    26. // Write datum into buffer
    27. buffer.open(QIODevice::WriteOnly);
    28. QDataStream data(&buffer);
    29. data << datum;
    30. buffer.close();
    31. std::cout << "0001 Datum1 written to buffer." << std::endl;
    32.  
    33. // Read datum2 from buffer
    34. std::shared_ptr<Datum> datum2;
    35. std::cout << "0002 Buffer about to open for reading." << std::endl;
    36. buffer.open(QIODevice::ReadOnly);
    37. std::cout << "0003 data2 about to initialize with buffer." << std::endl;
    38. QDataStream data2(&buffer);
    39. std::cout << "0004 data2 initialized with buffer." << std::endl;
    40.  
    41. try {
    42. std::cout << "0005 data2 about to read buffer into datum2." << std::endl;
    43. data2 >> datum2;
    44. std::cout << "0006 data2 just read buffer into datum2." << std::endl;
    45. buffer.close();
    46. std::cout << "0007 Datum2 read from buffer." << std::endl;
    47. } catch (std::exception &e) {
    48. std::cout << "0008 Error during read from buffer: " << e.what() << std::endl;
    49. }
    50.  
    51. std::cout << "0009 Program end." << std::endl;
    52.  
    53. return a.exec();
    54. }
    To copy to clipboard, switch view to plain text mode 

    Following the datum.h and datum.cpp files.

    Qt Code:
    1. #ifndef DATUM_H
    2. #define DATUM_H
    3.  
    4. #include <iostream>
    5. #include <memory>
    6.  
    7. #include <QDataStream>
    8. #include <QHash>
    9.  
    10. class Thing {
    11.  
    12. friend QDataStream &operator<< (QDataStream &ds, std::shared_ptr<const Thing>);
    13. friend QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Thing>);
    14.  
    15. public:
    16.  
    17. QString name() const { return m_name; }
    18. void name(const QString name) { m_name = name; }
    19.  
    20. private:
    21. QString m_name;
    22. };
    23.  
    24. class Datum
    25. {
    26.  
    27. friend QDataStream &operator<< (QDataStream &ds, std::shared_ptr<const Datum>);
    28. friend QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Datum>);
    29.  
    30. public:
    31. Datum();
    32. ~Datum();
    33.  
    34. void add(std::shared_ptr<Thing>);
    35.  
    36. private:
    37. QHash<QString, std::shared_ptr<Thing>> m_list;
    38. };
    39.  
    40. QDataStream &operator<< (QDataStream &ds, std::shared_ptr<const Thing>);
    41. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Thing>);
    42.  
    43. QDataStream &operator<< (QDataStream &ds, std::shared_ptr<const Datum>);
    44. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Datum>);
    45.  
    46. #endif // DATUM_H
    To copy to clipboard, switch view to plain text mode 

    Qt Code:
    1. #include "datum.h"
    2.  
    3. Datum::Datum()
    4. {
    5.  
    6. }
    7.  
    8. Datum::~Datum()
    9. {
    10.  
    11. }
    12.  
    13. void Datum::add(std::shared_ptr<Thing> thing) {
    14. m_list.insert(thing->name(), thing);
    15. }
    16.  
    17.  
    18. QDataStream &operator<< (QDataStream &ds, std::shared_ptr<const Thing> thing) {
    19. ds << thing->m_name;
    20. return ds;
    21. }
    22.  
    23. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Thing> thing) {
    24. ds >> thing->m_name;
    25. return ds;
    26. }
    27.  
    28. QDataStream &operator<< (QDataStream &ds, std::shared_ptr<const Datum> datum) {
    29. ds << datum->m_list;
    30. return ds;
    31. }
    32.  
    33. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Datum> datum) {
    34. ds >> datum->m_list;
    35. return ds;
    36. }
    To copy to clipboard, switch view to plain text mode 
    Last edited by gbonnema; 6th January 2015 at 12:06. Reason: correcting the title

  2. #2
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,418
    Thanks
    37
    Thanked 1,545 Times in 1,495 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Serializing a QHash leads to SEGFault

    At
    Qt Code:
    1. data2 >> datum2;
    To copy to clipboard, switch view to plain text mode 
    you are trying to stream in to the shared pointer equvialent of a null pointer. I.e. there is no instance "in" datum2.

    Cheers,
    _

  3. #3
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    5,138
    Thanks
    294
    Thanked 852 Times in 839 Posts
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: Serializing a QHash leads to SEGFault

    Need to add this:

    Qt Code:
    1. std::shared_ptr<Datum> datum2 = std::make_shared<Datum>();
    To copy to clipboard, switch view to plain text mode 

    before the line referenced by anda_skoa.

  4. #4
    Join Date
    Jan 2015
    Posts
    5
    Thanks
    1
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11

    Question Re: Serializing a QHash leads to SEGFault

    Quote Originally Posted by anda_skoa View Post
    At
    Qt Code:
    1. data2 >> datum2;
    To copy to clipboard, switch view to plain text mode 
    you are trying to stream in to the shared pointer equvialent of a null pointer. I.e. there is no instance "in" datum2.

    Cheers,
    _

    Yes, thanks you are right. Although this was not the case in the original program. After correcting it (I allocated std::make_shared<Datum>() to datum2), I still get an segfault.
    After tracing using cgdb I found that the segfault is in QString::resize. I have attached a screenprint of that session, so you can see exactly what happened.

    My suspicion is, that the shared_ptr in a QHash is the cause. I expected the operator>> to handle this, but I am out of my league where Qt knowledge is concerned (and maybe C++ knowledge too).

    What do you think: is a QHash consisting of a
    Qt Code:
    1. <QString, std::shared_ptr<Thing> >
    To copy to clipboard, switch view to plain text mode 
    a valid definition for serialization?Screenshot from 2015-01-06 22:54:30.jpg

    Sorry, the screenprint as shown is not legible. Let me copy parts of the screenprint in text:

    [CODE}
    Using host libthread_db library "/lib64/libthread_db.so.1".
    0000 Program start.
    0001 Datum1 written to buffer.

    Breakpoint 1, main (argc=1, argv=0x7fffffffdce8) at ../serialize-segfault/main.cpp:56
    Missing separate debuginfos, use: debuginfo-install glib2-2.42.1-1.fc21.x86_64 libgcc-4.9.2-1.fc21.x86_64 libicu-52.1-
    4.fc21.x86_64 libstdc++-4.9.2-1.fc21.x86_64 pcre-8.35-8.fc21.x86_64 zlib-1.2.8-7.fc21.x86_64
    (gdb) n
    (gdb) pr datum2
    Ambiguous command "pr datum2": print, print-object, printf.
    (gdb) print datum2
    $1 = std::shared_ptr (count 1, weak 0) 0x60e760
    (gdb) print datum2->m_list
    $2 = {{d = 0x7ffff7c6d9a0 <QHashData::shared_null>, e = 0x7ffff7c6d9a0 <QHashData::shared_null>}}
    (gdb) print datum2->m_list.keys()
    Cannot evaluate function -- may be inlined
    (gdb) n
    0002 Buffer about to open for reading.
    (gdb) n
    (gdb) n
    0003 data2 about to initialize with buffer.
    (gdb) n
    (gdb) n
    0004 data2 initialized with buffer.
    (gdb) n
    0005 data2 about to read buffer into datum2.
    (gdb) n

    Program received signal SIGSEGV, Segmentation fault.
    QString::resize (this=this@entry=0x0, size=size@entry=13) at tools/qstring.cpp:1585
    (gdb)
    [/CODE]

    I hope this makes things a little more clear.
    Last edited by gbonnema; 6th January 2015 at 22:10.

  5. #5
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    5,138
    Thanks
    294
    Thanked 852 Times in 839 Posts
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: Serializing a QHash leads to SEGFault

    I think QString::resize() is a red herring. Your code has probably corrupted memory somewhere earlier, and it just happens to segfault at that point. If you rearranged or added some code, it would likely break somewhere else.

    I think your problem is here:

    Qt Code:
    1. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Datum> datum)
    2. {
    3. ds >> datum->m_list;
    4. return ds;
    5. }
    To copy to clipboard, switch view to plain text mode 

    You may have fixed the problem with datum2 being NULL, but the std::shared_ptr< Thing > instances that are the values stored for the hash are still NULL. When your serialization hits this:

    Qt Code:
    1. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Thing> thing)
    2. {
    3. ds >> thing->m_name;
    4. return ds;
    5. }
    To copy to clipboard, switch view to plain text mode 

    it blows up because the std::shared_ptr< Thing > is uninitialized and contains a NULL Thing instance. Possibly you can check to see if "thing" is NULL, and if so, call std::make_shared< Thing >() at that point:

    Qt Code:
    1. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Thing> thing)
    2. {
    3. if ( !thing )
    4. thing = std::make_shared< Thing >();
    5. ds >> thing->m_name;
    6. return ds;
    7. }
    To copy to clipboard, switch view to plain text mode 

    You may need to pass in the "thing" argument as a reference, (std::shared_ptr< Thing > & thing) but I don't know the semantics of std::shared_ptr in this use.

    EDIT:

    Actually, on thinking about it, the QString::resize() probably was actually where things went wrong, because the QString it was trying to resize is the one that is contained in the NULL Thing instance - it's trying to resize something that doesn't exist. That was triggered by ds >> thing->m_name, when "thing" is NULL.
    Last edited by d_stranz; 7th January 2015 at 04:19.

  6. The following user says thank you to d_stranz for this useful post:

    gbonnema (7th January 2015)

  7. #6
    Join Date
    Jan 2015
    Posts
    5
    Thanks
    1
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11

    Question Re: Serializing a QHash leads to SEGFault

    Thank you very much! Not only do I now see what went wrong, but also I understand better how to approach this type of problem.
    In summary:

    1. My operator>> for Thing needs updating to cater for nullptr.
    2. The shared_ptr may or may not need to be a reference.

    Thanks again: I appreciate your help!

    EDIT: Just wanted to point to http://stackoverflow.com/questions/3...oostshared-ptr. It contains an excellent discussion of point 2, although to me on first reading it is still not clear whether using a shared_ptr as a reference is a wise thing to do.

    EDIT2: On second thought, I suppose referencing a shared_ptr probably defeats the goal of a shared_ptr which is to do bookkeeping on how many users of a storage area there are and only delete when the last one releases the pointer. If only 2 users are left and one uses a reference, then as soon as the other one releases the pointer the memory area will be deleted. So, probably if a shared_ptr is necessary, one should not want to reference it.

    EDIT3: The last code example you give the decision seems to point in the opposite direction: we *have* to use reference in this case.

    Qt Code:
    1. QDataStream &operator>> (QDataStream &ds, std::shared_ptr<Thing> thing) {
    2. if (thing == nullptr) {
    3. thing = std::make_shared<Thing>();
    4. }
    5. ds >> thing->m_name;
    6. return ds;
    7. }
    To copy to clipboard, switch view to plain text mode 

    What happens here is that a temporary gets created for the param thing because it is by value. Then the resulting address of make_shared is copied into the temporary thing.
    When the function goes out of scope the question is: will the memory area that make_shared created still exist by the time thing gets inserted into the Datum container or will it have been deleted by that time?

    I fear that a reference to the shared_ptr is a must in this case. I don't see any alternative. Any comments?
    Last edited by gbonnema; 7th January 2015 at 09:47.

  8. #7
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,418
    Thanks
    37
    Thanked 1,545 Times in 1,495 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Serializing a QHash leads to SEGFault

    If you look at the documentation, the "read" function/operator always uses a reference for the data element:
    http://doc.qt.io/qt-5/qdatastream.ht...her-qt-classes

    Cheers,
    _

  9. #8
    Join Date
    Jan 2015
    Posts
    5
    Thanks
    1
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11

    Default Re: Serializing a QHash leads to SEGFault

    Quote Originally Posted by anda_skoa View Post
    If you look at the documentation, the "read" function/operator always uses a reference for the data element:
    http://doc.qt.io/qt-5/qdatastream.ht...her-qt-classes

    Cheers,
    _
    Well, actually, I completely get why for a normal object this should always be a reference both in reading and in writing for operator>> and operator<< overrides.
    Also, thanks to your comments, I see that operator>> should always use a reference to shared_ptr as well, because one is modifying memory that one sometimes allocates through this variable. (see my previous comment).

    However, when reading the shared_ptr variable with operator<<, I am not convinced it should be a reference to shared_ptr. The reason is, that if you use a reference, then the shared_ptr will not increment its ref count.
    Therefore during the call to the << function if the other user deletes the pointer (i.e. goes out of scope) .... Wait, this could only happen in a multi threaded environment where the caller runs async from the << function.

    ok, I suspect with the << function you could use reference to prevent copying of the pointer when no concurrency is in play, but using parameter by value should work too. For the reading side of the equation I am not convinced for either case: by reference or by value.
    But, in the face of concurrency, by value might be the safest bet as the ref count gets incremented so the memory area remains valid no matter what.

    hmm, conclusion:

    1. For operator>> (i.e. write) definitely reference as you are potientially modifying the address for a nullptr
    2. For operator<< (i.e. read) I do not know which is best in the case of shared_ptr.

    Thanks for your comments. It has raised some pretty interesting points.

  10. #9
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,418
    Thanks
    37
    Thanked 1,545 Times in 1,495 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Serializing a QHash leads to SEGFault

    Quote Originally Posted by gbonnema View Post
    But, in the face of concurrency, by value might be the safest bet as the ref count gets incremented so the memory area remains valid no matter what.
    The caller of the operator still has a valid reference, the counter cannot drop to 0.

    Cheers,
    _

  11. #10
    Join Date
    Jan 2015
    Posts
    5
    Thanks
    1
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11

    Default Re: Serializing a QHash leads to SEGFault

    Quote Originally Posted by anda_skoa View Post
    The caller of the operator still has a valid reference, the counter cannot drop to 0.

    Cheers,
    _
    True, but only if called synchronously. In a chain of calls where only the first caller has a shared_ptr and the called functions all have references to shared_ptr, imagine the first function being called asynchronously.
    In that case, if the caller ends before the chain of calls ends, then they all refer to memory that was deleted when the caller ended.

    Furthermore the writer of the operator override can not know how it is being called. So, unless you control all callers, using a reference to shared_ptr, holds a risk of dangling pointer.
    If you are writing a class that could be used in multiple applications, you can not assume anything.

    It is this case that I meant. Passing by value is safest because it prevents the memory from being deleted. Although you could leave the concurrent case as "behaviour undefined".

  12. #11
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,418
    Thanks
    37
    Thanked 1,545 Times in 1,495 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: Serializing a QHash leads to SEGFault

    What kind of asynchronous call mechanism do you have in mind that would not trigger a copy but work with a non-shared pointer reference going out of scope?

    Cheers,
    _

Similar Threads

  1. Serializing QHash to a QByteArray
    By PeterThePuter in forum Qt Programming
    Replies: 3
    Last Post: 17th December 2010, 23:19
  2. application never quits
    By harmodrew in forum Newbie
    Replies: 2
    Last Post: 6th August 2010, 00:56
  3. Qt 4 quits application [solved]
    By osiris81 in forum Qt Programming
    Replies: 1
    Last Post: 3rd December 2008, 14:07
  4. Can QHash::capacity() be smaller than QHash::size()?!?
    By iw2nhl in forum Qt Programming
    Replies: 2
    Last Post: 24th August 2007, 01:17
  5. hide() quits?
    By Morea in forum Newbie
    Replies: 13
    Last Post: 25th February 2006, 19:40

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.