Can someone explain me this phenomenon with QByteArray(struct,memcpy...)?

[donglebob]

Hi,

I have the following struct, where I need a dynamic char array in it.
I allocate memory for the dynamic array, fill the struct and put the whole thing into a QByteArray.
This QByteArray will be used later to get back this struct and access the struct content.
While converting to QByteArray and back I detected something what confused me.
Maybe you can help me.

struct test{
    int value_a;
    unsigned short value_b;
    char *c;
};

struct test{
    int value_a;
    unsigned short value_b;
    char *c;
};

To copy to clipboard, switch view to plain text mode 

test *ptr = new test;    //creating a new instance
 
ptr->value_a = 5;       //filling the struct instance -----begin
ptr->value_b = 12;
 
ptr->c = new char[2 * sizeof(char)];
ptr->c[0] = 'A';
ptr->c[1] = 'B';         //filling ----------------------------end
 
 
 
char *p = (char*)ptr;  //need to cast it to char* to make a QByteArray
 
QByteArray bytearray(p, 4+ 2 + 2 * sizeof(char));  // 4bytes (integer) + 2bytes (unsigned short) + 2bytes (char array with 'A' and 'B')
 
 
 
test *xy = (test*)bytearray.data();   // now getting back the data from QbyteArray
int p = xy->a;
unsigned short q = xy->b;
 
char r[2];
memcpy(&r,xy->c,2);    // copy the original array with 'A' and 'B' to r[2]

test *ptr = new test;    //creating a new instance

ptr->value_a = 5;       //filling the struct instance -----begin
ptr->value_b = 12;

ptr->c = new char[2 * sizeof(char)];
ptr->c[0] = 'A';
ptr->c[1] = 'B';         //filling ----------------------------end



char *p = (char*)ptr;  //need to cast it to char* to make a QByteArray

QByteArray bytearray(p, 4+ 2 + 2 * sizeof(char));  // 4bytes (integer) + 2bytes (unsigned short) + 2bytes (char array with 'A' and 'B')



test *xy = (test*)bytearray.data();   // now getting back the data from QbyteArray
int p = xy->a;
unsigned short q = xy->b;

char r[2];
memcpy(&r,xy->c,2);    // copy the original array with 'A' and 'B' to r[2]

To copy to clipboard, switch view to plain text mode 

When I try to access here xy->c (the array), I get an invalid pointer.
Can't access it, so memcpy fails.

So now the confusing thing:
When I increase the dynamic array size to 3 like this:

ptr->c = new char[3 * sizeof(char)];
ptr->c[0] = 'A';
ptr->c[1] = 'B';  
ptr->c[2] = 'C';

ptr->c = new char[3 * sizeof(char)];
ptr->c[0] = 'A';
ptr->c[1] = 'B';  
ptr->c[2] = 'C';

To copy to clipboard, switch view to plain text mode 

And also change other places from "2 * sizeof(char)" to "3 * sizeof(char)"


It WORKS!!!
But why not with an array of 2 bytes?

[donglebob]

I think i solved it...

Qbytearray copied just my pointer not the content.

I thought that it makes a deep copy???

How can I do the above thing correct?

[aamer4yu]

May be QByteArray::fromRawData might be of some help to u

[caduel]

a) how could it make a deep copy? it can hardly know what structure hides behind that char*?
b) even with the bitwise copy: as the pointer is copied (and thus stil points at the same spot in memory) the string behind (your xy->c) should be accessible. What error do you get?
c) because of alignment issues you should NOT calculate sizes like that: just use sizeof(test)
d) Is there a specific reason for not using a std::string or some real container instead of that raw pointer? (A good reason would be: I have to interface that to C.)

HTH

[donglebob]

Hi,

a)
I give it the whole size of bytes to be copied from the source. So I thought it could make a "blind-copy".

OR

Behind char* is just an array of chars. I could put a "\0" at the end so it could know where to end the bit-wise copy of the content.

b) The content is accessable when I give QByteArray (Ctor) the correct size to be copied.
(+4byte for the char *)
Before I knew that it copied just the pointer I gave it 2 bytes, because the content of the array was just 'A' and 'B' => 2Bytes.

c)
alignment issues: If you mean what I mean:
I solved it with "pragma pack". So , this is not a problem.
The whole structure's size is known..there could not be a an extra-bit filled up by the compiler.

d)
The whole structure content must be send over serial port.
I need to use simple structure elements because the other side of the network just knows basic data-types.
int,char,ushort,,...

The QByteArray outside is not a MUST. I use it for internal stuff.

Is there another of doing this?

[caduel]

What I meant was: if you cast the test* to a char*, QByteArray can not know that this char* actually is a test* and contains a member c that you would like to have deep-copied...

Anyway:
Neither a std::string nor a char* can be sent "as is" over network. For both what you want is to send the actual string data. The receiver won't be happy with a pointer value from your machine (or process even).

You have to choices here:
1) use a fixed size buffer in test (i.e. something like char c[100]); that might be ok if you can guarantee the string to never exceed that
2) define a protocol for sending an arbitratry string. you might say: from here on is my string and it ends on the first 0 byte. another common approach is to first send the length and then the data.

For sending over a QIODeviceyou can either use a QByteArrayor just use QIODevice::write() with the char* interface. No real difference. The real issue about sending a string won't go away whatever you choose.

HTH

PS: Even if you use (compiler/platform dependent stuff like) #pragma, the code will break if you add further members to test. Therefore it is a good idea to use sizeof(test) anyway.

[donglebob]

Anyway:
Neither a std::string nor a char* can be sent "as is" over network. For both what you want is to send the actual string data. The receiver won't be happy with a pointer value from your machine (or process even).

Right
Thats what I wanted to do...but the mistake was that it copied the ptr .

1) use a fixed size buffer in test (i.e. something like char c[100]); that might be ok if you can guarantee the string to never exceed that

I know, this would be the easiest way. I wanted to make it a little bit flexible.
The user could enter 50 bytes or 150 bytes. That was the reason why I wanted to make a dynamic char array.

2) define a protocol for sending an arbitratry string. you might say: from here on is my string and it ends on the first 0 byte. another common approach is to first send the length and then the data.

I have already defined a protocol.
These structs are known on both sides.
The struct includes at the beginning the whole struct size in bytes. So its not a problem to find the beginning and the end of the data.
The main problem is to use flexible-sized data (dynamic char-array).

For sending over a QIODeviceyou can either use a QByteArrayor just use QIODevice::write() with the char* interface. No real difference. The real issue about sending a string won't go away whatever you choose

I use the WinAPI function WriteFile to send the byte-array out of the interface.

[donglebob]

Thanks for your help.
I decided to build an own semi-automatic "QByteBuilder-Function", which goes through the dynamic array and copies the values...

[akmiyoshi]

(1) sizeof(test) would be 12 bytes
= (sizeof(int) + sizeof(unsigned short) + sizeof(_padding_) + sizeof(char *));
Please note that the size of struct test would not change even if you change allocation size (array size) of
prt->c.
Deep copy means that ((uchar *)ptr != (uchar *)xy).
However, ((uchar *)ptr->c == (uchar *)xy->c), i.e. ptr->c would not be copied.

struct test{
    int value_a;
    unsigned short value_b;
 /* char _padding_[2]; */ //automatically added by compiler
    char *c;
};

struct test{
    int value_a;
    unsigned short value_b;
 /* char _padding_[2]; */ //automatically added by compiler
    char *c;
};

To copy to clipboard, switch view to plain text mode 

(2) The code below works:

test *ptr = new test;   //creating a new instance
 
    ptr->value_a = 5;       //filling the struct instance -----begin
    ptr->value_b = 12;
 
    ptr->c = new char[2 * sizeof(char)];
    ptr->c[0] = 'A';
    ptr->c[1] = 'B';        //filling ----------------------------end
 
    char *p = (char*)ptr;  //need to cast it to char* to make a QByteArray
 
    qDebug() << "[sizeof(test)]" << sizeof(test);
    QByteArray bytearray(p, sizeof(test));  // 12 bytes
 
    test *xy = (test*)bytearray.data();   // now getting back the data from QbyteArray
    int A = xy->value_a;
    qDebug() << "[A]" << A;
    unsigned short B = xy->value_b;
    qDebug() << "[B]" << B;
 
    char C[2];
    memcpy(&C,xy->c,2);    // copy the original array with 'A' and 'B' to r[2]
 
    qDebug() << C[0] << C[1];

test *ptr = new test;   //creating a new instance

    ptr->value_a = 5;       //filling the struct instance -----begin
    ptr->value_b = 12;

    ptr->c = new char[2 * sizeof(char)];
    ptr->c[0] = 'A';
    ptr->c[1] = 'B';        //filling ----------------------------end

    char *p = (char*)ptr;  //need to cast it to char* to make a QByteArray

    qDebug() << "[sizeof(test)]" << sizeof(test);
    QByteArray bytearray(p, sizeof(test));  // 12 bytes

    test *xy = (test*)bytearray.data();   // now getting back the data from QbyteArray
    int A = xy->value_a;
    qDebug() << "[A]" << A;
    unsigned short B = xy->value_b;
    qDebug() << "[B]" << B;

    char C[2];
    memcpy(&C,xy->c,2);    // copy the original array with 'A' and 'B' to r[2]

    qDebug() << C[0] << C[1];

To copy to clipboard, switch view to plain text mode 

"ptr=0x09be88a8" 
"ptr->c=0x09bd3638" 
[sizeof(test)] 12 
"xy=0x09bd3450" 
"xy->c=0x09bd3638" 
[A] 5 
[B] 12 
[C] A B

"ptr=0x09be88a8" 
"ptr->c=0x09bd3638" 
[sizeof(test)] 12 
"xy=0x09bd3450" 
"xy->c=0x09bd3638" 
[A] 5 
[B] 12 
[C] A B

To copy to clipboard, switch view to plain text mode 

[akmiyoshi]

#include <QtCore>
 
struct test{
    int value_a;
    unsigned short value_b;
    QByteArray c;
};
 
int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    QString P;
    test TA;
    TA.value_a = 5;
    TA.value_b = 12;
    TA.c = "AB";
    test TB = TA;
    qDebug() << QString("(1)TA.c.constData()=0x%1").arg((uint)TA.c.constData(), 8, 16, QChar('0'));
    qDebug() << QString("(2)TB.c.constData()=0x%1").arg((uint)TB.c.constData(), 8, 16, QChar('0'));
    TB.c = "CD";
    qDebug() << QString("(3)TA.c.constData()=0x%1").arg((uint)TA.c.constData(), 8, 16, QChar('0'));
    qDebug() << QString("(4)TB.c.constData()=0x%1").arg((uint)TB.c.constData(), 8, 16, QChar('0'));
}

#include <QtCore>

struct test{
    int value_a;
    unsigned short value_b;
    QByteArray c;
};

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    QString P;
    test TA;
    TA.value_a = 5;
    TA.value_b = 12;
    TA.c = "AB";
    test TB = TA;
    qDebug() << QString("(1)TA.c.constData()=0x%1").arg((uint)TA.c.constData(), 8, 16, QChar('0'));
    qDebug() << QString("(2)TB.c.constData()=0x%1").arg((uint)TB.c.constData(), 8, 16, QChar('0'));
    TB.c = "CD";
    qDebug() << QString("(3)TA.c.constData()=0x%1").arg((uint)TA.c.constData(), 8, 16, QChar('0'));
    qDebug() << QString("(4)TB.c.constData()=0x%1").arg((uint)TB.c.constData(), 8, 16, QChar('0'));
}

To copy to clipboard, switch view to plain text mode 

"(1)TA.c.constData()=0x003d5308" 
"(2)TB.c.constData()=0x003d5308" 
"(3)TA.c.constData()=0x003d5308" 
"(4)TB.c.constData()=0x003d5338"

"(1)TA.c.constData()=0x003d5308" 
"(2)TB.c.constData()=0x003d5308" 
"(3)TA.c.constData()=0x003d5308" 
"(4)TB.c.constData()=0x003d5338"

To copy to clipboard, switch view to plain text mode 

Qt uses copy-on-write. When you copy QByteArray located in some sturuct, QByteArray::constData() points to the same address (seemingly shallow copy).

However, when you modify (=write to) the (seemingly shallow-copied) QByteArray, it is then copied (detached) from the original one.

COW classes are efficient because copying them involves (basically) copying of internal data pointer (with reference counting); nevertheless from the programmers' point of view, copied data can be modified without affecting the original copy. COW automatically take care of `detaching' (copying internal data when needed), so that you can think of this `seemingly shallow copy' as `efficient deep copy'.

COW is used widely in Qt; so that you can pass QString, QByteArray, QImage, etc. efficiently like below:

QString a=b;
QImage a=b;

QString a=b;
QImage a=b;

To copy to clipboard, switch view to plain text mode 

As long as you don't modify, internal data (structure) is shared intelligentlly.
The example below would be ok, because this involves passing shared (reference counting) pointer of internal (bitmap) data from the function to caller.

QImage createImage()
{
    QImage result;
    ....
    return result;
}

QImage createImage()
{
    QImage result;
    ....
    return result;
}

To copy to clipboard, switch view to plain text mode 

[thtran]

Try this.

struct test{
int value_a;
unsigned short value_b;
char c[1]; // not this is not pointer any more. it is just a place holder now
};

int size = sizeof (struct test) + 2 * sizeof(char);


struct test *ptr = (struct test *)new char[size];

QByteArray bytearry(ptr, size);
....

Good Luck.
Tom T

[ChrisW67]

So, you joined the forum, resurrected a three year-old thread that had been thoroughly beaten to death the first and second time, and made your first post adding only marginal information. Was there some reason?